Re: [netmod] JSON to XML lossy conversion

Ladislav Lhotka <ladislav.lhotka@nic.cz> Fri, 17 July 2020 12:12 UTC

Return-Path: <ladislav.lhotka@nic.cz>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7FD203A0AF3 for <netmod@ietfa.amsl.com>; Fri, 17 Jul 2020 05:12:15 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.918
X-Spam-Level:
X-Spam-Status: No, score=-1.918 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, NICE_REPLY_A=-0.001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AJvlHJ1mxkMJ for <netmod@ietfa.amsl.com>; Fri, 17 Jul 2020 05:12:13 -0700 (PDT)
Received: from mail.nic.cz (mail.nic.cz [217.31.204.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BD1213A0AF1 for <netmod@ietf.org>; Fri, 17 Jul 2020 05:12:10 -0700 (PDT)
Received: from [IPv6:2a01:5e0:29:ffff:fc73:fa64:57e6:2115] (unknown [IPv6:2a01:5e0:29:ffff:fc73:fa64:57e6:2115]) by mail.nic.cz (Postfix) with ESMTPSA id D1605140990; Fri, 17 Jul 2020 14:12:06 +0200 (CEST)
To: Vladimir Vassilev <vladimir@lightside-instruments.com>, "netmod@ietf.org" <netmod@ietf.org>
References: <1da7-5f114c00-115-50d47000@26535751> <2a3fe9be-9319-2f19-e18d-be723379953f@nic.cz> <ec0176fd-7b37-f356-825a-c90e86f8f1c3@lightside-instruments.com>
From: Ladislav Lhotka <ladislav.lhotka@nic.cz>
Autocrypt: addr=ladislav.lhotka@nic.cz; keydata= mQINBFhU5JIBEACmGIeLglGrwf6JBcNrcLoO19yAdKcH7V132STGHlhEAgmdzwclgVE7GN7y 5+1ySQ8jhDM80QSQ+XaXsSbHTQl23vcVvSfKNdme11kGEQ7kO/bTbUYxpRRa5dqDjLeEHhol WiNk6nUHqEAExbpEoITiO216Lwxh8gD2+39ZH/UVJgMHoZD1VmrxHSnp9b1SpmDWkbILRc93 u6lADieU67SG5vWqGaBgp8JQAVI3F+ZhcLXHQiQLPliePT3YKfvubWg9NIprts9fv2JAUywv NN4R5gg1oFegOXouzlBySiUNXndzJsj1JAcI4psGA9iGYBhgdILXg+2Kwkok9rrx1gWsqtmb lcFDiJRx52FUohHz9obZ9gkOd4NoV4jatjnu+9HKA8V2T5JUgTCgOWeI2yHoSNFJvKO0ygIg /5ccrB9iRdmJIiA2cAmu5MaHnKxAcQ7eaXqQN+JRcpFUH2ooFHKm0750U3aAZY3an5+a5Njh XddaDDX/7f0a68jaWuoKFn7Mqa2HhNnLJs2Nsq9quLdWEUh59wOgTrp2TZnkAFtG1MGa2dNz M3YX22+jwbnfv/U72fEu33juXfMLWfwzJEzQG47dnpmsArt8fM+atOvTSvkLQDNJkOSVZT8m NR5/OeIYfxhe3FUSrljRQBXN8ioVoiMtcCR8EKXCwCnja20vdQARAQABtChMYWRpc2xhdiBM aG90a2EgPGxhZGlzbGF2Lmxob3RrYUBuaWMuY3o+iQJUBBMBCgA+FiEEtr1TxpCtJF8Aid5X uPkrCKn3bGcFAl3k3q0CGwMFCQecs7wFCwkIBwMFFQoJCAsFFgIDAQACHgECF4AACgkQuPkr CKn3bGeXZA/+IM3zgfkMwQ+rvYMEn1cHoYwA0j8iwGtQp6FQYO/Vj0d6Q07Q5/iUqM0UuniU HN6xC5APrPsxwyeSuJUW1zjGm6JD8U5WinbNJuBU/FMC7XyHgOwwXdlT1bLlhOYnvUrZx0bB QeMLnUinvTHLVFnpUCfFpyKRUKh3kkUJS5AXRz5ZgEqJUr79GSTg46dluXlz9mnPIkwluwYO ydlhtW/+s/xVuCIqRUdhhc30d8K5XB+smiYmHSzePjLGwsP1pHt1e8we57EFNvm7iiwXPCAI cy+PAtCxmq2zsIxsZxtZP4SWbRcj2Z16XTtT+hhNK6SWKhpDuacadpWChXpYc36xtuJ5Ll3C lZqa/6mDRcagJy016pQboW/4JIkFcFhu6ugNwvR1OBom04VPVBxqQRgazqpfkB0jSWWS4rwh l30kwDFW5eWcski3Ja5g2drj0YpUeEkMOQcToxLPXffI6dauKTMh7K6RF++LGCYxOukWt2r+ jk6/pB3rSVztAJDb4kc/EgE7yNwrVQOxkKHfk3rGCXkvijCgU3jS0k5g01/OwENxpEUk0Nln mlLJdb8Bju6Jp2y3ZX97+Szs2tH8M6Mpny213lFgc4EVCOFVtIPsyYfiR9On1IIdRqMx3mQC 5AV+3Wc0o3M7Ku0ZbuhkYLNIhlNrRjlxFRk12FpZwFwznY+5AQ0EWFTnYgEIAM6XJ37HGp7H nzztkWbCW+5zkLzff2LQh7yoH32CGJTK7vnQKOY+JKi16lR1qUDHOUtYfRj17jBNWaB0CY0k qkRWCDXG0pff8VT63Kg/DlS2LLbPQpO2NcjdMhPTTUadnKWeo7h6/mFF7VW4X2sm1NYeNWD5 VQxdPQUbnCJ4Pi23QZRJnBu2ycM9aXa7ERRvpN3i3mAc2UgkJWq4KGjI74cd7ytCsU0RN6RT S0sFE4zCB1r6nXqfHHiousATS6+3GnuIGTtaeR0nXHlIvl2AC91+hswsWaA7YB4dB2/GJ8PN L3d07gA6yR2DoZJZhAmtMUiUzap4jgf07UoTOY80MHkAEQEAAYkDcgQYAQoAJgIbAhYhBLa9 U8aQrSRfAIneV7j5Kwip92xnBQJcLzGrBQkHnLFJAUDAdCAEGQEKAB0WIQQ+THLKBBuTzVYa 3ozuYRDCyLOtngUCWFTnYgAKCRDuYRDCyLOtnhnlB/49PV5puyMGBJXY5XBdaPIKFfC5eBfy EOgMqDfJS9yhV7bhFRzzztX9Z7COod6mxmlogNaDFMk/yFH25DOe2tpjKs6sqa9bwyY3cOFw pp38VdeAq0FxF3qOhjKzF12RVQ4Ipu/ahzzQ6V5reia+onOghw+BrjOsPExJaDbiif5UqL/F yWvERlR46GGbFzK9gEvUo+52Wjx3t4hW2IMZKS58xvzkMkTQtxYHAM++jO7bH18wV603JJC3 mZxq9YKKFHVcxinp9l8yzVxfIjhiklednvb30lggY4VEG6X+BER/lcw1QgSDojjbHY72o+76 rBYZ+pSacY/idn5AkAEkQyxsCRC4+SsIqfdsZ1aAEACjuN1b73KNsdsNZuV/MSdmfC+LkoG2 +20kGWzHHiq1Owc4W3fDRohJThM5xyKt2RAcLZAqzfeef1AuoESTqqR6OheaF3u0HAa+U87E /dfjflBOsGZloNi/0/JcN2B8VJpXBwA55adA4YwNL/i8k/roIZ0Yh4wd8Bv8G4exrjvPZPCG klGNBvIJomaqgb9YDvdN22CW3f0zoCwdTX+XV7YL6IQsZroRViLWZv12UCKO1kVop8IMQolx XGbo4kVkqtVx0A4O+ZqCiG819QGZIAtaKSG6aK1xWxNg16ClbEoCEglPeLiOr1ttibO6emuK 4w+4vA5nu+q5niUZ611f/9OtnNO6w/EZ9jRjiqBbo7esNkzal4LjwZUa0B3poWRz+7B++CYS F0c/qdfqjYmiN4IEbXnC0EFhwOYejTayy/H/nA0yGjxF7xA2Krom8B3YTaOnaZ2lVx0gtQK6 Ih7SSc842yRyBnMoVko8mzcHb/0urD2uWkPI5VfXLF9OJhfe3J1xswtqfOxFtf1cZv6fnQID 4gfsvRP8s8Wow5m3gp1HLSlxeZcc57nK8/nKWaTssdOf6doSw8kLWC5Dcwx2Ld15SsjkHeCO MtOaIxq+Er6vnvcepj76pZZJznS/bMriYPyyw0FIYmAPcnm3v5W+HZ339ktp+Kk3fNl3qJo6 C+Sjv7kBDQRYVOgGAQgA0GW5sWxqhcD9hxi/LQZ/hzPBgrpTvyIgxHrJN1HZ2BzdpNU7UlnC DVDzvAV2/1kEIiCdTYnnly0+PCHhyEdP45M9//oH7KYd/F/JSQwI+56mGP2O0fBBu9WStHws RraGfzGGZDV1XutvIz/qzpkBGpXGzA73Bi2Xye+/9JtHXmjhoXnItQLO38Hqnb5kXk79cSC4 TStPAKdrB/CMolcArT8j9eijIUOPMkWCGCeICv/hECpA5CZma2tplU80cKiOIYknFG8E+5Pr FlW962P7njIaahWSIObM2/C48uu5KAwgyDj6DA14cwc/4wfRsXZFjUMNpI0xer1iLQV5y1vi 6QARAQABiQI8BBgBCgAmAhsMFiEEtr1TxpCtJF8Aid5XuPkrCKn3bGcFAlwvMc8FCQecsMkA CgkQuPkrCKn3bGfkEg//alC3LNxnyH3tNy/OKQqOFJLUCidIm77DT/ZQfjLMAZezzCAq1Adf hVg7uV5a/dFdQdVDnjEz+SQ6/EzyqGashdAMnlBDwiqPe3MhB0L97VQwjNQa0wf1AsHdQCv8 i3e7U6RjlEmSC42dA+RjhrFOCZPtUTlCVDoSnY1S0kZOvEjhHvw901MPEINikZY4TBc18Xnk fbGQhglvI9NqBawlBKHyDGTAVJL7ld60iHsWt3prq+h48iFjc7x2JT39oBxT5Jhl3pg01QSC vu2Fg99vUuRr3u8tavnMKX4Vk3bujpHhuyOTSm0X37nwwu9DpMqkeUNvYR2RM+EuGGltonrN yrb5S21omN7fLDvRtdUBUOCFJLxZunTDLlkig/QuT+QlW7CG59Ussuu37HxgpiICCeeBDOaF bkZku43Qwt5cTtdJ8BkzYgvvoC7NegBn9N+U+Etrrvs09Slo11wlQx1F3sk9T9rjz1QrPM8j blxEgepm+W75VJ22z1qY/JnNElygR7KeTaeOY+iCn4wgDeC964AHMuGzG7aUg8P3zjgClL1T pFJwQUml880L6C+VuWPxuFIS18BPCYtLGlfQc5jp4dGN87ftenT13W70XiGXdrmIAFBF6Jjk 3huuDoRvdbPavAGdxu5QGdHeH0/GjfTxq7lRwgamoY0lw1LIB6LOzXo=
Organization: CZ.NIC
Message-ID: <b86abf02-137c-3806-798b-45d8961f66e7@nic.cz>
Date: Fri, 17 Jul 2020 14:12:06 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <ec0176fd-7b37-f356-825a-c90e86f8f1c3@lightside-instruments.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-Virus-Scanned: clamav-milter 0.102.2 at mail
X-Virus-Status: Clean
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/QVdC3bd4hm7oyKv4TWkilimFivY>
Subject: Re: [netmod] JSON to XML lossy conversion
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Jul 2020 12:12:16 -0000


On 17. 07. 20 12:03, Vladimir Vassilev wrote:
> On 17/07/2020 09.11, Ladislav Lhotka wrote:
> 
>>
>> On 17. 07. 20 8:57, Michal Vaško wrote:
>>> Hi Carsten,
>>> you had an interesting idea to have tools that could warn about these
>>> problems (although that is hardly a proper solution) but it is not
>>> really possible because the problem may occur whenever there is union
>>> with a 'string' and 'int8' - 'int32', 'uint8' - 'uint32', or
>>> 'boolean', in any order. Meaning in lots of, if not most, unions. And
>>> I have considered only XML and JSON, I have not looked into CBOR,
>>> which may make it even worse.
>> What you can do is to define a metadata annotation specifying the union
>> member for a particular instance, and implement it in your tools. This
>> would be a solution independent of instance representation.
> 
> 
> IMO this does not solve the problem that XML can not encode all values
> of the value space in certain awkward unions (JSON and CBOR can't do
> that either only the constraints are alternative supersets).

Why not? In XML, everything is encoded as text, and the annotation tells
how to parse it and interpret. So either the text representation
conforms to that precise type or otherwise it is an error. There is no
ambiguity.

Lada

> 
> If the XML representation rules in the YANG definition were provided as
> an inherent constraint to the value space of the union type there was
> not going to be ambiguity and there would have been a bug in any
> encoding that does not enforce this exact constraint. This is not
> spelled out in RFC7950 but is a good rule of thumb to avoid unions that
> have value space that can not be represented in XML.
> 
> The alternative is to rework the XML representation to not have such
> limitation in a next version of YANG (e.g. adding XML attributes with
> the active member type index and name).
> 
> That said I think there is one underestimated feature of the union type
> being entirely mapped to a string also for integer types as it is in the
> current XML encoding definition. Without requirement for resolving which
> member type the data actually is representing but just validating there
> is at least one match allows implementations to encode the data without
> resolving a possibly complex logical relationship. For example a union
> of interface index and leafref to interface name (for whatever reason
> someone would design such a union). You really do not need to know if
> there is an interface named "2" in the candidate configuration when you
> send <edit-config> in XML. With JSON you have to resolve this.
> 
> /Vladimir
> 
> 
>>
>> Lada
>>
>>> Regards,
>>> Michal
>>>  
>>>>> In my view, if it is a bug, it is in the design of the union type
>>>>> in YANG - there is no general way to signal the actual union member
>>>>> type for an instance.
>>>> Right.  The ambiguity is normally not a problem for a type choice
>>>> (which is just a union of the possible values of all types given),
>>>> unless what specific alternative was chosen is intended to carry
>>>> semantics.
>>>>
>>>>> I believe it is a good thing that some encodings allow for at least
>>>>> partial signalling.
>>>>>
>>>>> Note that similar issues may arise even more often in CBOR, see
>>>>> e.g. comments for section 5.12 in
>>>>>
>>>>> https://mailarchive.ietf.org/arch/msg/core/Zrb2yhSSdlouS6PI9qfsA1bsDB4/
>>>>>
>>>> In the original YANG (XML-only), everything was represented as a
>>>> text string, so the ambiguity was the highest we see now.  YANG-JSON
>>>> and YANG-CBOR provide progressively more disambiguating information,
>>>> so the interpretation (which chosen alternative) may be different
>>>> from the one after converting to YANG-XML.
>>>>
>>>> It gets slightly worse if the non-text type has a conversion to text
>>>> that is not fully nailed down; I don’t know if that is a problem
>>>> with the current set of YANG types, but any new type could of course
>>>> worsen the situation.
>>>>
>>>> The onus is now on the author of the YANG module to make sure that
>>>> the varying levels of ambiguity do not cause a problem.  I would be
>>>> interested to know whether there is any tool support for this.
>>>>
>>>> Grüße, Carsten
>>>>
>>>    

-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67