Re: [netmod] JSON to XML lossy conversion

Vladimir Vassilev <vladimir@lightside-instruments.com> Fri, 17 July 2020 10:03 UTC

Return-Path: <vladimir@lightside-instruments.com>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 08F583A0475 for <netmod@ietfa.amsl.com>; Fri, 17 Jul 2020 03:03:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.901
X-Spam-Level:
X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, MSGID_FROM_MTA_HEADER=0.001, NICE_REPLY_A=-0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=netorgft4991094.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zWJF_rGBLy8v for <netmod@ietfa.amsl.com>; Fri, 17 Jul 2020 03:03:43 -0700 (PDT)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2087.outbound.protection.outlook.com [40.107.22.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 445A43A043A for <netmod@ietf.org>; Fri, 17 Jul 2020 03:03:43 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Oe1ZPjCyWldgCO3vNNngDKbOz47J7UFxKLSxTiZHIFgZPWqCteF+flAwv+hjwr4OSN/wcl1bLgJEx+HWDYrBPDDEr4yRs1w2aNO2owFXsomtT0cj2ZeNtFr4NrLxQQqwZNMx4C2nVUkoZVRqonFckfBdiK8dIlJLXgPTMimQFtAq636CIPJUMSPRPNx1YH2mnk65r7IpJQbBx577gbUHEPgipxb+M0E+CTzzytqhccKfmjtzmtrWeT2gzqEYABHuwca/ubFMj5i3rGAWWapv0BRYo8cdl17g3aOTiT7C6/ov0mI+aNfZlHFGKgo2sCS8Z6eAFpj8/MS/bFw17hjh2g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5NFg1FYeihDVBOsgo3vyp/yNGvjXVCjzFdpRGJpWlIQ=; b=RlPPjiuv9Y0ySJMH8T8J6GPWNlNRsSUKKsg9aAdu3nfVfSgkFEa46Tdy9kHlzf0PSYalx9N0lfxUnRoaDPPfpQpEbikR+vwFqINSnKpuFp91NxeziDb/OY1eUvIjOvGSYhe5Em+/UfHNC7kObHfo35S0yrYYniRrTHnulkTvEEJhxeybhfnO6aCqg4lYSqxjbKWcn3VFdzDRQ82s8L//4GTq0VrwPnmMT6XvJYPiioJ8lrQSDpt3B5vy3OQ2WyLxm56eiZ6A1XlXvjMMvVx4jhDoMt7r8KPyJhBLnzs6eV0claZOf2CpEptlSkbYtyLKapRmfeolw6+cLXwnSJz1Ag==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=lightside-instruments.com; dmarc=pass action=none header.from=lightside-instruments.com; dkim=pass header.d=lightside-instruments.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NETORGFT4991094.onmicrosoft.com; s=selector2-NETORGFT4991094-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5NFg1FYeihDVBOsgo3vyp/yNGvjXVCjzFdpRGJpWlIQ=; b=I2Kh596I7X8u8x2b+2k6wakXgGOBAooF49KpZib2g0p0+yoKSfNONrw8qQidl/6/rDauzL3ZL7795UE1eO2hbDqtYbuWV84xMOJ8fFl0CChLaPD85A1qgOtXHT1LX+oyoBqQdUjOe1RK1PrYfxvQlFpKsmZaeksKIumB6uU6cFE=
Authentication-Results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=lightside-instruments.com;
Received: from AM0PR08MB4084.eurprd08.prod.outlook.com (2603:10a6:208:129::25) by AM4PR0802MB2354.eurprd08.prod.outlook.com (2603:10a6:200:64::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.17; Fri, 17 Jul 2020 10:03:40 +0000
Received: from AM0PR08MB4084.eurprd08.prod.outlook.com ([fe80::118a:b8b0:367d:aa76]) by AM0PR08MB4084.eurprd08.prod.outlook.com ([fe80::118a:b8b0:367d:aa76%3]) with mapi id 15.20.3174.026; Fri, 17 Jul 2020 10:03:40 +0000
To: Ladislav Lhotka <ladislav.lhotka@nic.cz>, "netmod@ietf.org" <netmod@ietf.org>
References: <1da7-5f114c00-115-50d47000@26535751> <2a3fe9be-9319-2f19-e18d-be723379953f@nic.cz>
From: Vladimir Vassilev <vladimir@lightside-instruments.com>
Message-ID: <ec0176fd-7b37-f356-825a-c90e86f8f1c3@lightside-instruments.com>
Date: Fri, 17 Jul 2020 12:03:37 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.3.0
In-Reply-To: <2a3fe9be-9319-2f19-e18d-be723379953f@nic.cz>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-ClientProxiedBy: HE1PR0901CA0051.eurprd09.prod.outlook.com (2603:10a6:3:45::19) To AM0PR08MB4084.eurprd08.prod.outlook.com (2603:10a6:208:129::25)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [IPv6:2001:840:4b0b:1337:c68e:8fff:fef3:82a7] (2001:840:4b0b:1337:c68e:8fff:fef3:82a7) by HE1PR0901CA0051.eurprd09.prod.outlook.com (2603:10a6:3:45::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.17 via Frontend Transport; Fri, 17 Jul 2020 10:03:40 +0000
X-Originating-IP: [2001:840:4b0b:1337:c68e:8fff:fef3:82a7]
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 5911f4a7-aeb3-4459-3d4b-08d82a38ae52
X-MS-TrafficTypeDiagnostic: AM4PR0802MB2354:
X-Microsoft-Antispam-PRVS: <AM4PR0802MB2354B13DDFED859DD58E0AA59B7C0@AM4PR0802MB2354.eurprd08.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: cYvNlAbkER15tf8wcswzPF5g0SdP+5kvTgZhrfzV2AdYkbsYUWqbMFRSCRHe7d9/kRtMOKe6MDwmrCUy53YMSV/HcExA6d0/sSVGshDkl0KflVt+5yLrQM5sE8b4bg8TMno8zlZqlY71KbcDkcHPrThCwUbJe2dq+JcpZQa15Xti+3VbRBoUFzhAs5cWLSwrKqCCGjDCmomxvJZAjewpdVraIe7yOEonjFsZr6X5q4o5eMwk7qBxygcAaJYKAIw8Way8NxRNqHsN7MtmYyE0YXGxvErs9x8BDRQuZr7qMe1tLyiL0rVipGfctWortnx9yThFnY4Sl4vmu/UU9gon90Y0Azp0FRU/SjJ4DM0Pn2sKhYvK2tj0fm0riKntO+7fiAABczrUi9L4V3wSL1JFdxrHNccnQLqujri1PL8xjBmIUQjUDlP4q400+JtNZWiEyjJicQqpj91dtHayjOZAcA==
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR08MB4084.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(136003)(39830400003)(396003)(346002)(366004)(376002)(86362001)(31696002)(966005)(316002)(508600001)(2616005)(6486002)(110136005)(186003)(66476007)(31686004)(8676002)(16526019)(52116002)(36756003)(8936002)(66946007)(83380400001)(5660300002)(66556008)(66574015)(2906002)(6666004)(43740500002); DIR:OUT; SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: +JPGF3u1zwGudBtO0MM2dPRAhGuY5mZCO0qG0/wsBlxqaz+fChrKl26JxjKu/AC+xRxSQW7OQGyKo/qgWNOoGPMW47TwNptQlO2hcCl96gzq65euQEnAOS9W9zAwGKqLPeNj/fdnYWEXGq9Nbk+3W8zecDylYn8uCyQvBRV3tZE8GQA+EYPYJtcyZ+ATtLD1BkfFi+jypsXYPdJTu5lam/6vZMjvgLNfkjADcwDRwEMqePb0oanUUyWyui8/m1tpvTTxXr7qmRdN3VuTa3Hs4NUOM4ZFZBqEmixHoLS73A7sWVz+cms8r0RpLvLwWR1Srfe6WfBOu2xOiCSPVhDrc24GT166/0fJi5S15z0OyximWHbkkNLK+Vwpwp2blSHf9+fqWa3vNxifw6TmlCl1NbROz4tdFEeczIaSovWsvEfb3HkLRwPjoOpnOmnYhsWr//WBbphQQGel1/i/G8czj8He/Igjw/kSgF6tn1Q/7q1DmKQMqsszHA9sIkbR9GOMv0l6j/dsTIdXa9VxV1URWiYcJdw+hnZ5fPBdpWJ7nx4=
X-OriginatorOrg: lightside-instruments.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5911f4a7-aeb3-4459-3d4b-08d82a38ae52
X-MS-Exchange-CrossTenant-AuthSource: AM0PR08MB4084.eurprd08.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jul 2020 10:03:40.5784 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: c0326317-f373-4461-a96f-7946e0abb603
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EDT+fqwAMEBTeecniWmdopwTL4XqhqGcJ0jn7lkRO4q0f/GUZbqtY9T/7xAj7lEEKUjdkPtDzwPdO0N1tvJ9nhk8QKderfNgz3WhuZc7syTvenLlxcl4/geudHStC0qK
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0802MB2354
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/vKKIqnU9r4cy_GUy4d4ACZFW53k>
Subject: Re: [netmod] JSON to XML lossy conversion
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Jul 2020 10:03:45 -0000

On 17/07/2020 09.11, Ladislav Lhotka wrote:

>
> On 17. 07. 20 8:57, Michal Vaško wrote:
>> Hi Carsten,
>> you had an interesting idea to have tools that could warn about these problems (although that is hardly a proper solution) but it is not really possible because the problem may occur whenever there is union with a 'string' and 'int8' - 'int32', 'uint8' - 'uint32', or 'boolean', in any order. Meaning in lots of, if not most, unions. And I have considered only XML and JSON, I have not looked into CBOR, which may make it even worse.
> What you can do is to define a metadata annotation specifying the union
> member for a particular instance, and implement it in your tools. This
> would be a solution independent of instance representation.


IMO this does not solve the problem that XML can not encode all values 
of the value space in certain awkward unions (JSON and CBOR can't do 
that either only the constraints are alternative supersets).

If the XML representation rules in the YANG definition were provided as 
an inherent constraint to the value space of the union type there was 
not going to be ambiguity and there would have been a bug in any 
encoding that does not enforce this exact constraint. This is not 
spelled out in RFC7950 but is a good rule of thumb to avoid unions that 
have value space that can not be represented in XML.

The alternative is to rework the XML representation to not have such 
limitation in a next version of YANG (e.g. adding XML attributes with 
the active member type index and name).

That said I think there is one underestimated feature of the union type 
being entirely mapped to a string also for integer types as it is in the 
current XML encoding definition. Without requirement for resolving which 
member type the data actually is representing but just validating there 
is at least one match allows implementations to encode the data without 
resolving a possibly complex logical relationship. For example a union 
of interface index and leafref to interface name (for whatever reason 
someone would design such a union). You really do not need to know if 
there is an interface named "2" in the candidate configuration when you 
send <edit-config> in XML. With JSON you have to resolve this.

/Vladimir


>
> Lada
>
>> Regards,
>> Michal
>>   
>>>> In my view, if it is a bug, it is in the design of the union type in YANG - there is no general way to signal the actual union member type for an instance.
>>> Right.  The ambiguity is normally not a problem for a type choice (which is just a union of the possible values of all types given), unless what specific alternative was chosen is intended to carry semantics.
>>>
>>>> I believe it is a good thing that some encodings allow for at least partial signalling.
>>>>
>>>> Note that similar issues may arise even more often in CBOR, see e.g. comments for section 5.12 in
>>>>
>>>> https://mailarchive.ietf.org/arch/msg/core/Zrb2yhSSdlouS6PI9qfsA1bsDB4/
>>> In the original YANG (XML-only), everything was represented as a text string, so the ambiguity was the highest we see now.  YANG-JSON and YANG-CBOR provide progressively more disambiguating information, so the interpretation (which chosen alternative) may be different from the one after converting to YANG-XML.
>>>
>>> It gets slightly worse if the non-text type has a conversion to text that is not fully nailed down; I don’t know if that is a problem with the current set of YANG types, but any new type could of course worsen the situation.
>>>
>>> The onus is now on the author of the YANG module to make sure that the varying levels of ambiguity do not cause a problem.  I would be interested to know whether there is any tool support for this.
>>>
>>> Grüße, Carsten
>>>
>>   
>>   
>>