Re: [core] YANG to CBOR mapping version 1

Michel Veillette <Michel.Veillette@trilliantinc.com> Thu, 14 July 2016 11:37 UTC

Return-Path: <Michel.Veillette@trilliantinc.com>
X-Original-To: core@ietfa.amsl.com
Delivered-To: core@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 80DF212D753 for <core@ietfa.amsl.com>; Thu, 14 Jul 2016 04:37:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.922
X-Spam-Level:
X-Spam-Status: No, score=-1.922 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=trilliant.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ev2Ff8BiJkYZ for <core@ietfa.amsl.com>; Thu, 14 Jul 2016 04:37:30 -0700 (PDT)
Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0136.outbound.protection.outlook.com [104.47.36.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3973812D526 for <core@ietf.org>; Thu, 14 Jul 2016 04:37:30 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Trilliant.onmicrosoft.com; s=selector1-trilliantinc-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=kwSaYvEYhl/blOk5Zr/MWpghRppRTi6aDPNCkB5eefE=; b=qLmXfpqpG7ULY5F9jwt1Yl4pToLLxK/UWWHovcXNNTr9D3q2cPDT+N8ghs0NCXlnwkHQVlk2gnl24ipsSab343cqblE4xLkEsA0Iy0ZkZIEdCumsWbvhha6rJo09bfbQqCMSKqeeSUU46xHPeR9pb7nWw/SB0YDqaAhDkjPMHGo=
Received: from BLUPR06MB1763.namprd06.prod.outlook.com (10.162.224.149) by BLUPR06MB1764.namprd06.prod.outlook.com (10.162.224.150) with Microsoft SMTP Server (TLS) id 15.1.539.14; Thu, 14 Jul 2016 11:37:28 +0000
Received: from BLUPR06MB1763.namprd06.prod.outlook.com ([10.162.224.149]) by BLUPR06MB1763.namprd06.prod.outlook.com ([10.162.224.149]) with mapi id 15.01.0539.019; Thu, 14 Jul 2016 11:37:28 +0000
From: Michel Veillette <Michel.Veillette@trilliantinc.com>
To: "consultancy@vanderstok.org" <consultancy@vanderstok.org>
Thread-Topic: [core] YANG to CBOR mapping version 1
Thread-Index: AQHR1qK2U/a50X7m60OAxt73Sipz+qAOmnXwgAYCugCAAFEisIABBZSAgACEZ/CAACjkgIAAAPZAgAD7JwCAADlmwA==
Date: Thu, 14 Jul 2016 11:37:28 +0000
Message-ID: <BLUPR06MB176328C63F415719D140489BFE320@BLUPR06MB1763.namprd06.prod.outlook.com>
References: <1ed9a5d16aca7d13412259e94afc1aa2@xs4all.nl> <BLUPR06MB17639739570760661AEC32C3FE3C0@BLUPR06MB1763.namprd06.prod.outlook.com> <e18f0f45bfae89fc55df444fdd957550@xs4all.nl> <BLUPR06MB17638EB4CCB4235B5101D540FE300@BLUPR06MB1763.namprd06.prod.outlook.com> <d58ca58bd8f89b9856a2e23290a18ba8@xs4all.nl> <BLUPR06MB1763A212CD1C3386722B88D9FE310@BLUPR06MB1763.namprd06.prod.outlook.com> <8aa9950d6b8ff0cfe8968276d18cad7d@xs4all.nl> <BLUPR06MB1763D6B8ADAE036D5E63FBA3FE310@BLUPR06MB1763.namprd06.prod.outlook.com> <43805298839959a534d83c720ac6239c@xs4all.nl>
In-Reply-To: <43805298839959a534d83c720ac6239c@xs4all.nl>
Accept-Language: fr-CA, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=Michel.Veillette@trilliantinc.com;
x-originating-ip: [24.225.215.88]
x-ms-office365-filtering-correlation-id: 7dd11be8-36d3-447c-9c00-08d3abdb3c18
x-microsoft-exchange-diagnostics: 1; BLUPR06MB1764; 6:4CkK2CIu3CcwTxDQ4mSNUsqVHJWeEzdp1p21sFq1QpNqSQOwMPRXT6VGUb0FzRaFsnl5Y5rig0vI7OpGMtEcVRoEb9pC/Ug8IG0iu3U/oIHdj7eTcoZQ6bYLke5iEK9BHE6VizeD5TFXIDGjw9JyLkyjjw0K/P6QjSG5C6AivtFUiZQ0N15RuG8V5r/8Wm/v9JvJPjFWJzcFuGQZhOPQtgNdwudpTvC9lFD3rhOwbG1A5v/r7rm6YO0840O9FBt2dA5HCO8Goi5PU9Xm2gGzWs2l38hgBkGdBdxn3m2gm2U=; 5:ape3BbGBS6QwFw/Z5FJAT1EjADGec8LzNxgGOpHwL6yeh+NfZPdVPSZLJvCNv8J6Qav2A/YMCdZUJJEelUu7R8SM0i0FEhHvIYf2iBnyfjtwHi7oliFv4eaAXBfyPHMmMe2jRUiPKI8SHvPviY/NZg==; 24:E7gzgJIY9gEoXNg6EPwKp/8djDIf508KZx0fdLmlPvuwvzQ7QgjaIteI9WKdCz6ZV7SOb2ajineeUlP7NE+YywIlTZWfc8d2tsZYStw/J3Q=; 7:6jUf85R1avzpOuznMcK2aU8HbEMBJz8JUClh+Gyq2om3z1nX5d2TT5H5kUkMU01LcLn9cFqXaEBLaEFt+aAZfHt4JdzH5CRVz6NS9o5ieZ+upL1Kd02Crs7/+3UX/KNpHad08RrU9erCOrMSlVD95c0NowxJt5kI++RGejFo8pPvVRIvj2/IM26YgXQKoMG/HCna2Kigqmks6jmsWku0s6HahQVnQRK1IN+21ztp+zDLtDXAUlwJrsKasGRuzhoG
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR06MB1764;
x-microsoft-antispam-prvs: <BLUPR06MB17645256F30785B9CFF853C6FE320@BLUPR06MB1764.namprd06.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(158342451672863)(120809045254105);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001); SRVR:BLUPR06MB1764; BCL:0; PCL:0; RULEID:; SRVR:BLUPR06MB1764;
x-forefront-prvs: 00032065B2
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(189002)(377424004)(377454003)(13464003)(199003)(10400500002)(33656002)(7696003)(1730700003)(87936001)(8676002)(8936002)(105586002)(86362001)(11100500001)(66066001)(7846002)(81166006)(81156014)(76176999)(189998001)(110136002)(97736004)(7736002)(5003600100003)(74316002)(68736007)(50986999)(19580405001)(4326007)(305945005)(6116002)(2501003)(5002640100001)(2906002)(15975445007)(99286002)(101416001)(3846002)(102836003)(2950100001)(586003)(2900100001)(19580395003)(76576001)(77096005)(106116001)(92566002)(93886004)(54356999)(3660700001)(2351001)(106356001)(3280700002)(9686002)(1720100001)(122556002)(5640700001); DIR:OUT; SFP:1102; SCL:1; SRVR:BLUPR06MB1764; H:BLUPR06MB1763.namprd06.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en;
received-spf: None (protection.outlook.com: trilliantinc.com does not designate permitted sender hosts)
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: trilliantinc.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Jul 2016 11:37:28.2443 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4f6fbd13-0dfb-4150-85c3-d43260c04309
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR06MB1764
Archived-At: <https://mailarchive.ietf.org/arch/msg/core/8zFmRAHwOc8fYzuevsc50azg-cc>
Cc: Core <core@ietf.org>
Subject: Re: [core] YANG to CBOR mapping version 1
X-BeenThere: core@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "Constrained RESTful Environments \(CoRE\) Working Group list" <core.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/core>, <mailto:core-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/core/>
List-Post: <mailto:core@ietf.org>
List-Help: <mailto:core-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/core>, <mailto:core-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Jul 2016 11:37:32 -0000

Hi Peter

Validation mechanisms are defined by YANG  ('key', 'unique', 'range', 'pattern', 'length', 'min-elements', 'max-elements', 
'must', 'when', 'if-feature'), not by CoOL or CoMI. 
Any protocol based on YANG need to be able to support those mechanisms.
Adding meta data in the encoding to identity which data nodes are keys don't eliminate de need for clients and servers to have some knowledge of the schema.

The general rule established for "draft-ietf-core-yang-cbor" is simple.
" In order to minimize the size of the encoded data, the proposed
   mapping avoid any unnecessary meta-information beyond those natively
   supported by CBOR."
https://tools.ietf.org/html/draft-ietf-core-yang-cbor-02#section-3

In this case, you propose to add more meta data than "draft-ietf-netmod-yang-json"
which don't share the same limitations on the payload size.
https://tools.ietf.org/html/draft-ietf-netmod-yang-json-10#section-5.4

Regards,
Michel


-----Original Message-----

From: peter van der Stok [mailto:stokcons@xs4all.nl] 
Sent: Thursday, July 14, 2016 3:57 AM
To: Michel Veillette <Michel.Veillette@trilliantinc.com>
Cc: consultancy@vanderstok.org; Core <core@ietf.org>
Subject: RE: [core] YANG to CBOR mapping version 1

Hi Michel,

IMO, an error is an error and should be treated as such. Trying to salvage unhealthy situations leads to worse problems.

My approach would be that servers and clients are simple and don't need to parse the yang modules during operation; and keep extensive state information about each other.

The discussion seems to confirm my growing conviction that the CoMI and CoOL approaches are fundamentally different.

Peter

Michel Veillette schreef op 2016-07-13 19:06:
> Hi Peter
> 
> I'm not convinced that Including in the encoding which data nodes are 
> the keys completely solve the issue.
> If the client transmit a payload with one key and the server required 
> two keys, the payload will be rejected.
> If the client transmit a payload with two keys (marked as such) and 
> the server require one key, the payload will be rejected.
> To work, both sides need to known the number of keys used by the other 
> side and only 'ietf-cool-library' can resolve that.
> 
> Also, not making which data nodes are keys might help during the 
> migration.
> 
> Regards,
> Michel
> 
> -----Original Message-----
> From: peter van der Stok [mailto:stokcons@xs4all.nl]
> Sent: Wednesday, July 13, 2016 12:55 PM
> To: Michel Veillette <Michel.Veillette@trilliantinc.com>
> Cc: consultancy@vanderstok.org; Core <core@ietf.org>
> Subject: RE: [core] YANG to CBOR mapping version 1
> 
> Hi Michel,
> 
> That looks pretty complex to me; while the keys can be elegantly 
> expressed in the CBOR contents.
> 
> Peter
> 
> Michel Veillette schreef op 2016-07-13 16:36:
>> Hi Peter
>> 
>> The scenario you described is part of the generic discovery problem.
>> To perform its tasks, clients typically need to known the list of 
>> modules, features, deviations implemented by a server and associated 
>> versions.
>> This problem is addressed in
>> http://core-wg.github.io/yang-cbor/draft-veillette-core-cool-library-
>> l
>> atest.html
>> which is the constrained version of
>> https://datatracker.ietf.org/doc/rfc7895/
>> 
>> module: ietf-cool-library
>>    +--ro modules-state
>>       +--ro module-set-id    union
>>       +--ro module* [sid revision]
>>          +--ro sid                 sid
>>          +--ro revision            revision
>>          +--ro feature*            sid
>>          +--ro deviation* [sid revision]
>>          |  +--ro sid         sid
>>          |  +--ro revision    revision
>>          +--ro conformance-type    enumeration
>>          +--ro submodules
>>             +--ro submodule* [sid revision]
>>                +--ro sid         sid
>>                +--ro revision    revision
>> notifications:
>>    +---n yang-library-change
>>       +--ro module-set-id    -> /modules-state/module-set-id
>> 
>> Regards
>> Michel
>> 
>> -----Original Message-----
>> From: peter van der Stok [mailto:stokcons@xs4all.nl]
>> Sent: Wednesday, July 13, 2016 2:34 AM
>> To: Michel Veillette <Michel.Veillette@trilliantinc.com>
>> Cc: consultancy@vanderstok.org; Core <core@ietf.org>
>> Subject: RE: [core] YANG to CBOR mapping version 1
>> 
>> Hi Michel,
>> 
>> When a specification changes it set of of keys, I don't think the 
>> SIDs will change.
>> It is for sure that the hashes are not affected by the attribution of 
>> a key.
>> 
>> Therefore, what happens when client and server use different keys in 
>> otherwise identical list specification?
>> 
>> Peter
>> 
>> Michel Veillette schreef op 2016-07-12 17:07:
>>> Hi Peter
>>> 
>>> The validations performed by the entity which de-serialize the CBOR 
>>> payload is based on the YANG schema.
>>> This validation is based on all king of YANG statements such as:
>>> key, unique, range, pattern, length, min-elements, max-elements, 
>>> must, when, if-feature
>>> 
>>> The identification of which data nodes are keys is considered 
>>> unnecessary meta-data since available in the schema.
>>> Even more heavy weight representations such as xml and json don't 
>>> include this meta-data in the encode payload.
>>> 
>>> Regards,
>>> Michel
>>> 
>>> -----Original Message-----
>>> From: peter van der Stok [mailto:stokcons@xs4all.nl]
>>> Sent: Tuesday, July 12, 2016 6:08 AM
>>> To: Michel Veillette <Michel.Veillette@trilliantinc.com>
>>> Cc: consultancy@vanderstok.org; Core <core@ietf.org>
>>> Subject: RE: [core] YANG to CBOR mapping version 1
>>> 
>>> Hi Michel,
>>> 
>>>> 
>>>> However, I am missing how to transport a given list instance with 
>>>> its key values in the CBOR encoding. Section 4.4 describes the 
>>>> encoding of a complete list not a subset of its instances.
>>>> 
>>>> [MV] A list instance is a collection which is described in section 
>>>> 4.2.
>>>> ___________________________________________________________________
>>>> _
>>>> _
>>>> _
>>>> _______________________
>>> 
>>> With the current text I don't see how you distinguish instances with 
>>> the same key from instances with different keys.
>>> 
>>> Example:
>>> 
>>> list server {
>>>       key name;
>>> 
>>>       leaf name {
>>>         type string;
>>>       }
>>>       leaf iburst {
>>>         type boolean;
>>>         default false;
>>>     }
>>> 
>>> How to distinguish that in the list below that two instances are 
>>> identical (should not occur)
>>>     [
>>>       {
>>>         1755 : "NRC TIC server",          # name (SID 1755)
>>>         1754 : false,                     # iburst (SID 1754)
>>>       },
>>>       {
>>>         1755 : "NRC TIC server",          # name (SID 1755)
>>>         1754 : true,                     # iburst (SID 1754)
>>>       }
>>>     ]
>>> 
>>> And the following two instances are different  (valid array)
>>> 
>>>     [
>>>       {
>>>         1755 : "NRC TAC server",          # name (SID 1755)
>>>         1754 : true,                      # iburst (SID 1754)
>>>       },
>>>       {
>>>         1755 : "NRC TIC server",          # name (SID 1755)
>>>         1754 : true,                     # iburst (SID 1754)
>>>       }
>>>     ]