Re: [core] CBOR Encoding of Data Modeled with YANG
Andy Bierman <andy@yumaworks.com> Wed, 09 December 2015 17:43 UTC
Return-Path: <andy@yumaworks.com>
X-Original-To: core@ietfa.amsl.com
Delivered-To: core@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 167EB1B29EE for <core@ietfa.amsl.com>; Wed, 9 Dec 2015 09:43:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.876
X-Spam-Level:
X-Spam-Status: No, score=0.876 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FRT_BELOW2=2.154, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6WO6sLT6r0Z5 for <core@ietfa.amsl.com>; Wed, 9 Dec 2015 09:43:07 -0800 (PST)
Received: from mail-lf0-x232.google.com (mail-lf0-x232.google.com [IPv6:2a00:1450:4010:c07::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6B4A71B2AA0 for <core@ietf.org>; Wed, 9 Dec 2015 09:42:53 -0800 (PST)
Received: by lffu14 with SMTP id u14so39712387lff.1 for <core@ietf.org>; Wed, 09 Dec 2015 09:42:51 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yumaworks-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=WwUq9XcZRuQDN3mZmDhI7lPGSkHLRj5rLYVnZ/UsUP4=; b=kNGzWK8yoLLRcD/t6MqGyDCB49NC21WjEZCDw6p4j03SB6tUi7sPN33HnN/Ex7hZof oC7bdM6LI2d/EczPja0grkZASyKhGfYMCO+uFNuHmesD02milr0U/ivCac7eJR1RqS97 Qnu9iddpiS0PQUlegjwf7CNlI3+s7doxn9/9/2wiOnch+7DR+H6kj+AXBINbMajjzEQO PDtyPvdqnnyjS+eKH0sTYkYg4cgVX5tZ08Xj3yWLa3QsvKRhvErBVx/exKrsa09SHbBD T0RElB9ee/o+fvSsvseB5jkIvp8mRCALb5mAbjKXIDySCwT//oJRGAKBihX9Xn3/iD36 Fm3g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=WwUq9XcZRuQDN3mZmDhI7lPGSkHLRj5rLYVnZ/UsUP4=; b=eoo/K/Poeyqkkk5paeVyHYHG7B9xBvugm39YCFJKv2pxcFLHZ82hHoxNx8Z8UZLu1O RJo0OYcJbEGWSaODhOtBrrUvxiimeKz0S2orvl6wUb5jD/5KtHaVFJu1peBJdOFKNzwP 9xV+HQ+PV1h8omLuG/WJwJS89oKiRyGXzr+8mscWDbh1AD8dq/55XCFx/vt+rELtV3W+ 4CNIblux7Z0JEDmigeJpQL+sAOmxUypEmmgFGU7UzS+h/Lt92xBt1U6OzN1MCCw+XxZR 59J+UL/EkdeL+7Nd+V59X0dePqFeGu3dhZ+rLnLlRrTXMmHCPz3K2cyWPT2/IYrOBGVM h4Sw==
X-Gm-Message-State: ALoCoQm9G2EhW61fOEult7WxId5VFG9fmCH1TpWSvajmQjF9l6eY4N/If7ig3YKmQ60+aEnHuNHtyYN11M9aWS7V1hvb9cbCHw==
MIME-Version: 1.0
X-Received: by 10.25.35.194 with SMTP id j185mr3165469lfj.62.1449682971436; Wed, 09 Dec 2015 09:42:51 -0800 (PST)
Received: by 10.112.144.36 with HTTP; Wed, 9 Dec 2015 09:42:51 -0800 (PST)
In-Reply-To: <m2lh94oz11.fsf@birdie.labs.nic.cz>
References: <BLUPR06MB176391F16B5E9D6CCC531771FE0E0@BLUPR06MB1763.namprd06.prod.outlook.com> <20151207091044.GA59864@elstar.local> <BLUPR06MB1763ABFE8DE1E0E18F5F06A5FE090@BLUPR06MB1763.namprd06.prod.outlook.com> <20151207194229.GA61491@elstar.local> <BLUPR06MB1763B7F9EBF812C06DB04252FE090@BLUPR06MB1763.namprd06.prod.outlook.com> <20151207203826.GA61647@elstar.local> <BLUPR06MB1763E9AB0D1E4A0478D47C05FE090@BLUPR06MB1763.namprd06.prod.outlook.com> <20151208093350.GA62650@elstar.local> <5666AE1A.2040908@ackl.io> <m28u55lz0g.fsf@birdie.labs.nic.cz> <CABCOCHSJ1NdkO77HbCdvCDwGzmY4Ee1pYhGeNX7FNVwgB6Wt6g@mail.gmail.com> <F9C6FCA7-7AB8-463D-9313-DDAF2AD62A07@nic.cz> <CABCOCHT3K2BjPxkXEpK_paagfX348eGz_+GDk-ajR4zC_z7SoA@mail.gmail.com> <m2lh94oz11.fsf@birdie.labs.nic.cz>
Date: Wed, 09 Dec 2015 09:42:51 -0800
Message-ID: <CABCOCHTExn8WwUCXzYXzYeBWaP6Q8pEqE=hrGkLnG21BqGowDA@mail.gmail.com>
From: Andy Bierman <andy@yumaworks.com>
To: Ladislav Lhotka <lhotka@nic.cz>
Content-Type: multipart/alternative; boundary="001a113a9f048ccf7605267a9eeb"
Archived-At: <http://mailarchive.ietf.org/arch/msg/core/bQW2AQpRuUEuP7zsdeXZ6vkFdk8>
Cc: Core <core@ietf.org>
Subject: Re: [core] CBOR Encoding of Data Modeled with YANG
X-BeenThere: core@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Constrained RESTful Environments \(CoRE\) Working Group list" <core.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/core>, <mailto:core-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/core/>
List-Post: <mailto:core@ietf.org>
List-Help: <mailto:core-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/core>, <mailto:core-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 09 Dec 2015 17:43:11 -0000
On Tue, Dec 8, 2015 at 11:35 PM, Ladislav Lhotka <lhotka@nic.cz> wrote: > Andy Bierman <andy@yumaworks.com> writes: > > > On Tue, Dec 8, 2015 at 9:49 AM, Ladislav Lhotka <lhotka@nic.cz> wrote: > > > >> > >> > On 08 Dec 2015, at 17:29, Andy Bierman <andy@yumaworks.com> wrote: > >> > > >> > > >> > > >> > On Tue, Dec 8, 2015 at 7:51 AM, Ladislav Lhotka <lhotka@nic.cz> > wrote: > >> > Alexander Pelov <a@ackl.io> writes: > >> > > >> > > Dear Juergen, > >> > > > >> > > The algorithm is straightforward - assign in increasing order a data > >> > > node ID to each element on the schema tree. If you have a revision > of a > >> > > module, you do the diff between the old and the new tree, and add > the > >> > > new data node IDs at the end of the numbering. > >> > > >> > IMO this is not going to work: sibling data nodes do not have any > >> > fixed order. Different tools may print the tree in different ways, and > >> > it is also possible that consecutive revisions of a module > >> > have the same definitions of sibling data nodes ordered differently. > >> > > >> > I'd suggest to use a hash of the schema node identifier as the numeric > >> ID. > >> > > >> > > >> > This is what the CoMI draft does. > >> > The instance-identifier encoding in your YANG-to-JSON draft is used > >> > (except there will be no predicates in a schema-node identifier) > >> > >> Instance identifiers also don't include non-data nodes such as "choice" > or > >> "case". I am not sure whether they would needed here or not. > >> > > > > > > I don't see why they are needed. > > The object identifier hash is for nodes that can appear in an instance > > document. > > The nice predictable aspect of YANG Hash is that it never > > depends on any other objects except the one being hashed. > > The quantity and order of other objects has no affect on the algorithm. > > > > > > > >> > >> > > >> > > https://tools.ietf.org/html/draft-ietf-netmod-yang-json-06#section-6.11 > >> > > >> > The JSON instance-identifier is needed instead of XML because YANG > >> prefixes can change > >> > over time (as clarified in YANG 1.1). Your encoding only relies on > >> module names > >> > which are never allowed to change, and it is also simpler to code. > >> > >> There is still some uncertainty due to module revisions that cannot be > >> specified this way. > >> > >> > > What uncertainty? > > Why would the /path/from/root/to/node/foo change from 1 release to the > next? > > This is not allowed in YANG. > > Yes, but there may be new nodes unknown to a user of an old revision. > > But this is exactly the same implementation detail whether the new node is a string, a hash, or an assigned number. IMO, only permanent IDs are useful here, so a hash that included the module revision date would not help. > > > > >> Lada > >> > > > > Andy > > > Andy > > > >> > >> > > >> > > >> > Lada > >> > > >> > > >> > Andy > >> > > >> > > > >> > > If you want a short version that would allow you to play with this > >> > > numbering, try the following: > >> > > > >> > > Data Node ID generation for a module YANG_MODULE: > >> > > *# pyang -f tree YANG_MODULE.yang | grep '+' | grep -n '+' > >> > > > >> > > *pyang -f tree YANG_MODULE-OLD-VERSION.yang >> OLD-VERSION.tree > >> > > pyang -f tree YANG_MODULE-NEW-VERSION.yang >> NEW-VERSION.tree > >> > > *# ( cat **OLD-VERSION.tree ; diff **OLD-VERSION.tree > >> **NEW-VERSION.tree > >> > > ) | grep '+' | grep -n '+'* > >> > > * > >> > > *I'm currently working on a pyang version of the algorithm. There > is no > >> > > problem on that point. We cannot produce all documents in the same > >> time, > >> > > and the starting point is YANG-CBOR mapping. * > >> > > > >> > > *Best, > >> > > Alexander > >> > > > >> > > > >> > > Le 08/12/2015 10:33, Juergen Schoenwaelder a écrit : > >> > >> Hi Michel, > >> > >> > >> > >> I am looking for a precise description of the _algorithm_, I am > not so > >> > >> much looking for an example. Has someone implemented this numbering > >> > >> algorithm in pyang and verified that the algorithm always produces > the > >> > >> same data node ids for all the possible changes I can make to a > YANG > >> > >> model? See section 10 of RFC 6020 as a starting point what is > allowed > >> > >> to change in module revisions. See also section 11 of > >> > >> draft-ietf-netmod-rfc6020bis-08.txt. > >> > >> > >> > >> /js > >> > >> > >> > >> On Mon, Dec 07, 2015 at 09:31:37PM +0000, Michel Veillette wrote: > >> > >>> Hi Juergen > >> > >>> > >> > >>> The algorithm proposed to generate IDs is described in > >> > >>> > https://tools.ietf.org/html/draft-veillette-core-cool-00#section-6. > >> > >>> > >> > >>> - IDs are assigned based on there location in the schema tree. > >> > >>> > >> > >>> - Each type of object (Data node IDs, Notification IDs, > Notification > >> parameter IDs, > >> > >>> Protocol Operation IDs, Input parameter IDs, Output parameter > >> IDs) > >> > >>> have a different namespace. > >> > >>> > >> > >>> This approach help keeping the same IDs when updating a > module. > >> > >>> New objects can be added at the end of each list without > >> affecting the existing IDs. > >> > >>> > >> > >>> - When an object is added within a list or within the schema tree, > >> its ID can be > >> > >>> manually assigned using a YANG extension to avoid braking > >> backward compatibility. > >> > >>> Alternatively, data nodes can be added using the augment > >> statement. > >> > >>> In this case, IDs are associated to a different module ID. > >> > >>> > >> > >>> > >> https://tools.ietf.org/html/draft-veillette-core-cool-00#section-7.3 > show > >> an > >> > >>> example of use of the augment statement, see data node ID 68620 > >> bellow. > >> > >>> > >> > >>> CoAP response: > >> > >>> 2.05 Content Content-Format(application/cbor) > >> > >>> { > >> > >>> 66560 : { > >> > >>> 1 : { > >> > >>> 2 : [ > >> > >>> { > >> > >>> 3 : "eth0", > >> > >>> 4 : "Ethernet adapter Local Area Connection", > >> > >>> 5 : "ethernetCsmacd", > >> > >>> 6 : true, > >> > >>> 68620 : { > >> > >>> 13 : true, > >> > >>> 14 : true, > >> > >>> 15 : 1280, > >> > >>> 16 : [ > >> > >>> { > >> > >>> 17 : "fe80::200:f8ff:fe21:67cf", > >> > >>> 18 : 10 > >> > >>> } > >> > >>> ] > >> > >>> } > >> > >>> } > >> > >>> ] > >> > >>> } > >> > >>> } > >> > >>> } > >> > >>> > >> > >>> > >> > >>> -----Original Message----- > >> > >>> From: Juergen Schoenwaelder [mailto: > >> j.schoenwaelder@jacobs-university.de] > >> > >>> Sent: December-07-15 3:38 PM > >> > >>> To: Michel Veillette <Michel.Veillette@trilliantinc.com> > >> > >>> Cc: Core <core@ietf.org> > >> > >>> Subject: Re: [core] CBOR Encoding of Data Modeled with YANG > >> > >>> > >> > >>> On Mon, Dec 07, 2015 at 08:31:03PM +0000, Michel Veillette wrote: > >> > >>>> Hi Juergen > >> > >>>> > >> > >>>> My "smart" quote are now disabled. > >> > >>>> Thanks to motivate me to turn off this nonsense. > >> > >>>> > >> > >>>> About the format/assignment of the numeric data node IDs, the > >> consensus is to keep them out of the YANG to CBOR mapping draft in > order to > >> make progress. > >> > >>>> > >> > >>> But this does not make sense. The naming must be settled, even if > it > >> is painful. > >> > >>> > >> > >>>> Peoples agree that names encoded as string represents too much > >> overhead to address the needs of constrained devices and constrained > >> networks as defined by RFC 7228. However, there are lots of discussions > >> about how those names can be associated with small IDs encoded as > integers > >> and how small those integers need to be. > >> > >>>> > >> > >>>> The CoOL draft proposes structured IDs based on the following > >> concept: > >> > >>>> > >> > >>>> Only IDs associated with module names are registered, IDs > >> associated to data node identifiers are automatically generated. > >> > >>>> > >> > >>>> e.g. draft-ietf-netmod-yang-json-06 page 5 > >> > >>>> for the member-name "foomod:top" > >> > >>>> the ID associated with the name of the module "foomod" is > >> registered, > >> > >>>> the ID associated with the data node identifier "top" is > >> auto-generated. > >> > >>> How do you auto-generate data node identifiers? Does the algorithm > >> work with module revisions, augmentations, features, ...? > >> > >>> > >> > >>>> To simplify scaling of the registered module names, different > >> approaches have been proposed: > >> > >>>> - Possibility to allocation ranges of IDs (bundle) to developers > >> and SDOs for distributed assignment. > >> > >>>> - Possibility to define a range of private IDs (IDs locally > assigned > >> > >>>> and used, not globally unique, same concept as IPv4 10.x.x.x) > >> > >>>> - Possible to define disjoint registries, implemented using a > >> > >>>> different resource type & default URI path > >> > >>> I need to understand the algorithm for assigning data node > >> identifiers first to see whether this approach makes sense. > >> > >>> > >> > >>> /js > >> > >>> > >> > >>> -- > >> > >>> Juergen Schoenwaelder Jacobs University Bremen gGmbH > >> > >>> Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | > >> Germany > >> > >>> Fax: +49 421 200 3103 <http://www.jacobs-university.de/ > > > >> > > > >> > > _______________________________________________ > >> > > core mailing list > >> > > core@ietf.org > >> > > https://www.ietf.org/mailman/listinfo/core > >> > > >> > -- > >> > Ladislav Lhotka, CZ.NIC Labs > >> > PGP Key ID: E74E8C0C > >> > > >> > _______________________________________________ > >> > core mailing list > >> > core@ietf.org > >> > https://www.ietf.org/mailman/listinfo/core > >> > >> -- > >> Ladislav Lhotka, CZ.NIC Labs > >> PGP Key ID: E74E8C0C > >> > >> > >> > >> > >> > > -- > Ladislav Lhotka, CZ.NIC Labs > PGP Key ID: E74E8C0C >
- [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Juergen Schoenwaelder
- Re: [core] CBOR Encoding of Data Modeled with YANG weigengyu
- Re: [core] CBOR Encoding of Data Modeled with YANG Kepeng Li
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Juergen Schoenwaelder
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Juergen Schoenwaelder
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Juergen Schoenwaelder
- Re: [core] CBOR Encoding of Data Modeled with YANG Alexander Pelov
- Re: [core] CBOR Encoding of Data Modeled with YANG Juergen Schoenwaelder
- Re: [core] CBOR Encoding of Data Modeled with YANG Juergen Schoenwaelder
- Re: [core] CBOR Encoding of Data Modeled with YANG Alexander Pelov
- Re: [core] CBOR Encoding of Data Modeled with YANG Carsten Bormann
- Re: [core] CBOR Encoding of Data Modeled with YANG Juergen Schoenwaelder
- Re: [core] CBOR Encoding of Data Modeled with YANG Alexander Pelov
- Re: [core] CBOR Encoding of Data Modeled with YANG Juergen Schoenwaelder
- Re: [core] CBOR Encoding of Data Modeled with YANG Andy Bierman
- Re: [core] CBOR Encoding of Data Modeled with YANG Alexander Pelov
- Re: [core] CBOR Encoding of Data Modeled with YANG Andy Bierman
- Re: [core] CBOR Encoding of Data Modeled with YANG Alexander Pelov
- Re: [core] CBOR Encoding of Data Modeled with YANG Juergen Schoenwaelder
- Re: [core] CBOR Encoding of Data Modeled with YANG Andy Bierman
- Re: [core] CBOR Encoding of Data Modeled with YANG Ladislav Lhotka
- Re: [core] CBOR Encoding of Data Modeled with YANG Andy Bierman
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Carsten Bormann
- Re: [core] CBOR Encoding of Data Modeled with YANG Ladislav Lhotka
- Re: [core] CBOR Encoding of Data Modeled with YANG Andy Bierman
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Ladislav Lhotka
- Re: [core] CBOR Encoding of Data Modeled with YANG Ladislav Lhotka
- Re: [core] CBOR Encoding of Data Modeled with YANG peter van der Stok
- Re: [core] CBOR Encoding of Data Modeled with YANG Juergen Schoenwaelder
- Re: [core] CBOR Encoding of Data Modeled with YANG peter van der Stok
- Re: [core] CBOR Encoding of Data Modeled with YANG Andy Bierman
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Andy Bierman
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Juergen Schoenwaelder
- Re: [core] CBOR Encoding of Data Modeled with YANG Andy Bierman
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Ladislav Lhotka
- Re: [core] CBOR Encoding of Data Modeled with YANG Carsten Bormann
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Ladislav Lhotka
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Ladislav Lhotka
- Re: [core] CBOR Encoding of Data Modeled with YANG Michel Veillette
- Re: [core] CBOR Encoding of Data Modeled with YANG Andy Bierman
- Re: [core] CBOR Encoding of Data Modeled with YANG peter van der Stok