Re: [core] CBOR Encoding of Data Modeled with YANG

Andy Bierman <andy@yumaworks.com> Wed, 09 December 2015 17:43 UTC

Return-Path: <andy@yumaworks.com>
X-Original-To: core@ietfa.amsl.com
Delivered-To: core@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 167EB1B29EE for <core@ietfa.amsl.com>; Wed, 9 Dec 2015 09:43:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.876
X-Spam-Level:
X-Spam-Status: No, score=0.876 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FRT_BELOW2=2.154, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6WO6sLT6r0Z5 for <core@ietfa.amsl.com>; Wed, 9 Dec 2015 09:43:07 -0800 (PST)
Received: from mail-lf0-x232.google.com (mail-lf0-x232.google.com [IPv6:2a00:1450:4010:c07::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6B4A71B2AA0 for <core@ietf.org>; Wed, 9 Dec 2015 09:42:53 -0800 (PST)
Received: by lffu14 with SMTP id u14so39712387lff.1 for <core@ietf.org>; Wed, 09 Dec 2015 09:42:51 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yumaworks-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=WwUq9XcZRuQDN3mZmDhI7lPGSkHLRj5rLYVnZ/UsUP4=; b=kNGzWK8yoLLRcD/t6MqGyDCB49NC21WjEZCDw6p4j03SB6tUi7sPN33HnN/Ex7hZof oC7bdM6LI2d/EczPja0grkZASyKhGfYMCO+uFNuHmesD02milr0U/ivCac7eJR1RqS97 Qnu9iddpiS0PQUlegjwf7CNlI3+s7doxn9/9/2wiOnch+7DR+H6kj+AXBINbMajjzEQO PDtyPvdqnnyjS+eKH0sTYkYg4cgVX5tZ08Xj3yWLa3QsvKRhvErBVx/exKrsa09SHbBD T0RElB9ee/o+fvSsvseB5jkIvp8mRCALb5mAbjKXIDySCwT//oJRGAKBihX9Xn3/iD36 Fm3g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=WwUq9XcZRuQDN3mZmDhI7lPGSkHLRj5rLYVnZ/UsUP4=; b=eoo/K/Poeyqkkk5paeVyHYHG7B9xBvugm39YCFJKv2pxcFLHZ82hHoxNx8Z8UZLu1O RJo0OYcJbEGWSaODhOtBrrUvxiimeKz0S2orvl6wUb5jD/5KtHaVFJu1peBJdOFKNzwP 9xV+HQ+PV1h8omLuG/WJwJS89oKiRyGXzr+8mscWDbh1AD8dq/55XCFx/vt+rELtV3W+ 4CNIblux7Z0JEDmigeJpQL+sAOmxUypEmmgFGU7UzS+h/Lt92xBt1U6OzN1MCCw+XxZR 59J+UL/EkdeL+7Nd+V59X0dePqFeGu3dhZ+rLnLlRrTXMmHCPz3K2cyWPT2/IYrOBGVM h4Sw==
X-Gm-Message-State: ALoCoQm9G2EhW61fOEult7WxId5VFG9fmCH1TpWSvajmQjF9l6eY4N/If7ig3YKmQ60+aEnHuNHtyYN11M9aWS7V1hvb9cbCHw==
MIME-Version: 1.0
X-Received: by 10.25.35.194 with SMTP id j185mr3165469lfj.62.1449682971436; Wed, 09 Dec 2015 09:42:51 -0800 (PST)
Received: by 10.112.144.36 with HTTP; Wed, 9 Dec 2015 09:42:51 -0800 (PST)
In-Reply-To: <m2lh94oz11.fsf@birdie.labs.nic.cz>
References: <BLUPR06MB176391F16B5E9D6CCC531771FE0E0@BLUPR06MB1763.namprd06.prod.outlook.com> <20151207091044.GA59864@elstar.local> <BLUPR06MB1763ABFE8DE1E0E18F5F06A5FE090@BLUPR06MB1763.namprd06.prod.outlook.com> <20151207194229.GA61491@elstar.local> <BLUPR06MB1763B7F9EBF812C06DB04252FE090@BLUPR06MB1763.namprd06.prod.outlook.com> <20151207203826.GA61647@elstar.local> <BLUPR06MB1763E9AB0D1E4A0478D47C05FE090@BLUPR06MB1763.namprd06.prod.outlook.com> <20151208093350.GA62650@elstar.local> <5666AE1A.2040908@ackl.io> <m28u55lz0g.fsf@birdie.labs.nic.cz> <CABCOCHSJ1NdkO77HbCdvCDwGzmY4Ee1pYhGeNX7FNVwgB6Wt6g@mail.gmail.com> <F9C6FCA7-7AB8-463D-9313-DDAF2AD62A07@nic.cz> <CABCOCHT3K2BjPxkXEpK_paagfX348eGz_+GDk-ajR4zC_z7SoA@mail.gmail.com> <m2lh94oz11.fsf@birdie.labs.nic.cz>
Date: Wed, 09 Dec 2015 09:42:51 -0800
Message-ID: <CABCOCHTExn8WwUCXzYXzYeBWaP6Q8pEqE=hrGkLnG21BqGowDA@mail.gmail.com>
From: Andy Bierman <andy@yumaworks.com>
To: Ladislav Lhotka <lhotka@nic.cz>
Content-Type: multipart/alternative; boundary="001a113a9f048ccf7605267a9eeb"
Archived-At: <http://mailarchive.ietf.org/arch/msg/core/bQW2AQpRuUEuP7zsdeXZ6vkFdk8>
Cc: Core <core@ietf.org>
Subject: Re: [core] CBOR Encoding of Data Modeled with YANG
X-BeenThere: core@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Constrained RESTful Environments \(CoRE\) Working Group list" <core.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/core>, <mailto:core-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/core/>
List-Post: <mailto:core@ietf.org>
List-Help: <mailto:core-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/core>, <mailto:core-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 09 Dec 2015 17:43:11 -0000

On Tue, Dec 8, 2015 at 11:35 PM, Ladislav Lhotka <lhotka@nic.cz> wrote:

> Andy Bierman <andy@yumaworks.com> writes:
>
> > On Tue, Dec 8, 2015 at 9:49 AM, Ladislav Lhotka <lhotka@nic.cz> wrote:
> >
> >>
> >> > On 08 Dec 2015, at 17:29, Andy Bierman <andy@yumaworks.com> wrote:
> >> >
> >> >
> >> >
> >> > On Tue, Dec 8, 2015 at 7:51 AM, Ladislav Lhotka <lhotka@nic.cz>
> wrote:
> >> > Alexander Pelov <a@ackl.io> writes:
> >> >
> >> > > Dear Juergen,
> >> > >
> >> > > The algorithm is straightforward - assign in increasing order a data
> >> > > node ID to each element on the schema tree. If you have a revision
> of a
> >> > > module, you do the diff between the old and the new tree, and add
> the
> >> > > new data node IDs at the end of the numbering.
> >> >
> >> > IMO this is not going to work: sibling data nodes do not have any
> >> > fixed order. Different tools may print the tree in different ways, and
> >> > it is also possible that consecutive revisions of a module
> >> > have the same definitions of sibling data nodes ordered differently.
> >> >
> >> > I'd suggest to use a hash of the schema node identifier as the numeric
> >> ID.
> >> >
> >> >
> >> > This is what the CoMI draft does.
> >> > The instance-identifier encoding in your YANG-to-JSON draft is used
> >> > (except there will be no predicates in a schema-node identifier)
> >>
> >> Instance identifiers also don't include non-data nodes such as "choice"
> or
> >> "case". I am not sure whether they would needed here or not.
> >>
> >
> >
> > I don't see why they are needed.
> > The object identifier hash is for nodes that can appear in an instance
> > document.
> > The nice predictable aspect of YANG Hash is that it never
> > depends on any other objects except the one being hashed.
> > The quantity and order of other objects has no affect on the algorithm.
> >
> >
> >
> >>
> >> >
> >> >
> https://tools.ietf.org/html/draft-ietf-netmod-yang-json-06#section-6.11
> >> >
> >> > The JSON instance-identifier is needed instead of XML because YANG
> >> prefixes can change
> >> > over time (as clarified in YANG 1.1).  Your encoding only relies on
> >> module names
> >> > which are never allowed to change, and it is also simpler to code.
> >>
> >> There is still some uncertainty due to module revisions that cannot be
> >> specified this way.
> >>
> >>
> > What uncertainty?
> > Why would the /path/from/root/to/node/foo change from 1 release to the
> next?
> > This is not allowed in YANG.
>
> Yes, but there may be new nodes unknown to a user of an old revision.
>
>

But this is exactly the same implementation detail whether the new node is
a string,
a hash, or an assigned number. IMO, only permanent IDs are useful
here, so a hash that included the module revision date would not help.



>
> >
> >> Lada
> >>
> >
> > Andy
> >
>

Andy


> >
> >>
> >> >
> >> >
> >> > Lada
> >> >
> >> >
> >> > Andy
> >> >
> >> > >
> >> > > If you want a short version that would allow you to play with this
> >> > > numbering, try the following:
> >> > >
> >> > > Data Node ID generation for a module YANG_MODULE:
> >> > > *# pyang -f tree YANG_MODULE.yang | grep '+' | grep -n '+'
> >> > >
> >> > > *pyang -f tree YANG_MODULE-OLD-VERSION.yang >> OLD-VERSION.tree
> >> > > pyang -f tree YANG_MODULE-NEW-VERSION.yang >> NEW-VERSION.tree
> >> > > *# ( cat **OLD-VERSION.tree ; diff **OLD-VERSION.tree
> >> **NEW-VERSION.tree
> >> > > ) | grep '+' | grep -n '+'*
> >> > > *
> >> > > *I'm currently working on a pyang version of the algorithm. There
> is no
> >> > > problem on that point. We cannot produce all documents in the same
> >> time,
> >> > > and the starting point is YANG-CBOR mapping. *
> >> > >
> >> > > *Best,
> >> > > Alexander
> >> > >
> >> > >
> >> > > Le 08/12/2015 10:33, Juergen Schoenwaelder a écrit :
> >> > >> Hi Michel,
> >> > >>
> >> > >> I am looking for a precise description of the _algorithm_, I am
> not so
> >> > >> much looking for an example. Has someone implemented this numbering
> >> > >> algorithm in pyang and verified that the algorithm always produces
> the
> >> > >> same data node ids for all the possible changes I can make to a
> YANG
> >> > >> model? See section 10 of RFC 6020 as a starting point what is
> allowed
> >> > >> to change in module revisions. See also section 11 of
> >> > >> draft-ietf-netmod-rfc6020bis-08.txt.
> >> > >>
> >> > >> /js
> >> > >>
> >> > >> On Mon, Dec 07, 2015 at 09:31:37PM +0000, Michel Veillette wrote:
> >> > >>> Hi Juergen
> >> > >>>
> >> > >>> The algorithm proposed to generate IDs  is described in
> >> > >>>
> https://tools.ietf.org/html/draft-veillette-core-cool-00#section-6.
> >> > >>>
> >> > >>> - IDs are assigned based on there location in the schema tree.
> >> > >>>
> >> > >>> - Each type of object (Data node IDs, Notification IDs,
> Notification
> >> parameter IDs,
> >> > >>>     Protocol Operation IDs, Input parameter IDs, Output parameter
> >> IDs)
> >> > >>>     have a different namespace.
> >> > >>>
> >> > >>>     This approach help keeping the same IDs  when updating a
> module.
> >> > >>>     New objects can be added at the end of each list without
> >> affecting the existing IDs.
> >> > >>>
> >> > >>> - When an object is added within a list or within the schema tree,
> >> its ID can be
> >> > >>>     manually assigned using a YANG extension to avoid braking
> >> backward compatibility.
> >> > >>>     Alternatively, data nodes can be added using the augment
> >> statement.
> >> > >>>     In this case, IDs are associated to a different module ID.
> >> > >>>
> >> > >>>
> >> https://tools.ietf.org/html/draft-veillette-core-cool-00#section-7.3
> show
> >> an
> >> > >>>    example of use of the augment statement, see data node ID 68620
> >> bellow.
> >> > >>>
> >> > >>>     CoAP response:
> >> > >>>       2.05 Content Content-Format(application/cbor)
> >> > >>>       {
> >> > >>>         66560 : {
> >> > >>>           1 : {
> >> > >>>             2 : [
> >> > >>>               {
> >> > >>>                 3 : "eth0",
> >> > >>>                 4 : "Ethernet adapter Local Area Connection",
> >> > >>>                 5 : "ethernetCsmacd",
> >> > >>>                 6 : true,
> >> > >>>                 68620 : {
> >> > >>>                   13 : true,
> >> > >>>                   14 : true,
> >> > >>>                   15 : 1280,
> >> > >>>                   16 : [
> >> > >>>                     {
> >> > >>>                       17 : "fe80::200:f8ff:fe21:67cf",
> >> > >>>                       18 : 10
> >> > >>>                     }
> >> > >>>                   ]
> >> > >>>                 }
> >> > >>>               }
> >> > >>>             ]
> >> > >>>           }
> >> > >>>         }
> >> > >>>       }
> >> > >>>
> >> > >>>
> >> > >>> -----Original Message-----
> >> > >>> From: Juergen Schoenwaelder [mailto:
> >> j.schoenwaelder@jacobs-university.de]
> >> > >>> Sent: December-07-15 3:38 PM
> >> > >>> To: Michel Veillette <Michel.Veillette@trilliantinc.com>
> >> > >>> Cc: Core <core@ietf.org>
> >> > >>> Subject: Re: [core] CBOR Encoding of Data Modeled with YANG
> >> > >>>
> >> > >>> On Mon, Dec 07, 2015 at 08:31:03PM +0000, Michel Veillette wrote:
> >> > >>>> Hi Juergen
> >> > >>>>
> >> > >>>> My "smart" quote are now disabled.
> >> > >>>> Thanks to motivate me to turn off this nonsense.
> >> > >>>>
> >> > >>>> About the format/assignment of the numeric data node IDs, the
> >> consensus is to keep them out of the YANG to CBOR mapping draft in
> order to
> >> make progress.
> >> > >>>>
> >> > >>> But this does not make sense. The naming must be settled, even if
> it
> >> is painful.
> >> > >>>
> >> > >>>> Peoples agree that names encoded as string represents too much
> >> overhead to address the needs of constrained devices and constrained
> >> networks as defined by RFC 7228. However, there are lots of discussions
> >> about how those names can be associated with small IDs encoded as
> integers
> >> and how small those integers need to be.
> >> > >>>>
> >> > >>>> The CoOL draft proposes structured IDs based on the following
> >> concept:
> >> > >>>>
> >> > >>>>     Only IDs  associated with module names are registered, IDs
> >> associated to data node identifiers are automatically generated.
> >> > >>>>
> >> > >>>>     e.g. draft-ietf-netmod-yang-json-06 page 5
> >> > >>>>     for the member-name "foomod:top"
> >> > >>>>     the ID associated with the name of the module "foomod" is
> >> registered,
> >> > >>>>     the ID associated with the data node identifier "top" is
> >> auto-generated.
> >> > >>> How do you auto-generate data node identifiers? Does the algorithm
> >> work with module revisions, augmentations, features, ...?
> >> > >>>
> >> > >>>> To simplify scaling of the registered module names, different
> >> approaches have been proposed:
> >> > >>>> - Possibility to allocation ranges of IDs (bundle) to developers
> >> and SDOs for distributed assignment.
> >> > >>>> - Possibility to define a range of private IDs (IDs locally
> assigned
> >> > >>>> and used, not globally unique, same concept as IPv4 10.x.x.x)
> >> > >>>> - Possible to define disjoint registries, implemented using a
> >> > >>>> different resource type & default URI path
> >> > >>> I need to understand the algorithm for assigning data node
> >> identifiers first to see whether this approach makes sense.
> >> > >>>
> >> > >>> /js
> >> > >>>
> >> > >>> --
> >> > >>> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> >> > >>> Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen |
> >> Germany
> >> > >>> Fax:   +49 421 200 3103         <http://www.jacobs-university.de/
> >
> >> > >
> >> > > _______________________________________________
> >> > > core mailing list
> >> > > core@ietf.org
> >> > > https://www.ietf.org/mailman/listinfo/core
> >> >
> >> > --
> >> > Ladislav Lhotka, CZ.NIC Labs
> >> > PGP Key ID: E74E8C0C
> >> >
> >> > _______________________________________________
> >> > core mailing list
> >> > core@ietf.org
> >> > https://www.ietf.org/mailman/listinfo/core
> >>
> >> --
> >> Ladislav Lhotka, CZ.NIC Labs
> >> PGP Key ID: E74E8C0C
> >>
> >>
> >>
> >>
> >>
>
> --
> Ladislav Lhotka, CZ.NIC Labs
> PGP Key ID: E74E8C0C
>