Re: [netmod] yang next issue #46 binary encoding support

"Rob Wilton (rwilton)" <rwilton@cisco.com> Sun, 31 March 2019 10:32 UTC

Return-Path: <rwilton@cisco.com>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6C686120188 for <netmod@ietfa.amsl.com>; Sun, 31 Mar 2019 03:32:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.501
X-Spam-Level:
X-Spam-Status: No, score=-14.501 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zsoWSkJl6b30 for <netmod@ietfa.amsl.com>; Sun, 31 Mar 2019 03:32:10 -0700 (PDT)
Received: from alln-iport-1.cisco.com (alln-iport-1.cisco.com [173.37.142.88]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A5CE912017B for <netmod@ietf.org>; Sun, 31 Mar 2019 03:32:09 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=37170; q=dns/txt; s=iport; t=1554028329; x=1555237929; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=uMWosl4URUnb+sYmaNBswpgTnELUAv0Qotuxvio5Sb0=; b=f+7ASCjPLb4smU2Juzj5OL4ddceqFGLwsWdXhL/4/JjSj6gnad1rJOHM eTjDhVoWngaC7nDgfVLUkCH5U6V7E5dAL3lvXBETsezl66PGIYFYs/arl 1737AZcHpgUkpr2kiFLUCOgR452Dp32LTa8ngRWky5YTCELMc6cX0GSZy U=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0AFAACJlqBc/5RdJa1gAxkBAQEBAQE?= =?us-ascii?q?BAQEBAQEHAQEBAQEBgVIDAQEBAQELAYEOgQJogQMnCoQElVOJOI8FgXsOAQE?= =?us-ascii?q?YAQqEA0YCF4UgIjUIDQEBAwEBCQEDAm0cDIVKAQEBBAEBIQpBCw4CAgEIDgI?= =?us-ascii?q?BBAEBASAHAwICAhkGBgsUCQgCBAENBQgTgwiBEUwDFQ+oSoEvhDUCg0INgho?= =?us-ascii?q?FBYEqAYsyF4FAP4ERgxI+ghpHAQEDgTcRLQoVEYJDglcDijkGgkGEI4IJkWg?= =?us-ascii?q?2CQKHb4gvgzgilCyLP4YRgTyMAQIRFYEuIAE2gVZwFRohgmyBZjAXg36EYYU?= =?us-ascii?q?/QTGODoEugR8BAQ?=
X-IronPort-AV: E=Sophos;i="5.60,292,1549929600"; d="scan'208,217";a="252290077"
Received: from rcdn-core-12.cisco.com ([173.37.93.148]) by alln-iport-1.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 31 Mar 2019 10:32:08 +0000
Received: from XCH-ALN-007.cisco.com (xch-aln-007.cisco.com [173.36.7.17]) by rcdn-core-12.cisco.com (8.15.2/8.15.2) with ESMTPS id x2VAW7HZ026356 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=FAIL); Sun, 31 Mar 2019 10:32:08 GMT
Received: from xch-rcd-007.cisco.com (173.37.102.17) by XCH-ALN-007.cisco.com (173.36.7.17) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Sun, 31 Mar 2019 05:32:07 -0500
Received: from xch-rcd-007.cisco.com ([173.37.102.17]) by XCH-RCD-007.cisco.com ([173.37.102.17]) with mapi id 15.00.1473.003; Sun, 31 Mar 2019 05:32:07 -0500
From: "Rob Wilton (rwilton)" <rwilton@cisco.com>
To: Mahesh Jethanandani <mjethanandani@gmail.com>, Andy Bierman <andy@yumaworks.com>
CC: NetMod WG <netmod@ietf.org>
Thread-Topic: [netmod] yang next issue #46 binary encoding support
Thread-Index: AQHU5iFVCJpun3mXh0SXqXlmbRjEP6YjGoYAgAAC0YCAAAOdgIAAJgUAgAADHwCAACzfAIACFdEQ
Date: Sun, 31 Mar 2019 10:32:07 +0000
Message-ID: <24cd67f42eab465a90c33ff37ece5919@XCH-RCD-007.cisco.com>
References: <20190329111930.k2dt6wctsazxa7rp@anna.jacobs.jacobs-university.de> <CABCOCHS=VhfpKHYhB_eQ8Y9i5FK6+R1q4a8Soc=z=HRYJLV5OA@mail.gmail.com> <20190329161723.xuh3avyrdepdw3px@anna.jacobs.jacobs-university.de> <CABCOCHS6cNhG_YeeW_ueYMOvo1TQHfpFi8TQGDrka12yoRvZLA@mail.gmail.com> <20190329184624.4sg6lbasv5b5u4hw@anna.jacobs.jacobs-university.de> <CABCOCHR=ZEYFK5ifnsTYnMgmKb+yPkLXZ0+kqoGWzhcEHkhSQg@mail.gmail.com> <CE01FFB0-25B3-442E-B5DB-903065BE742C@gmail.com>
In-Reply-To: <CE01FFB0-25B3-442E-B5DB-903065BE742C@gmail.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.61.93.191]
Content-Type: multipart/alternative; boundary="_000_24cd67f42eab465a90c33ff37ece5919XCHRCD007ciscocom_"
MIME-Version: 1.0
X-Outbound-SMTP-Client: 173.36.7.17, xch-aln-007.cisco.com
X-Outbound-Node: rcdn-core-12.cisco.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/3pZFDFgRut1DH1V9L0bJU9vFvlQ>
Subject: Re: [netmod] yang next issue #46 binary encoding support
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 31 Mar 2019 10:32:13 -0000

I also agree that we should reopen this issue to further discuss any language implications, and add it to the “Further Discuss” bucket.

I suggest that we just do this, unless someone objects.

Thanks,
Rob


From: netmod <netmod-bounces@ietf.org> On Behalf Of Mahesh Jethanandani
Sent: 29 March 2019 21:38
To: Andy Bierman <andy@yumaworks.com>
Cc: NetMod WG <netmod@ietf.org>
Subject: Re: [netmod] yang next issue #46 binary encoding support

Based on this discussion, I think we should reopen and change the title of this issue as “binary encoding in YANG support”, while I open a new issue in netconf-next for “support for binary encoding in NETCONF”.


On Mar 29, 2019, at 11:57 AM, Andy Bierman <andy@yumaworks.com<mailto:andy@yumaworks.com>> wrote:


On Fri, Mar 29, 2019 at 11:46 AM Juergen Schoenwaelder <j.schoenwaelder@jacobs-university.de<mailto:j.schoenwaelder@jacobs-university.de>> wrote:
On Fri, Mar 29, 2019 at 09:30:19AM -0700, Andy Bierman wrote:
> On Fri, Mar 29, 2019 at 9:17 AM Juergen Schoenwaelder <
> j.schoenwaelder@jacobs-university.de<mailto:j.schoenwaelder@jacobs-university.de>> wrote:
>
> > On Fri, Mar 29, 2019 at 09:07:18AM -0700, Andy Bierman wrote:
> > > On Fri, Mar 29, 2019 at 4:19 AM Juergen Schoenwaelder <
> > > j.schoenwaelder@jacobs-university.de<mailto:j.schoenwaelder@jacobs-university.de>> wrote:
> > >
> > > > Hi,
> > > >
> > > > this is issue is closed but I wonder whether this is correct. I have
> > > > several questions looking at the issue on github:
> > > >
> > > > - Why is this not a YANG issue?
> > > > - Which workaround is better?
> > > > - Why is this tagged as a NETCONF issue?
> > > >
> > > >
> > > Did you mean this should be NETCONF issue?
> > > It is more of a protocol problem then a modeling problem.
> > > The goal is to use the model unaltered.
> >
> > I think it would be valuable if say the definition of ipv4-address
> > could state that a canonical binary representation is of type binary {
> > length 4; }. Doing this is only meaningful for some types but it would
> > allow to add more binary representations over time.
> >
> > > > If we want to support binary encodings, we need to allow modelers to
> > > > define which types map to a canonical binary representation in
> > > > addition to the canonical string representation. As stated in the
> > > > issue description, hard-wiring some types in the encoding
> > > > specifications is very limited.
> > > >
> > > > In terms of backwards compatibility, this issue should IMHO be tagged
> > > > high (adding binary encoding for some types does not cause any
> > > > backwards compatibility problem since so far we have only strings).
> > > >
> > > >
> > > Not so sure.
> > > The base64 encoding could look like a valid string.
> > > The receiver must know a binary type is being sent (XML and JSON both
> > fail
> > > here, but not CBOR).
> >
> > I am talking about CBOR, not about XML or JSON. I want to provide
> > hints to CBOR (or similar binary encodings) that values can be
> > represented in a different format. I do not expect these hints to be
> > used by XML or JSON. If you need binary encoding efficiency, use CBOR
> > instead of JSON.
> >
> > > > While I do not have a solution proposal, I think this issue is worth
> > > > to look at and we should not close it right now.
> > > >
> > > >
> > > I have a solution proposal, but I have not implemented it yet, so it it
> > not
> > > detailed...
> > >
> > > Both sender and receiver need to agree on the binary encoding and how the
> > > data is tagged as binary.
> > >
> > > This expired draft should address that problem:
> > > https://tools.ietf..org/html/draft-mahesh-netconf-binary-encoding-01<https://tools.ietf.org/html/draft-mahesh-netconf-binary-encoding-01>
> > >
> > > For every type T that they agree on, there are standard T.b2y() and
> > T.y2b()
> > > conversion functions.
> > > There are also some extensions to define conversion templates so vendors
> > > can add their own types.
> > >
> > > The YANG modules do not need to actually be altered.  The peers will
> > > negotiate the
> > > set of types that will be sent as binary when the session starts.
> > > The receiver knows T and the SID for each object, and will accept either
> > > the YANG or binary encoding.
> >
> > Sounds complex for me to negotiate this. I rather say once that a
> > binary encoding can ship an IPv6 address as type binary { length 16; }
> > and then CBOR will simply do the right thing.
> >
> >
> OK, but this would require new type names.
> You cannot simply change some standard type to be a union with a binary
> type.
>
> This forces all implementations of that type to support the binary variant.
> That breaks old clients that worked with the version before the binary
> variant.
>
> The ripple effect on the models changing types would be non-trivial.
> Using this union-type approach forces every protocol to support the binary
> encoding,
> yet base64 in a union with strings is very error-prone.
>

I am not proposing do change the type definitions we have. My idea was
to have an optional additional definition for binary encodings. Here
is an ad-hoc example (I do not like the details of the syntax, but
perhaps this helps to understand the idea):

     typedef ipv4-address {
       type string {
         pattern
           '(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\.){3}'
         +  '([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])';
       }
       description
         "The ipv4-address type represents an IPv4 address in
          dotted-quad notation.";

       binary-representation {
         type binary {
           length 4;
         }
         description
           "The binary representation uses as 4-byte binary string
            in network byte ordering.";
       }
     }

The CBOR encoder (or other binary encoders) would then encode the
value as a 4 byte binary value, the XML and JSON encoder would use the
canonical string representation.  If the binary-representation is not
specified, then the generic CBOR encoding rules apply. I assume that
additional binary representation definitions will only be needed for a
couple of types (and I might even be fine to restrict that to
typedefs). Anyway, details need work, but if we want to support more
efficient binary encodings, then I think we should keep the issue #46
open.


OK -- this is what I had in mind but off to the side, like a deviations module.
If the client and server agree on the module containing the standard extension usages
it will not be that complex in the protocol.

  ex:binary-representation ietf-inet-types:ipv4-address {
     ex:binary-length 4;
     ex:binary-pattern "b0.b1.b2.b3";
  }

I agree YANG 1.2 should have real statements instead of extensions.



/js

Andy


--
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
netmod@ietf.org<mailto:netmod@ietf.org>
https://www.ietf.org/mailman/listinfo/netmod

Mahesh Jethanandani
mjethanandani@gmail.com<mailto:mjethanandani@gmail.com>