Re: [netmod] yang next issue #46 binary encoding support

[Mahesh Jethanandani <[e-mail]>]

Based on this discussion, I think we should reopen and change the title of this issue as “binary encoding in YANG support”, while I open a new issue in netconf-next for “support for binary encoding in NETCONF”.

> On Mar 29, 2019, at 11:57 AM, Andy Bierman <[e-mail]> wrote:
> 
> 
> 
> On Fri, Mar 29, 2019 at 11:46 AM Juergen Schoenwaelder <[e-mail] <mailto:[e-mail]>> wrote:
> On Fri, Mar 29, 2019 at 09:30:19AM -0700, Andy Bierman wrote:
> > On Fri, Mar 29, 2019 at 9:17 AM Juergen Schoenwaelder <
> > [e-mail] <mailto:[e-mail]>> wrote:
> > 
> > > On Fri, Mar 29, 2019 at 09:07:18AM -0700, Andy Bierman wrote:
> > > > On Fri, Mar 29, 2019 at 4:19 AM Juergen Schoenwaelder <
> > > > [e-mail] <mailto:[e-mail]>> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > this is issue is closed but I wonder whether this is correct. I have
> > > > > several questions looking at the issue on github:
> > > > >
> > > > > - Why is this not a YANG issue?
> > > > > - Which workaround is better?
> > > > > - Why is this tagged as a NETCONF issue?
> > > > >
> > > > >
> > > > Did you mean this should be NETCONF issue?
> > > > It is more of a protocol problem then a modeling problem.
> > > > The goal is to use the model unaltered.
> > >
> > > I think it would be valuable if say the definition of ipv4-address
> > > could state that a canonical binary representation is of type binary {
> > > length 4; }. Doing this is only meaningful for some types but it would
> > > allow to add more binary representations over time.
> > >
> > > > > If we want to support binary encodings, we need to allow modelers to
> > > > > define which types map to a canonical binary representation in
> > > > > addition to the canonical string representation. As stated in the
> > > > > issue description, hard-wiring some types in the encoding
> > > > > specifications is very limited.
> > > > >
> > > > > In terms of backwards compatibility, this issue should IMHO be tagged
> > > > > high (adding binary encoding for some types does not cause any
> > > > > backwards compatibility problem since so far we have only strings).
> > > > >
> > > > >
> > > > Not so sure.
> > > > The base64 encoding could look like a valid string.
> > > > The receiver must know a binary type is being sent (XML and JSON both
> > > fail
> > > > here, but not CBOR).
> > >
> > > I am talking about CBOR, not about XML or JSON. I want to provide
> > > hints to CBOR (or similar binary encodings) that values can be
> > > represented in a different format. I do not expect these hints to be
> > > used by XML or JSON. If you need binary encoding efficiency, use CBOR
> > > instead of JSON.
> > >
> > > > > While I do not have a solution proposal, I think this issue is worth
> > > > > to look at and we should not close it right now.
> > > > >
> > > > >
> > > > I have a solution proposal, but I have not implemented it yet, so it it
> > > not
> > > > detailed...
> > > >
> > > > Both sender and receiver need to agree on the binary encoding and how the
> > > > data is tagged as binary.
> > > >
> > > > This expired draft should address that problem:
> > > > https://tools.ietf..org/html/draft-mahesh-netconf-binary-encoding-01 <https://tools.ietf.org/html/draft-mahesh-netconf-binary-encoding-01>
> > > >
> > > > For every type T that they agree on, there are standard T.b2y() and
> > > T.y2b()
> > > > conversion functions.
> > > > There are also some extensions to define conversion templates so vendors
> > > > can add their own types.
> > > >
> > > > The YANG modules do not need to actually be altered.  The peers will
> > > > negotiate the
> > > > set of types that will be sent as binary when the session starts.
> > > > The receiver knows T and the SID for each object, and will accept either
> > > > the YANG or binary encoding.
> > >
> > > Sounds complex for me to negotiate this. I rather say once that a
> > > binary encoding can ship an IPv6 address as type binary { length 16; }
> > > and then CBOR will simply do the right thing.
> > >
> > >
> > OK, but this would require new type names.
> > You cannot simply change some standard type to be a union with a binary
> > type.
> >
> > This forces all implementations of that type to support the binary variant.
> > That breaks old clients that worked with the version before the binary
> > variant.
> > 
> > The ripple effect on the models changing types would be non-trivial.
> > Using this union-type approach forces every protocol to support the binary
> > encoding,
> > yet base64 in a union with strings is very error-prone.
> > 
> 
> I am not proposing do change the type definitions we have. My idea was
> to have an optional additional definition for binary encodings. Here
> is an ad-hoc example (I do not like the details of the syntax, but
> perhaps this helps to understand the idea):
> 
>      typedef ipv4-address {
>        type string {
>          pattern
>            '(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\.){3}'
>          +  '([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])';
>        }
>        description
>          "The ipv4-address type represents an IPv4 address in
>           dotted-quad notation.";
> 
>        binary-representation {
>          type binary {
>            length 4;
>          }
>          description
>            "The binary representation uses as 4-byte binary string
>             in network byte ordering.";
>        }
>      }
> 
> The CBOR encoder (or other binary encoders) would then encode the
> value as a 4 byte binary value, the XML and JSON encoder would use the
> canonical string representation.  If the binary-representation is not
> specified, then the generic CBOR encoding rules apply. I assume that
> additional binary representation definitions will only be needed for a
> couple of types (and I might even be fine to restrict that to
> typedefs). Anyway, details need work, but if we want to support more
> efficient binary encodings, then I think we should keep the issue #46
> open.
> 
> 
> 
> OK -- this is what I had in mind but off to the side, like a deviations module.
> If the client and server agree on the module containing the standard extension usages
> it will not be that complex in the protocol.
> 
>   ex:binary-representation ietf-inet-types:ipv4-address {
>      ex:binary-length 4;
>      ex:binary-pattern "b0.b1.b2.b3";
>   }
> 
> I agree YANG 1.2 should have real statements instead of extensions.
> 
> 
>  
> /js
> 
> 
> Andy
> 
>  
> -- 
> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> Phone: [phone]         Campus Ring 1 | 28759 Bremen | Germany
> Fax:   [phone]         <https://www.jacobs-university.de/ <https://www.jacobs-university.de/>>
> _______________________________________________
> netmod mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/netmod <https://www.ietf.org/mailman/listinfo/netmod>
Mahesh Jethanandani
[e-mail]