Re: [netconf] Issues with draft-ietf-netconf-udp-notif-04

[Andy Bierman <[e-mail]>]

On Sat, Nov 20, 2021 at 10:14 AM <[e-mail]> wrote:

> Hi Andy,
>
>
>
> Looking at the list of media types I disagree with your observation.
> Please correct me if I am wrong, only a small subset is applicable to YANG
> push.
>
>
>
> The possible encodings are currently being defined in both transport drafts
>
>
> https://datatracker.ietf.org/doc/html/draft-ietf-netconf-https-notif#section-8.3
>
>
> https://datatracker.ietf.org/doc/html/draft-ietf-netconf-udp-notif-04#section-3.2
>
>
>




It is not clear if this is supposed to be an application protocol or not.
Supposedly, the header cannot have a subscription-id because the
application layer defines that.
But this document defines the encoding type of the application payload,
called "the notification message" in 3.1

Perhaps 3 complete examples showing an XML, JSON, and CBOR encoding of "the
notification message"
would help clarify the intended usage of these standard ET values.


Andy

However, you are raising a very valid point I noticed that the netconf
working group haven't addressed clearly yet. I understood from
previous threads with draft-ietf-netconf-https-notif and
draft-ietf-netconf-udp-notif that multiple times questions were raised
which are the valid transport encodings for YANG push.
>
>
>
> For me it is unclear wherever CBOR could be used as encoding in draft-ietf-netconf-https-notif or not. Wherever draft-ietf-netconf-https-notif is encoding agnostic or not. I assume both transport drafts are encoding agnostic. Therefor we as working group should address the topic, which are the supported encodings for YANG push at present time, an how in the future when new encodings are being developed, we address this topic.
>
>
>
> My suggestion is to keep the initial registries for both drafts equal. Defining JSON, XML and CBOR as the initial encoding. And if in the future new encodings are being added, ensure that both registries, for udp-notif and https-notif are being updated equally.
>
>
>
> From my point of view, we can roughly foresee that YANG push supports never more than 16 encoding types. However, due to the late YANG pus transport standardization at IETF and already existing non standardized YANG push encoding implementations,  I understood that private encodings is something we can't ignore. Therefore it makes sense to support it in the initial version of the transport documents.
>
>
>
> Best wishes
>
> Thomas
>
>
>
> *From:* netconf <[e-mail]> *On Behalf Of *Andy Bierman
> *Sent:* Friday, November 19, 2021 3:13 AM
> *To:* Pierre Francois <[e-mail]>
> *Cc:* Netconf <[e-mail]>
> *Subject:* Re: [netconf] Issues with draft-ietf-netconf-udp-notif-04
>
>
>
> Hi,
>
>
>
> It is unclear why this document needs to reinvent content types using a
> 4-bit ET field.
>
>
>
> There is already a registry of integer mappings for CoAP
>
>
> https://www.iana.org/assignments/core-parameters/core-parameters.xhtml#content-formats
> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.iana.org%2Fassignments%2Fcore-parameters%2Fcore-parameters.xhtml%23content-formats&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408346521%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=7WSsymSFBxP1W5U4iro6cg8I61dHc23YgfWIucH0KJ8%3D&reserved=0>
>
>
>
> IMO this draft needs to (at least) define a header option for a 16-bit
> "CF" field.
>
> Implementations SHOULD provide a CF option if none of the standard ET
> types apply.
>
> One of the 16 standard ET values should mean "See CF option".
>
>
>
>
>
> Andy
>
>
>
>
>
>
>
>
>
>
>
> On Wed, Nov 17, 2021 at 11:31 AM Andy Bierman <[e-mail]> wrote:
>
>
>
>
>
> On Wed, Nov 17, 2021 at 3:03 AM Pierre Francois <
> [e-mail]> wrote:
>
> Hello Andy,
>
>
>
> Thanks for the feedback, answers inline.
>
>
>
> Le mar. 16 nov. 2021 à 04:45, Andy Bierman <[e-mail]> a écrit :
>
> Hi,
>
>
>
> I am wondering if there implementations of this draft already.
>
> IMO there are problems with in that prevent reasonable and
>
> optimal implementations.
>
>
>
> There is support of udp-notif publisher in VRP.
>
> There is a C collector lib for udp-notif in
> https://github.com/insa-unyte/udp-notif-c-collector
> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Finsa-unyte%2Fudp-notif-c-collector&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408346521%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=3q8mNKLzfYKxqsR3ii0flAXmGhskv8RUN8ekduTJhZs%3D&reserved=0>
>
>
>
>
>
>
> 1) Use of UDP as the Application Message Framing
>
> UDP is a transport protocol.  Tightly coupling one
>
> NETCONF notification message to one UDP frame (+ fragments)
>
> is not very optimal.
>
>
>
> There is a segmentation option in the protocol, so a udp notif message is
> not tightly coupled to a UDP frame.
>
>
>
> But this segmentation is optional to implement.
>
> My point is that application protocols like NETCONF have their own message
> framing
>
> which does not exist here because the transport framing is used instead.
>
>
>
>
>
>
>
> 1A) wastes space in the UDP frame if the NETCONF notification
>
>     happens to be short
>
>
>
> I don't get it, I'm sorry. To me a UDP header + small udp-notif header is
> rather short.
>
>
>
> If I can fit 100 notification messages in 1 UDP frame then I save 99% of
> the redundant headers.
>
>
>
>
>
>
>
> 1B) imposes an application-level max-message-size
>
>
>
> Not with the segmentation option, do we agree?
>
>
>
>
> https://datatracker.ietf.org/doc/html/draft-ietf-netconf-udp-notif#section-4.1
> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-netconf-udp-notif%23section-4.1&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408356481%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Xl%2BMQemsr4GneiC%2FUyWmC9bBruBPgHtZwLPBuvLeiWw%3D&reserved=0>
>
>
>
> There can be 32767 segments of a max 65535 each: about 2G of data.
>
> However the segment-size is allowed to be smaller and segmentation is
> completely optional-to-implement.
>
>
>
> Application layer framing allows unlimited message size without any need
>
> to support UDP segmentation at all,
>
>
>
>
>
>
>
>
>
> 2) Not Efficient for Streaming Servers (no Message Chunking)
>
> The application framing in NETCONF and RESTCONF (HTTP) is not
>
> coupled to any transport protocol. Instead, a sender may emit as
>
> many message chunks as desired. This is how a streaming server works.
>
> Content comes from many different sources and it is sent on-the-fly
>
> in chunks, instead of constructing large replies.
>
>
>
>     - Application framing allows multiple notifications in one UDP frame
>
>     - Application framing does not need a max-message-size.
>
>       Streaming servers often do not know the size of the data that will
> be sent
>
>       in a notification.
>
>
>
> I think we meet these goals. Let me think about this further.
>
>
>
>
>
> I agree, by ignoring the idea that this protocol provides any normative
> information
>
> about the application layer protocol in use.  This draft should remove any
> hint
>
> that the application payload is defined in this document.  It is OK to say
>
> a different RFC (or spec) defines the content for a particular Encoding
> Type.
>
>
>
> The "notification" in Figure 1 implies that the payload is one NETCONF
> notification.
>
> The draft should be clear that there is no structure to the payload
> defined in this document.
>
>
>
>
>
>
>
> Q1) Should proprietary encoding be used to support application-layer
> framing
>
>        or should the standard support it?
>
>
>
> I think we'd need to meet on this or else it will end up with thousands of
> long emails ;)
>
>
>
>
>
> I think the draft should let other documents define the exact content
> expected in the payload.
>
> I am used to application protocols defining specific message flows. Since
> this draft does
>
> not do that, it should not attempt to impose any rules on the application
> payload at all.
>
>
>
>
>
>
>
>
>
> 3) Protocol Message Encoding for CBOR
>
> The NETCONF notification message is not modelled in YANG.
>
> This is an open issue for YANG to CBOR encoding.
>
> https://github.com/core-wg/yang-cbor/issues/96
> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcore-wg%2Fyang-cbor%2Fissues%2F96&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408356481%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GychPbAURCdRp1pdWoQfPnaFo3gK3Al5AA8rdcQKt0E%3D&reserved=0>
>
>
>
> The "notification" and "eventTime" elements do not have YANG definitions.
>
> SID assignments for these elements will be needed.
>
> This draft is the likely place for these SID assignment requests to
>
> the IESG and IANA.
>
>
>
> Shouldn't these be transport agnostic? I don't see why this would be bound
> to udp-notif only and not, say, https-notif.
>
>
>
> This is related to application/yang-data+cbor.
>
> A separate document for the SID assignments can be done.
>
>
>
>
>
>
>
>
>
> 4) Subscription-ID Missing from Header
>
> It is quite possible for one message generator to create
>
> messages for multiple subscriptions.  The subscription-ID
>
> will be needed to dispatch notifications to subscribers correctly.
>
> The Observation-Domain-ID is not sufficient.
>
>
>
> Same answer as above, we understand obs-domain-id to be used in the
> transport to meet load-balancing goals, but thought it would violate layers
> to place subscription-IDs in the transport itself.
>
>
>
>
>
> So I will put the Subscription-ID here instead I guess because a common
>
> use-case is to allow a Collector to support subscriptions from client
> applications.
>
> Only datastore subscriptions include a subscription-id, not event stream
> notifications.
>
>
>
>
>
>
>
> 5) Private Encoding Option does not seem interoperable
>
> The "Enc. Descr" field is completely opaque.
>
>
>
> Correct.
>
>
>
>
>
> Then perhaps there is no need for this at all since it does not
>
> attempt to provide any interoperability?
>
>
>
>
>
> This field could be structured to provide the "ET number space owner"
>
> e.g, urn [wsp description]
>
>
>
> Mmm it's a valid option. Our opinion was that once you want it
> interoperable, you should make it standard instead of private.
>
> If the wg thinks we should kinda standardize private options, I'm
> personally okay to add that to the draft.
>
>
>
> Use of structured strings like URNs does not seem like a bad thing that
> should be discouraged.
>
> The ET field is not interopable if every implementation can define their
> own 4-bit values
>
> and there is no standard way to tell them apart. Perhaps just a domain
> name if a URN is too much of a burden.
>
>
>
>
>
>
>
>
>
> Andy
>
>
>
> Thanks a lot for your comments,
>
>
>
> Pierre.
>
>
>
>
>
> Andy
>
>
>
>
>
> Le mar. 16 nov. 2021 à 04:45, Andy Bierman <[e-mail]> a écrit :
>
> Hi,
>
>
>
> I am wondering if there implementations of this draft already.
>
> IMO there are problems with in that prevent reasonable and
>
> optimal implementations.
>
>
>
> 1) Use of UDP as the Application Message Framing
>
> UDP is a transport protocol.  Tightly coupling one
>
> NETCONF notification message to one UDP frame (+ fragments)
>
> is not very optimal.
>
>
>
> 1A) wastes space in the UDP frame if the NETCONF notification
>
>     happens to be short
>
> 1B) imposes an application-level max-message-size
>
>
>
> 2) Not Efficient for Streaming Servers (no Message Chunking)
>
> The application framing in NETCONF and RESTCONF (HTTP) is not
>
> coupled to any transport protocol. Instead, a sender may emit as
>
> many message chunks as desired. This is how a streaming server works.
>
> Content comes from many different sources and it is sent on-the-fly
>
> in chunks, instead of constructing large replies.
>
>
>
>     - Application framing allows multiple notifications in one UDP frame
>
>     - Application framing does not need a max-message-size.
>
>       Streaming servers often do not know the size of the data that will
> be sent
>
>       in a notification.
>
>
>
> Q1) Should proprietary encoding be used to support application-layer
> framing
>
>        or should the standard support it?
>
>
>
> 3) Protocol Message Encoding for CBOR
>
> The NETCONF notification message is not modelled in YANG.
>
> This is an open issue for YANG to CBOR encoding.
>
> https://github.com/core-wg/yang-cbor/issues/96
> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcore-wg%2Fyang-cbor%2Fissues%2F96&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408366442%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=y%2FCrGoffl8qo7KHMVKjvQuz%2BWlAeavbt4i9hiIKDKdI%3D&reserved=0>
>
>
>
> The "notification" and "eventTime" elements do not have YANG definitions.
>
> SID assignments for these elements will be needed.
>
> This draft is the likely place for these SID assignment requests to
>
> the IESG and IANA.
>
>
>
> 4) Subscription-ID Missing from Header
>
> It is quite possible for one message generator to create
>
> messages for multiple subscriptions.  The subscription-ID
>
> will be needed to dispatch notifications to subscribers correctly.
>
> The Observation-Domain-ID is not sufficient.
>
>
>
> 5) Private Encoding Option does not seem interoperable
>
> The "Enc. Descr" field is completely opaque.
>
> This field could be structured to provide the "ET number space owner"
>
> e.g, urn [wsp description]
>
>
>
>
>
> Andy
>
>
>
>
>
>
>
> _______________________________________________
> netconf mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/netconf
> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fnetconf&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408366442%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mraDYXFzdEK%2BYtyORq0jp94MtekZfQ3LFIDfJbktLDw%3D&reserved=0>
>
>