Re: [netconf] Issues with draft-ietf-netconf-udp-notif-04

[[e-mail]]

Hi Andy,

Looking at the list of media types I disagree with your observation. Please correct me if I am wrong, only a small subset is applicable to YANG push.

The possible encodings are currently being defined in both transport drafts
https://datatracker.ietf.org/doc/html/draft-ietf-netconf-https-notif#section-8.3
https://datatracker.ietf.org/doc/html/draft-ietf-netconf-udp-notif-04#section-3.2


However, you are raising a very valid point I noticed that the netconf working group haven't addressed clearly yet. I understood from previous threads with draft-ietf-netconf-https-notif and draft-ietf-netconf-udp-notif that multiple times questions were raised which are the valid transport encodings for YANG push.



For me it is unclear wherever CBOR could be used as encoding in draft-ietf-netconf-https-notif or not. Wherever draft-ietf-netconf-https-notif is encoding agnostic or not. I assume both transport drafts are encoding agnostic. Therefor we as working group should address the topic, which are the supported encodings for YANG push at present time, an how in the future when new encodings are being developed, we address this topic.



My suggestion is to keep the initial registries for both drafts equal. Defining JSON, XML and CBOR as the initial encoding. And if in the future new encodings are being added, ensure that both registries, for udp-notif and https-notif are being updated equally.



>From my point of view, we can roughly foresee that YANG push supports never more than 16 encoding types. However, due to the late YANG pus transport standardization at IETF and already existing non standardized YANG push encoding implementations,  I understood that private encodings is something we can't ignore. Therefore it makes sense to support it in the initial version of the transport documents.


Best wishes
Thomas

From: netconf <[e-mail]> On Behalf Of Andy Bierman
Sent: Friday, November 19, 2021 3:13 AM
To: Pierre Francois <[e-mail]>
Cc: Netconf <[e-mail]>
Subject: Re: [netconf] Issues with draft-ietf-netconf-udp-notif-04

Hi,

It is unclear why this document needs to reinvent content types using a 4-bit ET field.

There is already a registry of integer mappings for CoAP
https://www.iana.org/assignments/core-parameters/core-parameters.xhtml#content-formats<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.iana.org%2Fassignments%2Fcore-parameters%2Fcore-parameters.xhtml%23content-formats&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408346521%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=7WSsymSFBxP1W5U4iro6cg8I61dHc23YgfWIucH0KJ8%3D&reserved=0>

IMO this draft needs to (at least) define a header option for a 16-bit "CF" field.
Implementations SHOULD provide a CF option if none of the standard ET types apply.
One of the 16 standard ET values should mean "See CF option".


Andy





On Wed, Nov 17, 2021 at 11:31 AM Andy Bierman <[e-mail]<mailto:[e-mail]>> wrote:


On Wed, Nov 17, 2021 at 3:03 AM Pierre Francois <[e-mail]<mailto:[e-mail]>> wrote:
Hello Andy,

Thanks for the feedback, answers inline.

Le mar. 16 nov. 2021 à 04:45, Andy Bierman <[e-mail]<mailto:[e-mail]>> a écrit :
Hi,

I am wondering if there implementations of this draft already.
IMO there are problems with in that prevent reasonable and
optimal implementations.

There is support of udp-notif publisher in VRP.
There is a C collector lib for udp-notif in https://github.com/insa-unyte/udp-notif-c-collector<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Finsa-unyte%2Fudp-notif-c-collector&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408346521%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=3q8mNKLzfYKxqsR3ii0flAXmGhskv8RUN8ekduTJhZs%3D&reserved=0>


1) Use of UDP as the Application Message Framing
UDP is a transport protocol.  Tightly coupling one
NETCONF notification message to one UDP frame (+ fragments)
is not very optimal.

There is a segmentation option in the protocol, so a udp notif message is not tightly coupled to a UDP frame.

But this segmentation is optional to implement.
My point is that application protocols like NETCONF have their own message framing
which does not exist here because the transport framing is used instead.



1A) wastes space in the UDP frame if the NETCONF notification
    happens to be short

I don't get it, I'm sorry. To me a UDP header + small udp-notif header is rather short.

If I can fit 100 notification messages in 1 UDP frame then I save 99% of the redundant headers.



1B) imposes an application-level max-message-size

Not with the segmentation option, do we agree?

https://datatracker.ietf.org/doc/html/draft-ietf-netconf-udp-notif#section-4.1<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-netconf-udp-notif%23section-4.1&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408356481%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Xl%2BMQemsr4GneiC%2FUyWmC9bBruBPgHtZwLPBuvLeiWw%3D&reserved=0>

There can be 32767 segments of a max 65535 each: about 2G of data.
However the segment-size is allowed to be smaller and segmentation is completely optional-to-implement.

Application layer framing allows unlimited message size without any need
to support UDP segmentation at all,




2) Not Efficient for Streaming Servers (no Message Chunking)
The application framing in NETCONF and RESTCONF (HTTP) is not
coupled to any transport protocol. Instead, a sender may emit as
many message chunks as desired. This is how a streaming server works.
Content comes from many different sources and it is sent on-the-fly
in chunks, instead of constructing large replies.

    - Application framing allows multiple notifications in one UDP frame
    - Application framing does not need a max-message-size.
      Streaming servers often do not know the size of the data that will be sent
      in a notification.

I think we meet these goals. Let me think about this further.


I agree, by ignoring the idea that this protocol provides any normative information
about the application layer protocol in use.  This draft should remove any hint
that the application payload is defined in this document.  It is OK to say
a different RFC (or spec) defines the content for a particular Encoding Type.

The "notification" in Figure 1 implies that the payload is one NETCONF notification.
The draft should be clear that there is no structure to the payload defined in this document.



Q1) Should proprietary encoding be used to support application-layer framing
       or should the standard support it?

I think we'd need to meet on this or else it will end up with thousands of long emails ;)


I think the draft should let other documents define the exact content expected in the payload.
I am used to application protocols defining specific message flows. Since this draft does
not do that, it should not attempt to impose any rules on the application payload at all.




3) Protocol Message Encoding for CBOR
The NETCONF notification message is not modelled in YANG.
This is an open issue for YANG to CBOR encoding.
https://github.com/core-wg/yang-cbor/issues/96<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcore-wg%2Fyang-cbor%2Fissues%2F96&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408356481%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GychPbAURCdRp1pdWoQfPnaFo3gK3Al5AA8rdcQKt0E%3D&reserved=0>

The "notification" and "eventTime" elements do not have YANG definitions.
SID assignments for these elements will be needed.
This draft is the likely place for these SID assignment requests to
the IESG and IANA.

Shouldn't these be transport agnostic? I don't see why this would be bound to udp-notif only and not, say, https-notif.

This is related to application/yang-data+cbor.
A separate document for the SID assignments can be done.




4) Subscription-ID Missing from Header
It is quite possible for one message generator to create
messages for multiple subscriptions.  The subscription-ID
will be needed to dispatch notifications to subscribers correctly.
The Observation-Domain-ID is not sufficient.

Same answer as above, we understand obs-domain-id to be used in the transport to meet load-balancing goals, but thought it would violate layers to place subscription-IDs in the transport itself.


So I will put the Subscription-ID here instead I guess because a common
use-case is to allow a Collector to support subscriptions from client applications.
Only datastore subscriptions include a subscription-id, not event stream notifications.



5) Private Encoding Option does not seem interoperable
The "Enc. Descr" field is completely opaque.

Correct.


Then perhaps there is no need for this at all since it does not
attempt to provide any interoperability?


This field could be structured to provide the "ET number space owner"
e.g, urn [wsp description]

Mmm it's a valid option. Our opinion was that once you want it interoperable, you should make it standard instead of private.
If the wg thinks we should kinda standardize private options, I'm personally okay to add that to the draft.

Use of structured strings like URNs does not seem like a bad thing that should be discouraged.
The ET field is not interopable if every implementation can define their own 4-bit values
and there is no standard way to tell them apart. Perhaps just a domain name if a URN is too much of a burden.




Andy

Thanks a lot for your comments,

Pierre.


Andy


Le mar. 16 nov. 2021 à 04:45, Andy Bierman <[e-mail]<mailto:[e-mail]>> a écrit :
Hi,

I am wondering if there implementations of this draft already.
IMO there are problems with in that prevent reasonable and
optimal implementations.

1) Use of UDP as the Application Message Framing
UDP is a transport protocol.  Tightly coupling one
NETCONF notification message to one UDP frame (+ fragments)
is not very optimal.

1A) wastes space in the UDP frame if the NETCONF notification
    happens to be short
1B) imposes an application-level max-message-size

2) Not Efficient for Streaming Servers (no Message Chunking)
The application framing in NETCONF and RESTCONF (HTTP) is not
coupled to any transport protocol. Instead, a sender may emit as
many message chunks as desired. This is how a streaming server works.
Content comes from many different sources and it is sent on-the-fly
in chunks, instead of constructing large replies.

    - Application framing allows multiple notifications in one UDP frame
    - Application framing does not need a max-message-size.
      Streaming servers often do not know the size of the data that will be sent
      in a notification.

Q1) Should proprietary encoding be used to support application-layer framing
       or should the standard support it?

3) Protocol Message Encoding for CBOR
The NETCONF notification message is not modelled in YANG.
This is an open issue for YANG to CBOR encoding.
https://github.com/core-wg/yang-cbor/issues/96<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcore-wg%2Fyang-cbor%2Fissues%2F96&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408366442%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=y%2FCrGoffl8qo7KHMVKjvQuz%2BWlAeavbt4i9hiIKDKdI%3D&reserved=0>

The "notification" and "eventTime" elements do not have YANG definitions.
SID assignments for these elements will be needed.
This draft is the likely place for these SID assignment requests to
the IESG and IANA.

4) Subscription-ID Missing from Header
It is quite possible for one message generator to create
messages for multiple subscriptions.  The subscription-ID
will be needed to dispatch notifications to subscribers correctly.
The Observation-Domain-ID is not sufficient.

5) Private Encoding Option does not seem interoperable
The "Enc. Descr" field is completely opaque.
This field could be structured to provide the "ET number space owner"
e.g, urn [wsp description]


Andy



_______________________________________________
netconf mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/netconf<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fnetconf&data=04%7C01%7CThomas.Graf%40swisscom.com%7Cc0e59f0fa7764b44771108d9ab023f4e%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C637728848408366442%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mraDYXFzdEK%2BYtyORq0jp94MtekZfQ3LFIDfJbktLDw%3D&reserved=0>