Re: [netconf] Adoption-suitability for draft-unyte-netconf-udp-notif

[[e-mail]]

Hi Andy,

Sure,


  *   It appears that there is no application protocol here in this proposal.

Correct.


  *   Which means this proposal is for a general-purpose protocol for sending data as fast as possible
  *   over UDP, without regard for security or congestion.

It's a UDP-based transport for configured subscriptions. Where configured subscriptions refers to section 2.5 of (https://tools.ietf.org/html/rfc8639#section-2.5)
Section 3.2 (https://tools.ietf.org/html/draft-unyte-netconf-udp-notif-00#section-3.2) described the UDP header which facilitates the "Message-Generator-ID" and " Message ID" for loss detection. "ET" facilitates the encoding type.

How to handle fragmentation is described in Section 3.3.1 (https://tools.ietf.org/html/draft-unyte-netconf-udp-notif-00#section-3.3.1) by facilitating the options header.

There is no congestion control mechanism as described in Section 4 (https://tools.ietf.org/html/draft-unyte-netconf-udp-notif-00#section-4).

At IETF 109 another draft is going to be presented at netconf wg where DTLS as encryption option is described.


  *   Take the "XML" encoding for example.  That usually means the application content
  *   is an XML instance document -- a single XML element.  OK, which one? Which schema
  *   does the receiver use to validate the XML instance document?

I believe you are referring here to the subscription-id identifier described in RFC 8639 section 2.7.1 (https://tools.ietf.org/html/rfc8639#section-2.7.1) and section 3 (https://tools.ietf.org/html/rfc8639#section-3.3). When a configured subscription for an xpath is created, a subscription-id is assigned and sent with each message.

draft-unyte-netconf-udp-notif describes one transport option of RFC 8639. RFC 8639 describes in section 1.1 (https://tools.ietf.org/html/rfc8639#section-1.1) that it facilitates that multiple subscriptions can be handled on one transport sessions.

Let me know if this clarifies your questions. Feedback to draft-unyte-netconf-udp-notif or draft-unyte-netconf-distributed-notif are very welcome.

Best Wishes
Thomas

From: Andy Bierman <[e-mail]>
Sent: Tuesday, September 1, 2020 5:14 PM
To: Graf Thomas, INI-NET-TCZ-ZH1 <[e-mail]>
Cc: Zhoutianran <[e-mail]>; Netconf <[e-mail]>; Robert Wilton <rwilton=[e-mail]>
Subject: Re: [netconf] Adoption-suitability for draft-unyte-netconf-udp-notif



On Tue, Sep 1, 2020 at 6:33 AM <[e-mail]<mailto:[e-mail]>> wrote:

Hi Andy,



Thanks for the feedback. I think I understand in which direction your question goes, but haven't understood the detail, possible proposal and implications yet.  Would it be ok to setup a call between you, the authors and maybe Rob as well to discuss this off the list and feedback the summary to the list back? Looking forward to your reply.

I don't know if a conf-call is needed.
It appears that there is no application protocol here in this proposal.
Take the "XML" encoding for example.  That usually means the application content
is an XML instance document -- a single XML element.  OK, which one? Which schema
does the receiver use to validate the XML instance document?

If the answer is "any XML instance document you want" rather than "the application protocol
defines the message content", then there is no application, right?
Which means this proposal is for a general-purpose protocol for sending data as fast as possible
over UDP, without regard for security or congestion.




Best Wishes

Thomas


Andy


From: netconf <[e-mail]<mailto:[e-mail]>> On Behalf Of Andy Bierman
Sent: Monday, August 31, 2020 7:40 PM
To: Tianran Zhou <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>; Rob Wilton (rwilton) <rwilton=[e-mail]<mailto:[e-mail]>>
Subject: Re: [netconf] Adoption-suitability for draft-unyte-netconf-udp-notif

Hi,

I am still very confused about the content. Not the encoding. That is a separate issue.
Sec 2, and 3.1 refer to the "subscription-started" notification and
"the NETCONF/RESTCONF notification message".

This text seems to imply that the <notification> element defined in RFC 5277
is used for the content.  But sec. 3.4 mentions bundled notifications so maybe not.

Usually the application protocol defines the message flows that are expected.
This document is a transport protocol for an unspecified application.  There is
an assumption that publisher and receiver both know the protocol somehow.
For GDB, they magically know how to distinguish different proto files.

IMO this level of disregard for interoperability is unacceptable for a standard protocol.


Andy




On Fri, Aug 28, 2020 at 2:01 AM Tianran Zhou <[e-mail]<mailto:[e-mail]>> wrote:
Hi Andy,

Thanks again for your further comments.

Tianran

From: Andy Bierman [mailto:[e-mail]<mailto:[e-mail]>]
Sent: Friday, August 28, 2020 3:14 AM
To: Tianran Zhou <[e-mail]<mailto:[e-mail]>>
Cc: Rob Wilton (rwilton) <rwilton=[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Subject: Re: [netconf] Adoption-suitability for draft-unyte-netconf-udp-notif



On Wed, Aug 26, 2020 at 6:50 PM Tianran Zhou <[e-mail]<mailto:[e-mail]>> wrote:
Hi Andy,

Thanks for your reply and suggestions.

Cheers,
Tianran

From: Andy Bierman [mailto:[e-mail]<mailto:[e-mail]>]
Sent: Thursday, August 27, 2020 1:29 AM
To: Tianran Zhou <[e-mail]<mailto:[e-mail]>>
Cc: Rob Wilton (rwilton) <rwilton=[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Subject: Re: [netconf] Adoption-suitability for draft-unyte-netconf-udp-notif



On Wed, Aug 26, 2020 at 1:17 AM Tianran Zhou <[e-mail]<mailto:[e-mail]>> wrote:
Hi Andy,

Please see inline.

Tianran

From: netconf [mailto:[e-mail]<mailto:[e-mail]>] On Behalf Of Andy Bierman
Sent: Wednesday, August 19, 2020 6:18 AM
To: Rob Wilton (rwilton) <rwilton=[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>
Subject: Re: [netconf] Adoption-suitability for draft-unyte-netconf-udp-notif



On Tue, Aug 18, 2020 at 6:42 AM Rob Wilton (rwilton) <rwilton=[e-mail]<mailto:[e-mail]>> wrote:
Hi,

[Also as a contributor]

My comments are broadly similar to Kent's.

I believe that a UDP transport for dataplane telemetry where getting accurate fresh data quickly is more important than getting every update.  This is particularly true if a subsequent notification will cover any lost values anyway (e.g. periodic statistics).


agreed.
There is also the resync-subscription mechanism to help recover from missing data.

[ztr] Our intention of using UDP is for data that do not care about occasional loss. Typically for period data, and new data will anyway came and update.
Does the "resync-subscription" mean the retransmission? Here do you want to add the "resync-subscription mechanism" when packet loss is detected in this draft?



This is explained in RFC 8641

ZTR> Yes, I looked into RFC8641. It's the same as what I understand as "retransmission". Two key points on "resync-subscription" from my perspective.
1. "This RPC is supported only for on-change subscriptions previously established using an "establish-subscription" RPC."
2. "On receipt, a publisher must either (1) accept the request and quickly follow with a "push-update" or ..."

So, what's your suggestion here?
To support this? Or not support this and add text explicitly in the document.


Nothing to add to this document.
I was just noting that YANG Push does not rely on every push-change-update
notification to be delivered.  The application layer will re-synch by itself.

ZTR> Agreed. Thanks.


I suspect that having a mechanism to allow for the telemetry data being encrypted is probably also important.  If the WG were to adopt a draft without this, then as Benoit mentioned, I would have to test the water with the IESG to determine whether that would be acceptable.

I suspect the lack of congestion control might get their attention as well.

[ztr] Yes, I agree, both security and congestion considerations need to be described in the document. We have section 4 for congestion control. Do you have any idea on this?


no -- I can't solve problems that IETF creates for itself.
I think the draft says something like "do not use this protocol if congestion is a concern".



I also note that the draft allows for a GPB encoding of the telemetry data, but I'm not aware of any formal standard encoding of YANG data in GPB, and there is a choice between whether the GPB encoding is generic for all YANG data, or specific GPB encodings are useful for the specific data that is being encoded.



I was confused by the same thing.
Is there a YANG schema for a notification element that is used?
Which means the .proto file is hardwired?
If not then how does the receiver know what is sent?
There are some technical issues that can be addressed if the draft is adopted.

[ztr] In this draft we only consider to include a code point to indicate the GPB encoding. How to map YANG to GPB is out of the scope of this draft.
In practice, there are several ways in my opinion.
We can map YANG to proto firstly, and the proto actually describes the data structure. I think YANG can almost translate to proto 1:1.
We can also develop YANG to GPB directly, just like YANG to XML and JSON.
In brief, the receiver can still know what is sent by YANG.



I don't think the protocol should include placeholders for proprietary solutions.
Either leave out GPB completely or support it in an interoperable way.

ZTR> We have no position on this. If the WG agreed, we would like to leave out GPB.


What about just adding a 32-bit report-id field to the header
This is enough. Assignment of the IDs is out of scope.

 ZTR> Sorry, I did not get you on what's this 32-bit report-id? We have a 4 bit encoding type field, which can support 16 encodings. We can set this as a registry and defer the assignment of GPB.



More issues:

This protocol is not usable by servers that stream data instead of building an entire response
in memory. With a streaming design the server doesn't know what will be sent in advance so
it sure doesn't know the length in advance.  CBOR does a great job of supporting both
types of server design.  NETCONF and HTTP support chunking, which is used extensively by
streaming servers. This draft should mention streaming and explain why it is not supported.

ZTR> If you mean bit streaming like TCP, I think UDP cannot support this. Do you have any pointer on how CBOR can support the streaming. And how NETCONF and HTTP support chunking.
In general, I understand this requirement with small memory or partial data. But do you have any real case in the context of YANG push? So that we can work on an optimal design.


I think it is OK to say that a server is expected to build an entire response
and the length needs to be known in advance, if fragmentation is not enabled.
A server can stream data OK if UDP fragmentation is enabled. CoAP uses the Block option
to support streaming.

ZTR> Got it. Yes, this draft enables UDP fragmentation which can support data streaming.



The generic wrappers (like gNMI and YANG notification schema) do not handle "anydata" very well at all.
The binary payload becomes mostly JSON.  Only CBOR+SID handles anydata efficiently.
The solution (maybe outside the scope of this protocol) should consider how YANG anydata
will be transmitted efficiently and converted from anydata to a real schema correctly by the receiver.




Regards,
Rob


Andy


Andy



Andy




From: netconf <[e-mail]<mailto:[e-mail]>> On Behalf Of Kent Watsen
Sent: 07 August 2020 21:16
To: [e-mail]<mailto:[e-mail]>
Subject: Re: [netconf] Adoption-suitability for draft-unyte-netconf-udp-notif

[as a contributor]


   1) is the problem important for the NETCONF WG to solve?

I believe that it is important to enable publishers to send notifications using a UDP-based transport.   This belief is based on my experience from when at Juniper dealing with very high-end firewalls with enormous log output.

I believe that the NETCONF WG is the appropriate WG for this work, having defined RFC 8639 (SN), RFC 8640 (NN), and RFC 8650 (RN).


   2) is the draft a suitable basis for the work?

I have read the current version of the draft and find it to be a reasonable start.

Presuming the "receiver-instances" augmentation defined in https://tools.ietf.org/html/draft-ietf-netconf-https-notif-04#section-3<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Fdraft-ietf-netconf-https-notif-04%23section-3&data=02%7C01%7CThomas.Graf%40swisscom.com%7Cb05b26231b224226877308d84e89b330%7C364e5b87c1c7420d9beec35d19b557a1%7C1%7C0%7C637345700604407893&sdata=%2BgoArjA9BcziMZi%2F9qYt%2FndWc5D3lhjRp0VAshxvPXA%3D&reserved=0> takes off, the module defined in this draft should be updated to augment into it instead.

I appreciate Section 5 (Applicability) noting that the UDP-transport is primarily for the data plane (not the control plane), as it doesn't matter so much if data plane notifications are lost.  This addresses (I think) the issue that Rob Shakir raised before: https://datatracker.ietf.org/doc/minutes-103-netconf<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fminutes-103-netconf&data=02%7C01%7CThomas.Graf%40swisscom.com%7Cb05b26231b224226877308d84e89b330%7C364e5b87c1c7420d9beec35d19b557a1%7C1%7C0%7C637345700604417849&sdata=JPt9fV4Ymz2RxR8Ita9FA0Fmi1py2lw6qW3L%2BGGsRmE%3D&reserved=0> (search for "Rob S").  That said, it is unclear to me how a receiver could configure this while, e.g., configuring control plane notifications to be sent via a TCP-based transport such as "https-notif".


3) regarding Juergen's questions:

  a) I am willing to substantially review the drafts.
  b) I am willing to contribute to the discussion of any issue.
  c) I do NOT plan to implement the technology defined.



Kent
_______________________________________________
netconf mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/netconf<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fnetconf&data=02%7C01%7CThomas.Graf%40swisscom.com%7Cb05b26231b224226877308d84e89b330%7C364e5b87c1c7420d9beec35d19b557a1%7C1%7C0%7C637345700604417849&sdata=18cERfor9I1hj7A4dcLX9AREPNQ%2BkqrqvcQU5Cdo0F8%3D&reserved=0>