[netconf] Re: Adoption call for notif-yang-04

[Kent Watsen <[e-mail]>]

Hi Thomas,

> On Jun 18, 2024, at 2:02 AM, [e-mail] wrote:
> 
> Dear Kent,
>  
> Just two comments/clarifications from my side
>  
> The udp-notif draft already uses “ietf-notification” but doesn’t have normative text for why.
>  
> I think you are refering to below text and why it is "informativ" vs. "normative". Correct?
>  
> In the -13 revision there will be also an additional example in the appendix section detailing the layering of the packet including the notification message.
>  
> I have been reading "https://datatracker.ietf.org/doc/statement-iesg-iesg-statement-normative-and-informative-references-20060419/" to judge wherever it should be normative or not. I think we agree that looking at YANG-Push notification as a whole, an implementer implementing udp-notif needs to understand RFC 8639 and therefore also draft-ahuang-netconf-notif-yang. Since RFC 8639 is already normative in draft-ietf-netconf-udp-notif, I concur that draft-ahuang-netconf-notif-yang should be normative as well. That implies that when a new notification structure is being defined, the document needs to update draft-ietf-netconf-udp-notif then. Agree?
>  
>  
> https://datatracker.ietf.org/doc/html/draft-ietf-netconf-udp-notif-12#section-3.1
>  
> As specified in Sub-Notif, the YANG data is encapsulated in a NETCONF/RESTCONF notification message, which is then encapsulated and carried using a transport protocols such as TLS or HTTP2. This document defines a UDP based transport. Figure 1 <https://datatracker.ietf.org/doc/html/draft-ietf-netconf-udp-notif-12#fig_ups_message> illustrates the structure of an UDP-Notif message. 
>   The Message Header contains information that facilitate the message transmission before deserializing the notification message.
> 
>   Notification Message is the encoded content that is transported by the publication stream. The common encoding methods are listed in Section 3.2 <https://datatracker.ietf.org/doc/html/draft-ietf-netconf-udp-notif-12#sec_ups_format>. The structure of the Notification Message is defined in Section 2.6 of [RFC8639 <https://www.rfc-editor.org/info/rfc8639>] and a YANG model has been proposed in [I-D.ahuang-netconf-notif-yang <https://datatracker.ietf.org/api/v1/doc/document/draft-ahuang-netconf-notif-yang/>]. [I-D.ietf-netconf-notification-messages <https://datatracker.ietf.org/doc/html/draft-ietf-netconf-notification-messages-08>] proposes a structure to send bundled notifications in a single message.
> 
> +-------+  +--------------+  +--------------+
> |  UDP  |  |   Message    |  | Notification |
> |       |  |   Header     |  | Message      |
> +-------+  +--------------+  +--------------+
> Figure 1 <https://datatracker.ietf.org/doc/html/draft-ietf-netconf-udp-notif-12#figure-1>: UDP-Notif Message Overview <https://datatracker.ietf.org/doc/html/draft-ietf-netconf-udp-notif-12#name-udp-notif-message-overview>

What I mean is that the udp-notif doesn’t seem to have a section like this one in the https-notif draft: https://datatracker.ietf.org/doc/html/draft-ietf-netconf-https-notif-15#section-4.1.

The idea is that each “notif" draft (udp-notif, https-notif, kafka-notif, etc.) defines how notifications are encoded, for the encodings it supports.  Ideally, for each encoding, this is done by pointing to common encoding-specific drafts, but the transport drafts ultimately have the last word.  

As an example of having the last word, the https-notif draft say that, for JSON, the module name "ietf-restconf” is replaced by "ietf-https-notif”, but this should be changed to “ietf-notification”  (Mahesh, can you do this?).

K.



> By “associated schema”, do you mean a singular schema are perhaps two schema, one for the envelop and the other for the payload?
>  
> I just wanted to share my opinion- Maybe it helps the conversation. In these conversations, I am thinking about the overall architecture first. Networks are built in layers so responsibilities and clear boundaries can be defined. Different to BMP where BGP PDU's are being mirrored and encapsulated, YANG-Push does not encapsulate (I do not find any normative references). It encodes. Therefore the entire message structure from the netconf notification header down to the subscribed content is one message. Removing the netconf notification header from the YANG-Push notification message would render the message unusable since there is no longer a message timestamp. Therefore I believe that the netconf notification header is part of the message and therefore the YANG-Push message should be treated as one message vs YANG-Push notification encapsulated into netconf notification. Keeping the integrity of the message layer.
>  
> Best wishes
> Thomas
>  
> From: Kent Watsen <[e-mail] <mailto:[e-mail]>>
> Sent: Monday, June 17, 2024 5:17 PM
> To: Rob Wilton (rwilton) <[e-mail] <mailto:[e-mail]>>
> Cc: [e-mail] <mailto:[e-mail]>
> Subject: [netconf] Re: Adoption call for notif-yang-04
>  
> Be aware: This is an external email.
>  
> Hi Rob, 
>  
> 
> 
> On Jun 17, 2024, at 9:45 AM, Rob Wilton (rwilton) <[e-mail] <mailto:[e-mail]>> wrote:
>  
> Hi Kent, & authors,
>  
> Sorry, I thought that I had replied to this adoption call, but no, I see that I have a draft written sitting that I failed to actually send (but will post shortly) …
>  
> But taking a few steps back here.  As I suspect folks are already aware, I am in in the process on implementing YANG push on a core router to expose the operational state.  My initial focus is on-change notifications using the JSON encoding.  From my reading of the RFCs, if you are not using RESTCONF, then no RFC currently specifies how to encode YANG notifications in JSON (or CBOR).  E.g., Andy says that the top level notification should be namespaced to ietf-restconf (as per RFC 8040, section 6.4), but given that we are not implementing RESTCONF then for the moment I’m basing the encoding on the example from the YANG JSON encoding (RFC 7951), where there is an example using “ietf-notification:notification”, which seems to makes more sense to me if the subscription isn’t bound to RESTCONF anyway.  I appreciate that this text is just an example and not normative, but I still don’t think how JSON notifications are expected to be encoded outside of RESTCONF is properly specified anywhere.
>  
> Agree that “ietf-restconf” is not a great module name when RESTCONF isn’t used.
>  
>  
> Hence, at a minimum, I believe that we need an RFC to specify how NETCONF notifications are encoded over JSON when not using RESTCONF.
>  
> The good news is that the only published use of the “ietf-restconf” module-name  (that I’m aware of) appears RFC 8040 and RFC 8650 (for *dynamic* subscriptions).  So the creep isn’t too bad.
>  
> For *configured* subscriptions, the "https-notif" and "udp-notif" drafts appear normative.  Currently the https-notif draft says to use string “letf-https-notif”, but this can (should, IMO) be changed to “ietf-notification”.  The udp-notif draft already uses “ietf-notification” but doesn’t have normative text for why.
>  
> If the “https-notif” and “udp-notif” drafts are updated to use “ietf-notification”, normalizing the module name seems mostly resolve.
>  
> 
> 
> But also considering the YANG CBOR encoding (RFC 9254), that also doesn’t seem to be particularly clear about how YANG notifications are encoded in CBOR and could probably do with some additional clarity about what namespace the root notification element belongs to, and what associated SIDs should be used.
>  
> Likely, but also not this WG’s problem  ;)
>  
> Still, to the extent the configured-subscriptions are used, the https-notif and udp-notif drafts apply.
>  
>  
> Andy is arguably right that this doesn’t mean that it is necessary to define a YANG structure for a notification, but I can see some obvious benefits in being able to define such a structure, allowing implementations to choose to generically verify the message against a YANG schema without needing to hard code it.  I.e., it seems like specifying the structure in a more programmatic way may well benefit some implementations and shouldn’t be detrimental to other implementations that want to hard code the notification handling code.  I also think that we should consider that the goal here isn’t just for the collector to be able to use the schema to validate the message and then write it to storage, but instead to propagate these messages into a message broker (e.g., Kafka) and on to other services, whilst also propagating alongside an appropriate reference to the associated schema for the entire message (i.e., the top level JSON structure including the notification fields).
>  
> By “associated schema”, do you mean a singular schema are perhaps two schema, one for the envelop and the other for the payload?
>  
> I can imagine two schema but, due to limitations in YANG, the one for the envelop would not be able to validate whole message, with the “payload" present.  I can imagine these possible solutions, all entailing app-logic on the receiver side:
>  
> 1. Receiver extracts payload from whole message and then validates them separately using, e.g., the sx:structure schema for the envelop and the YANG-notification’s module for the payload.
>  
> 2. Receiver just-in-time creates a schema that augments the YANG “notification” statement into the sx:structure, and then validates them together.
>  
> 3. Update YANG so overcome the limitation then: Receiver validates the whole message (which really just validate the envelop) then extracts the payload and validates it using the YANG-notification’s module for the payload.   [this isn’t much better than #1]
>  
> 
> 
> Finally, we are also aware that there is a desire to add some additional meta-data annotation into the notifications structure.  Not all of these are useful for all notifications and hence could be defined in other augmenting modules (but only if there is a sensible base structure defined in YANG that can be augmented), otherwise each of these extra meta-data annotations is also stuck defining a bespoke encoding for XML, JSON, & CBOR, some of which could even end up using protocol specific namespaces (e.g., for RESTCONF).
>  
> Different headers in different contexts?   This plus that https-notif and udp-notif (presumably) can be used outside of RFC 8639 configured subscriptions, could create interoperability issues.  This is why I suggested to either:
>  
> 1. Hardcode the header-fields per protocol version.  E.g., RFC 8639 == has 8639-headers, rfc8639bis has bis-headers, and so on.  This seems sensible to me.
>  
> 2) Let client-configuration specify the header-fields to be used.  This seems possible, but may impact performance.
>  
> 
> 
> Regards,
> Rob
>  
> Kent // contributor