[netconf] Re: Adoption call for notif-yang-04

[Kent Watsen <[e-mail]>]

Hi Thomas,

> KW> The word “manual” was out of place.   What I meant was that the code validates in two-steps: first the envelop is validated using generic schema, then the payload is validated using the specific schema.
>  
> TG> I think we are on the same page but lets try to be precise that we have a solid base. I would like to learn wherever you have the same understanding. You are refering to the XSD defined in https://datatracker.ietf.org/doc/html/rfc5277#section-4 for the event notification envelop and the notifications schemas in the ietf-subscribed-notifications YANG module described in https://datatracker.ietf.org/doc/html/rfc8639#section-4 resp. ietf-yang-push YANG module described in https://datatracker.ietf.org/doc/html/rfc8641#section-5. Correct? 

I only refer to the XSD in RFC 5277, and YANG modules having “notification” statements in them.  I’ve since abandoned the XSD and use code-logic to validate the envelop for JSON-based notifications.  Something like this:

assert(type(notif) == dict)
assert(len(notif) == 1)
assert("ietf-restconf:notification" in notif)
assert(type(notif["ietf-restconf:notification"]) == dict)
assert(len(notif["ietf-restconf:notification"]) == 2)
assert("eventTime" in notif[“ietf-restconf:notification"])

eventTime = notif["ietf-restconf:notification"].pop("eventTime")
payload = notif["ietf-restconf:notification"]

Where “payload” is something that can be passed to a YANG validator (e.g., yanglint -t notif …).


> TG> If correct, then I think the word payload is misplaced. I would word that the YANG-push notification schema is defined in XSD for the notification statement and in YANG for the rest of the YANG-push notification schema. This would be in line with
>  
> https://datatracker.ietf.org/doc/html/rfc8639#section-1.4
>  
>    o  The <notification> message of [RFC5277], Section 4 <https://datatracker.ietf.org/doc/html/rfc5277#section-4> is used.

I think that you’re raising a terminology point.  The words “payload” and “envelop” are not formally defined by NETCONF.  Looking at RFC 7277, the term “Event” is defined:

   Event:  An event is something that happens that may be of interest -
      a configuration change, a fault, a change in status, crossing a
      threshold, or an external input to the system, for example.
      Often, this results in an asynchronous message, sometimes referred
      to as a notification or event notification, being sent to
      interested parties to notify them that this event has occurred.

Note the 2nd sentence says "asynchronous message, sometimes referred to as a notification or event notification”.  

This “notification” is the whole document (envelop + payload in my vernacular), and NOT the YANG “notification” (just the payload in my vernacular).


> TG> And lets agree that schema is equally necessary for encoding, decoding and validating of messages.

>  
> TG> Is my assessment correct? Do you also come to the same conclusion?

In general, schema is good and helpful.  But stating that schema is “necessary” is likely a stretch,  Surely protocols have been successfully developed without schema.  As Andy states: a "protocol specification is not required to provide a YANG model for every message.”


> TG> I find reading that YANG-Push is based on notifications in https://datatracker.ietf.org/doc/html/rfc8639#section-1.4 and notifications can be encoded in JSON according to https://www.rfc-editor.org/rfc/rfc7951#section-1 for an implementer rather misleading. Would you agree?

Okay, it does seem that line is RFC 8639 is misleading, as RFC 7951 only covers encoding the data defined by the YANG “notification” statement, not the XSD “notification” element in RFC 5277.  To understand how to encode the envelop too, one needs to look at RFC 8040 or, alternately, https://datatracker.ietf.org/doc/html/draft-ietf-netconf-https-notif-15#section-4, which says:

JSON-encoded notifications are encoded the same as specified in Section 6.4 in RESTCONF [RFC8040] with the following deviations:

- The notifications do not contain the "data:" prefix used by Server-Sent Events (SSE).

- Instead of saying that, for JSON-encoding purposes, the module name for the "notification" element is "ietf-restconf", the module name will instead be "ietf-https-notif".

PS: I notice that the UDP-notif uses “ietf-notification” for the JSON encoding without explanation/definition.

PPS: it might be good the s/ietf-https-notif/ietf-notification/ in https-notif, since it hasn’t been published yet.  It’s too late for RFCs 8040 and 8650, but at least the “ietf-restconf” name is always being used for RESTCONF protocol.



> KW> I don’t understand.  Can you say some more?  Also, what remediation do you have in mind?
>  
> TG> Sure. RFC 7951 and RFC 9254 describing encoding rules how YANG. Both RFC's describing in the abstract and introduction that notifications are in scope but missing encoding rules for YANG. The only related information I was able to find is an example in the encoding rule section on anydata https://www.rfc-editor.org/rfc/rfc7951#section-5.5. In RFC 8639 and RFC 8641 it is assumed that the notification messages are JSON and CBOR encodable. However the encoding rules for notifications in RFC 7951 and RFC 9254 and the YANG module for https://datatracker.ietf.org/doc/html/rfc5277#section-4 is missing. Does that clarify?

I understand your position now.  I’ll agree that things don’t line up perfectly.  I disagree that a YANG-equivalent to the XSD in 5277 is necessary.  RFC 8639 points to 8640 and 8650 for dynamic subscriptions and, for configured subscriptions, doesn’t say anything (https://datatracker.ietf.org/doc/html/rfc8639#section-2.5.7) leaving it to docs like https-notif and udp-notif to fill in encoding details.


>  
> TG> Where I am puzzled is that this problem has already being reported in an errata for RC5277: https://www.rfc-editor.org/errata/eid6770 and discussed on the netconf mailing list here: https://mailarchive.ietf.org/arch/msg/netconf/VF8QwvhG8nLii6dXTYZYThrNhgc/. It received the same push back reasoning as Med did on https://mailarchive.ietf.org/arch/msg/netconf/Abw9mRHZos_yK9-x1HWHCVyv_xM/. 

I don’t see lack of YANG for the entire notification message a serious problem.  

Even if we could start over, the YANG language doesn’t support "wildcard nodes” (abstract substitution groups?), and so it is impossible to define the generic.

Getting back to the notif-yang draft, it’s unclear how that sx:structure would help.  Presumably the idea is that every  “notification” statement denied in YANG modules would be augmented into that structure (via the augment-structure statement), but:
  - the notif-yang draft doesn’t say this, so one is left to wonder how it is suppose to work.
  - the notif-yang draft doesn't provide examples showing the objective achieved.
  - if the augment-structure occurs in YANG modules, it doesn’t seem nice.
  - if the augment-structure occurs by implementations, it doesn’t need to be standardized.


> TG> I believe the root cause that notifications can't be encoded in JSON and CBOR due to missing YANG module does not lie in RFC 5277, it lies in Section 1.4 of RFC 8639. I highlighted in green the paragraphs describing where RFC 8639 is defining new elements and in yellow where it is based on RFC 5277. Agree?

See below.

> https://datatracker.ietf.org/doc/html/rfc8639#section-1.4
>  
>    o  This document defines a transport-independent capability;
>       [RFC5277 <https://datatracker.ietf.org/doc/html/rfc5277>] is specific to NETCONF.

This statement is somewhat true.

>  
>    o  For the new operations, the data model defined in this document is
>       used instead of the data model defined in Section 3.4 of <https://datatracker.ietf.org/doc/html/rfc5277#section-3.4>
>       [RFC5277] <https://datatracker.ietf.org/doc/html/rfc5277#section-3.4>.

This statement is false, an errata should be filed.

>  
>    o  The RPC operations in this document replace the operation
>       <create-subscription> as defined in [RFC5277], Section 4 <https://datatracker.ietf.org/doc/html/rfc5277#section-4>.

This statement is somewhat true.

 
>    o  The <notification> message of [RFC5277], Section 4 <https://datatracker.ietf.org/doc/html/rfc5277#section-4> is used.

This statement is false, an errata should be filed.

But note that the text in Section 1 is correct:  “Bindings for subscribed event
record delivery for NETCONF and RESTCONF are defined in [RFC8640] and
[RESTCONF-Notif], respectively.”


>    o  The included contents of the "NETCONF" event stream are identical
>       between this document and [RFC5277 <https://datatracker.ietf.org/doc/html/rfc5277>].

This statement is true.

>    o  A publisher MAY implement both the Notification Management Schema
>       and RPCs defined in [RFC5277 <https://datatracker.ietf.org/doc/html/rfc5277>] and this document concurrently.
>  
>    o  Unlike [RFC5277 <https://datatracker.ietf.org/doc/html/rfc5277>], this document enables a single transport session
>       to intermix notification messages and RPCs for different
>       subscriptions.
>  
>    o  A subscription "stop-time" can be specified as part of a
>       notification replay.  This supports a capability analogous to the
>       <stopTime> parameter of [RFC5277 <https://datatracker.ietf.org/doc/html/rfc5277>].  However, in this
>       specification, a "stop-time" parameter can also be applied without
>       replay.



Kent // contributor