Re: [Netconf] IETF 101 SN Question 1: Proper designation of receiver

[Martin Bjorklund <[e-mail]>]

Kent Watsen <[e-mail]> wrote:
> 
> 
> >> >> > Correct.  But your question was "can you use netconf-notif without
> >> >> > a leafref
> >> >> to...".
> >> >> > Needing both drafts is absolutely the case for dynamic subscription
> >> >> > support, and ietf-netconf-server would not be needed here.
> >> >>
> >> >> I read the above a few times, but I'm having a hard time understanding it.
> >> >> Can you say it differently or provide an example?
> >> >
> >> > Dynamic subscriptions over NETCONF requires draft-ietf-netconf-netconf-
> >> event-
> >> > notifications.
> >> 
> >> Where is the dependency?  I don't see anywhere in the 3 RPCs and associated
> >> error-info definitions that have a reference to the identity in that draft.
> >
> > The dependency is a document requirements dependency: deployment of NETCONF
> > based dynamic subscriptions requires support of both relevant requirements
> > sections 5, 7, & 8 from draft-ietf-netconf-netconf-event-notifications in
> > addition to draft-ietf-netconf-subscribed-notifications.  
> 
> Sure, I get this, but we're talking about if there is YANG-level dependency,
> for which I believe we've concluded that the answer is "no".
> 
> What this means is, for servers that only want to support NETCONF-based 
> dynamic subscriptions (no configured subscriptions), then the 
> ietf-netconf-subscribed-notifications module can be listed in yang-library
> as *not implemented*.

No, since the server must implement the rpcs, the module must be
listed as "implemented" (the feature "configured" would not be
advertised though).


/martin


> And for servers that want to support NETCONF-based 
> configured subscriptions, then ietf-netconf-subscribed-notifications can
> be listed in yang-library as *implemented*.
> 
> Looking at the thread that led up to this point, this means that it would
> be okay for ietf-netconf-subscribed-notifications to have a leafref to a
> global /netconf-server/call-home/netconf-client, while not forcing the
> implementation of the ietf-netconf-server module, for servers that only
> want to support dynamic subscriptions.
> 
> And, to the question that started this fork in the thread "is it possible
> that a device wants to use SN but doesn't *implement* ietf-netconf-server",
> the answer is "yes".
> 
> 
> 
> 
>  
> >> >> >> (b) this seems like a possibility, but then I think this make the
> >> >> >> case for why a leafref to the global *conf servers definitions won't always
> >> >> work.
> >> >> >
> >> >> > Agree that nothing here will always work.  Deployments commonly will
> >> >> > have a heterogeneous mixture of model ecosystem models.
> >> >> >
> >> >> > This actually makes a *very* strong case for why the leafref should be
> >> >> > added as an augmentation from the *conf-server models.  That way
> >> >> > leafref augmentations are explicitly tied to the actual implementation of
> >> the
> >> >> model against which they refer.
> >> >>
> >> >> Not in the *conf-server models, the augments go into the *conf-notif
> >> models, I
> >> >> assume that is what you meant.
> >> >
> >> > My assertion is a good solution would be updating ietf-netconf-server.yang
> >> > per what is below.  Note that an answer even further below regarding the
> >> > sharing of a single NETCONF session across multiple subscriptions and typical
> >> > RFC6241 protocol interactions is assumed.  But we could also insert your
> >> > ietf-netconf-server.yang grouping just as effectively where the leafref is seen.
> >> >
> >> > Anyway here are the following changes which would be made to ietf-
> >> netconf-server.yang
> >> >
> >> >  import ietf-subscribed-notifications { prefix sn; }
> >> >  import ietf-netconf-subscribed-notifications { prefix nsn; }
> >> >
> >> >  feature subscription-support {
> >> >    description
> >> >        "The 'subscription-support' feature indicates that the NETCONF server
> >> >         supports configured subscriptions over call-home connections.";
> >> >       reference
> >> >        "RFC xxxx: Customized Subscriptions to a Publisher's Event Streams";
> >> >     }
> >> >
> >> > augment "/sn:subscriptions/sn:subscription/sn:receivers/sn:receiver" {
> >> >   if-feature "subscription-support";
> >> >   when 'derived-from(../../../transport, "nsn:netconf")';
> >> >   description
> >> >      "This augmentation allows NETCONF specific parameters to be exposed for
> >> a receiver.";
> >> >    leaf netconf-endpoint {
> >> >      type leafref {
> >> >        path "/ncs:netconf-server/ncs:call-home/ncs:netconf-client/ncs:name";
> >> >      }
> >> >      description
> >> >        "Remote client which need to initiate the NETCONF transport if an
> >> existing
> >> > NETCONF session from that client is not available.";
> >> >    }
> >> >  }
> >> >
> >> > With such a construct, it is impossible to add a leafref (or grouping) within
> >> > ietf-subscribed-notifications unless ietf-netconf-server.yang exists.
> >> 
> >> True, and thanks for providing a concreate example.  Though I thought we
> >> concluded
> >> before that there might be cases where the global netconf-server isn't
> >> implemented?
> >> Now you're okay making that a requirement?  (I'm okay with that, if it works)
> >
> > I am ok with making it a requirement for drafts
> 
> Okay, assuming we resolve the "I'm not entirely sure if I understand if
> what is planned is legal" issue discussed below.
> 
> 
> > ...subsequent to the current draft-ietf-netconf-netconf-event-notifications.
> 
> This is TBD, per the discussion below, but we can try...
> 
> 
> >   Either a revision to draft-ietf-netconf-netconf-event-notifications, or
> > an update to the ietf-netconf-server.yang.
> 
> Right.   But if the dependency only goes one way, then I think the choice
> is made for us already. 
> 
> 
> >> FWIW, I think that an import statement can also assert that a dependent
> >> module is
> >> implemented.  For instance, in the below case, the xpath in the leafref forces
> >> that the module is implemented:
> >> 
> >>   module ietf-netconf-subscribed-notifications {
> >>     prefix nsn;
> >>     import ietf-netconf-server { prefix ncs; }
> >>     import ietf-subscribed-notifications { prefix sn; }
> >> 
> >>     augment "/sn:subscriptions/sn:subscription/sn:receivers/sn:receiver" {
> >>       if-feature "subscription-support";
> >>       when 'derived-from(../../../transport, "nsn:netconf")';
> >>       description
> >>         "This augmentation allows NETCONF specific parameters to be
> >>          exposed for a receiver.";
> >>       leaf netconf-endpoint {
> >>         type leafref {
> >>           path "/ncs:netconf-server/ncs:call-home/ncs:netconf-client/ncs:name";
> >>         }
> >>         description
> >>           "Remote client which need to initiate the NETCONF transport if
> >>            an existing NETCONF session from that client is not available.";
> >>       }
> >>     }
> >>     ...
> >>   }
> >> 
> >> I prefer this arrangement because it gives tangible meaning for what it means
> >> to *implement* the netconf-subscribed-notifications module.
> >
> > I understand.  As long as we make the choice as to where to land this future
> > leafref after the current draft-ietf-netconf-subscribed-notifications completes,
> > I am good.
> 
> Pending the discussion below...
> 
> 
> 
> >> >> >> This is why I
> >> >> >> was thinking before that your modules might themselves *use* the
> >> >> >> *conf- server-groupings (while pruning out unneeded parts, e.g., the
> >> >> >> "listen" subtree), so that it's independent of what the system has
> >> >> >> implemented at the global level.
> >> >> >
> >> >> > If you have 500 subscriptions, you then have to populate 500 identical
> >> >> groupings.
> >> >>
> >> >> No, you have one grouping, with 500 /netconf-server/call-home/netconf-
> >> client
> >> >> instances.
> >> >
> >> > Yes.    But I don't know why someone would voluntarily do add 500 repeated
> >> > elements to a configuration datastore.
> >> 
> >> At first I was going to point out that, even if using to global netconf server
> >> container, there would still be 500 /netconf-server/call-home/netconf-client
> >> instances, but in looking ahead, I'm wondering if I misunderstand the intended
> >> relationship between transports, subscriptions, and receivers.
> >> 
> >> If it turns out that receivers from different subscriptions can leafref the
> >> same /netconf-server/call-home/netconf-client, then the 500 becomes 1, and
> >> the duplicate data-entry concern goes away.
> >
> > Exactly.  This has always been the objective.
> 
> Okay.  Sorry for being slow to get this.  Please take a close look at SN draft
> to ensure this is super clear there.
> 
> 
> 
> >> >> >  And yes this is possible.  But it makes the part of me which likes
> >> >> > Normalized  data quite uncomfortable.
> >> >> >
> >> >> > But as I said before, it the WG wants such redundancy, fine.  Either
> >> >> > choice need not impact decisions as part of LC.
> >> >>
> >> >> I don't believe that is a WG-preference thing, so much as an outcome of the
> >> >> current design, which is that each receiver for each subscription has its own
> >> >> state-machine and protocol messages.  There is no sharing; no two receivers
> >> can
> >> >> use the same RFC 6241 NETCONF session, which effectively translates to
> >> each
> >> >> receiver having its own /netconf-server/call-home/netconf-client instance,
> >> >> right?
> >> >
> >> > This is incorrect.    Protocol and state-machine messages have been
> >> decoupled
> >> > from the transport session.
> >> 
> >> As mentioned above, I'm wondering if I misunderstand the intended
> >> relationship
> >> between transports, subscriptions, receivers, and maybe publishers too.  Can
> >> you put together a diagram that describes these relationships?
> >
> > A configured subscription on a publisher can have many receivers.
> >
> > A configured subscription on a publisher may only use one type of transport (and one type of encoding).
> >
> > A configured receiver can receive information from multiple configured subscriptions on a single transport session from a publisher.
> 
> I'd like to have statements like these in the SN draft.  Maybe as part 
> of the term definitions, but that might be too much information (busy)
> for terms. The info could be sprinkled throughout the doc, but I wonder
> if that might not already be the case and, if so, then it didn't work
> out too well before (witness my confusion here), so perhaps some other 
> section would be better?
> 
> 
> 
> >> > I am not sure why you think that subscriptions are unable to use a common
> >> > NETCONF session?   Implementations of dynamic NETCONF subscriptions
> >> have
> >> > been doing this for years.    Subscription multiplexing of configured and
> >> > dynamic subscriptions over a common transport is a pre-requisite for
> >> > solution scalability.
> >> 
> >> I think because its underspecified in the SN draft, and there was confusion
> >> with the address and port leafs, and ietf-netconf-subscribed-notifications
> >> only defines an identity (no configuration data model).
> >
> > In a parallel thread to Martin, I have added a sentence aimed here.   Beyond
> > that, configuration data model forces choice of the identity for the configured
> > subscription.
> 
> In that thread, you wrote:
> 
> """
> I have added the following to the NETCONF-Notif document section on configured subscriptions:
> 
> "It is possible to have multiple configured subscriptions sharing a common transport to a single receiver.  The method of identifying that a receiver happens to be the same as used with another subscription is left up to implementers of this specification."
> """
> 
> This is a good sentence (assuming the discussion regarding *why* this is simpler pans out),
> but shouldn't it be in the SN document (not netconf-notif)?
> 
> 
> 
> >> Okay, the answer is that its considered "simpler" to use a single kind
> >> (not instance) of transport.  So, the outcome is, if one receiver of a
> >> subscription is using a NETCONF-based transport, then all the other
> >> receivers of that subscription MUST also be using a NETCONF-based
> >> transport, albeit a different instance of a NETCONF-based transport
> >> (as it would be redundant otherwise).  Correct?
> >
> > Yes
> > 
> >
> >> Assuming this is the case, my question is, why is this "simpler"?  I mean,
> >> assuming an event occurs that a subscription matches, the publisher will
> >> encode a notification message to send, and then iterate over its list of
> >> receivers, sending the same encoded-message to each.  But why is it less
> >> simple if different transports (netconf, restconf, etc.) are used?
> >
> > As can be heard in the recording, and seen on dozens of WG emails, these
> > issues were deeply debated.  As can been seen my slight preference actually
> > was different transports.  And that is how earlier versions of the model
> > covered the issue.  However the WG chose a single transport for rational
> > reasons at and after IETF 100.  The issue was closed and the drafts
> > updated accordingly.  
>  
> Eric, I'm asking for a technical answer.  In a nutshell, what are
> the "rational reasons"?   Yes, I recall your having a preference for
> heterogeneous transports...
> 
> 
> >> BTW, separately, I kind of but not really understand why there is a desire
> >> for the fixed encoding for all the receivers in a subscription.  I understand
> >> the efficiency angle (see prev paragraph), but I get stuck on the idea that,
> >> if there is a *need* to send a different encoding (e.g., "encode-json"),
> >> another encoded message structure is going to have to be created anyway; it
> >> seems like the same number of instructions from that perspective.  Then it
> >> goes to looping over one-subscription-tree or one-tree-per-encoding.  Okay,
> >> then, what makes it better? 
> >
> > Some implementations have claimed it is easy to bind the subscription with
> > the encoding, and difficult to perform filtering before the encoding.  So
> > it is better to force this separation.
> 
> Okay.  (but see next paragraph).
> 
> 
> >> The only thing I can come up with is that it
> >> might be difficult otherwise to express in YANG what encoding is being used
> >> for that receiver.  For instances, if there is a leafref to /restconf-server\
> >> /call-home/restconf-client, nowhere is there an "encoding" field.  Hmmm,
> >> maybe the encodings a restconf server supports could be specified at a
> >> higher level (e.g., /restconf-server/encodings/...), and then it would be
> >> known, on a per-receiver basis, what encoding is used (netconf is always
> >> xml, restconf is per configuration).  Anyway, I'm just wondering if this
> >> is why the encoding for all the receivers in a subscription must be the
> >> same, or is it something else?
> 
> I just sent a question to the WG regarding if ietf-restconf-server should
> have a way configure which encodings it supports.  If this pans out, the
> impact here is that we might want a "must" statement to ensure that the
> selected encoding is supported by the leafref-ed /rcs:restconf-server/
> instance.
> 
> 
> 
> >> In this particular fork in the thread, I think that we're discussing the merits
> >> if leafref-ing vs using a grouping.  If it is the case that the same transport
> >> can be used across subscriptions, then 1) it swings things back to leafref
> >> approach being needed and 2) this fork in the thread is done.  [Assuming that
> >> it’s a leafref, we still need to finalize if it's a leafref to the global
> >> server instance or some SN-specific instance.]
> >
> > I believe leafref is good.  And as long as the leafref is inserted after
> > the current drafts in WGLC complete, I am good.
> 
> Yes, leafref seems needed.  Whether the leafref is global vs. local, and to what
> the leafref points to, are still TBD.
> 
> 
> 
> >> >> >> There is a difference between a server not *implementing* a ietf-*conf-
> >> >> >> server module and the *conf-notif not *using* the *conf-server-grouping
> >> >> >> statements.  My suggestion has been, that the *conf-notif drafts should
> >> >> >> have their own lists of netconf-servers (via "uses" statements), and
> >> >> >> thereby not be dependent on the existence of a global ietf-*conf-server
> >> >> >> instance (which may not exist).
> >> >> >
> >> >> > While technically correct, there are several reasons why this is problematic.
> >> >> > (1) redundancy (see the 500 above)
> >> >>
> >> >> This is a non-issue (see above)
> >> >
> >> > This is still an issue, as the drafts in WGLC support a single NETCONF session
> >> > for all subscriptions and normal protocol operations.
> >> 
> >> As said before, sharing the same transport across subscriptions wasn't clear
> >> to me before.  Still, even as 500 becomes 1, there remains the discussion if
> >> the one is the global server instance or some SN-specific server instance.
> >
> >Same comment as above.
> 
> Which is that you're okay with the *conf-notif drafts necessitating the existence
> of /*conf-server/ instances (i.e., the ietf-*conf-server module is implemented)
> and, of course, you're hoping that this dependency can be introduced in some
> future bis version of the *conf-notif drafts.
> 
> 
> 
> >> >> > (2) availability of the group means that a platform will have exposed
> >> >> > *conf-server.  Explaining that a model is only available for its
> >> >> > grouping would be quite a confusing deviation.
> >> >>
> >> >> No, it's easy, this is the difference between a module being *implemented*
> >> >> or not.  The implementation status of each module is yang-library.
> >> >
> >> > Yes, what you say is possible.  It is also more complex.
> >> 
> >> Not just possible, it is actually how it happens.  The client-server modules
> >> are highly sensitive to implementation status.  FWIW, I never expect the
> >> ietf-*conf-client modules to ever be implemented, and the ietf-*conf-server
> >> modules to be implemented "sometimes".  FWIW, the global server instances
> >> we keep talking about only happen *if* the ietf-*conf-server modules are
> >> implemented.
> >
> > I have seen implementations of YANG models without having a yang-library.
> > I prefer a yang-library of course.
> 
> From a SDO perspective, yang-library is expected to be implemented (it's a
> MUST in RFC 8040 and in nmda-restconf).  We should fully assume that the
> server implements yang-library.  That said, if I were the implementer of
> a receiver that does a dynamic subscription, I would probably write the
> code to just send the establish-subscription request and check to see 
> if the server returned an <rpc-error>, without first checking if the
> module is listed in yang-library...
> 
> 
> >> > So can we take out address and finally be done?   That would be a good
> >> thing.
> >> 
> >> Yes, take out the address leaf but I think that, if we want to progress the
> >> SN draft along with a transport binding definition that doesn't depend on
> >> the ietf-*conf-server modules, then we might define something else like:
> >> 
> >>   module ietf-netconf-no-crypto-subscribed-notifications {
> >>     prefix nncsn;
> >>     import ietf-subscribed-notifications { prefix sn; }
> >> 
> >>     container implicit-netconf-receivers {
> >>       list implicit-netconf-receiver {
> >>         key name;
> >>         leaf name { ... }
> >>         leaf address { ... }
> >>         leaf port { ... }
> >>       }
> >>     }
> >>     augment "/sn:subscriptions/sn:subscription/sn:receivers/sn:receiver" {
> >>       if-feature "subscription-support";
> >>       when 'derived-from(../../../transport, "nsn:netconf")';
> >>       leaf netconf-endpoint {
> >>         type leafref {
> >>           path "/nncsn:implicit-netconf-receivers/nnccs:implicit-netconf-"
> >>                + "receiver/nnccs:name";
> >>         }
> >>       }
> >>     }
> >>     ...
> >>   }
> >
> > **Martin, are you ok with this.   If you are and there are no other 
> > objections, I will add this and we can be done with this thread.  Which
> > would be progress.   Otherwise, let's just leave things as they are.
> >
> > BTW: adding back address and port also solves the "how do we have a
> > common transport across multiple configured receivers".
> 
> Look at the YANG again, it first defines protocol accessible nodes for
> "receivers" (i.e., distinct transports), and then it augments in a
> leafref to an instance in that list.  I think this is more explicit
> than rules around matching address and port values.
> 
> 
> 
> >> I don't quite understand how the server is supposed to know how to
> >> configure the call-home parameters or the transport parameters, but
> >> at least this would be on par with what you had before.
> >
> > Yes.
> 
> If we do it, the *conf-notif draft would have to explain such details
> in text, since they'd be missing from the YANG module...
> 
> 
> 
> >> > The NETCONF-Notif draft needs to be implemented now for dynamic
> >> subscriptions.
> >> 
> >> From above, and I can't ascertain why this is, when dynamic subscriptions
> >> don't appear to utilize the "netconf" identity in any way...
> >
> > No, but non-YANG Sections 5, 7, & 8 is needed.  Plus many of the examples.
> 
> From above, it seems that we can key everything off if the *conf-notif 
> module listing in yang-library is implemented.
> 
> For servers that only support NETCONF-based dynamic subscriptions (no 
> configured subscriptions), then the ietf-netconf-subscribed-notifications
> module can be listed in yang-library as *not implemented*.  
> 
> For servers that only support NETCONF-based configured subscriptions, 
> then ietf-netconf-subscribed-notifications can be listed in yang-library
> as *implemented*.
> 
> Good?
> 
> 
> 
> >> > An update to NETCONF-notif for configured subscriptions is possible to insert
> >> > the call-home leafref (or insert new grouping).   But this update becomes
> >> > unnecessary if ietf-netconf-server.yang is augmented as described above.
> >> 
> >> Perhaps, but it seems unnatural to do it this way.  What makes sense to me is
> >> for the module that claims to be the transport-binding module to provide the
> >> configuration for binding the transport.
> >
> > At this point we do have a relatively minor difference of option which need
> > not impact the closing the current draft-ietf-netconf-netconf-event-notifications.
> 
> I assume you meant "opinion", and I agree that it's relatively minor, but I think
> that you meant that it doesn't impact the closing of the SN draft, as it certainly
> impacts the closing of the notif drafts, right?
> 
> 
> 
> >>> >> >> That said, I have to say that I'm not entirely sure if I understand
> >>> >> >> if what is planned is legal.  For instance, in a normal NETCONF call
> >>> >> >> -home situation, the NETCONF session begins with both sides sending
> >>> >> >> <hello> messages and then the server waiting for the client to send
> >>> >> >> RPCs, which might include a 5277 <create-subscription>, after which
> >>> >> >> the <notifications> begin to flow.  Is this the same here, or are 
> >>> >> >> you expecting the <notification> messages to start flowing 
> >>> >> >> immediately?
> >> >> >
> >> >> > A subscription-started notification will be sent after the hellos are
> >> >> > successful.  Can you point to something in RFC 6241 which says a 
> >> >> > <notification> can't be sent until an RPC is sent from the client?
> >> >>
> >> >> It's not a very good reference, but I found this (emphasis added):
> >> >>
> >> >>    o  client: Invokes protocol operations on a server.  In addition, a
> >> >>       client can *subscribe* to receive notifications from a server.
> >> >>
> >> >> We should ask the WG.  All I know is that it's always been that the
> >> >> client does something to initiate server behavior.  Admittedly, this
> >> >> is kind of a new thing, and it might be okay, but I think it warrants
> >> >> review by others.
> >> >
> >> > You are welcome to make the request.
> >> 
> >> Eric, you are the Editor.  But beware, this could blow up and we decide to
> >> drop the netconf and restconf protocols bindings entirely and only focus
> >> on transport bindings for things like gRPC and udp-pub-channel.  If NC/RC
> >> are needed, then the server could configure a standard call-home connection
> >> (via the ietf-*conf-server modules) on which the client can start a dynamic
> >> subscription.  Just thinking this might be a better win.
> >
> > Things are far easier with HTTP based transports, because you must get an
> > explicit OK from a subscription-started before sending any <notification>.
> > See RESTCONF-notif for configured subscriptions which used no RESTCONF at
> > all for this function.
> 
> I'm unsure if I understand this.  Can you explain how/why this is so?
> 
> Also, going forwards, please try call out sections when you can.  It took
> me awhile (too long) to see that you meant (I think) section 4.2.
> 
> BTW, in one possible outcome of the current discussions in play, is that
> there may be a multiplicity of "notif" modules, such as:
> 
>   ieft-netconf-notif
>   ieft-netconf-wo-crypto-notif  // better name needed
>   ieft-restconf-notif
>   ieft-restconf-wo-crypto-notif // better name needed
>   ietf-https-notif              // unsure about this one
>   ietf-grpc-notif
>   ietf-udp-notif
>   etc.
> 
> 
> > Eric
> 
> Kent // contributor
> 
> 
> 
>