Re: [Netconf] IETF 101 SN Question 1: Proper designation of receiver

[Kent Watsen <[e-mail]>]

>> >> > Correct.  But your question was "can you use netconf-notif without
>> >> > a leafref
>> >> to...".
>> >> > Needing both drafts is absolutely the case for dynamic subscription
>> >> > support, and ietf-netconf-server would not be needed here.
>> >>
>> >> I read the above a few times, but I'm having a hard time understanding it.
>> >> Can you say it differently or provide an example?
>> >
>> > Dynamic subscriptions over NETCONF requires draft-ietf-netconf-netconf-
>> event-
>> > notifications.
>> 
>> Where is the dependency?  I don't see anywhere in the 3 RPCs and associated
>> error-info definitions that have a reference to the identity in that draft.
>
> The dependency is a document requirements dependency: deployment of NETCONF
> based dynamic subscriptions requires support of both relevant requirements
> sections 5, 7, & 8 from draft-ietf-netconf-netconf-event-notifications in
> addition to draft-ietf-netconf-subscribed-notifications.  

Sure, I get this, but we're talking about if there is YANG-level dependency,
for which I believe we've concluded that the answer is "no".

What this means is, for servers that only want to support NETCONF-based 
dynamic subscriptions (no configured subscriptions), then the 
ietf-netconf-subscribed-notifications module can be listed in yang-library
as *not implemented*.  And for servers that want to support NETCONF-based 
configured subscriptions, then ietf-netconf-subscribed-notifications can
be listed in yang-library as *implemented*.

Looking at the thread that led up to this point, this means that it would
be okay for ietf-netconf-subscribed-notifications to have a leafref to a
global /netconf-server/call-home/netconf-client, while not forcing the
implementation of the ietf-netconf-server module, for servers that only
want to support dynamic subscriptions.

And, to the question that started this fork in the thread "is it possible
that a device wants to use SN but doesn't *implement* ietf-netconf-server",
the answer is "yes".




 
>> >> >> (b) this seems like a possibility, but then I think this make the
>> >> >> case for why a leafref to the global *conf servers definitions won't always
>> >> work.
>> >> >
>> >> > Agree that nothing here will always work.  Deployments commonly will
>> >> > have a heterogeneous mixture of model ecosystem models.
>> >> >
>> >> > This actually makes a *very* strong case for why the leafref should be
>> >> > added as an augmentation from the *conf-server models.  That way
>> >> > leafref augmentations are explicitly tied to the actual implementation of
>> the
>> >> model against which they refer.
>> >>
>> >> Not in the *conf-server models, the augments go into the *conf-notif
>> models, I
>> >> assume that is what you meant.
>> >
>> > My assertion is a good solution would be updating ietf-netconf-server.yang
>> > per what is below.  Note that an answer even further below regarding the
>> > sharing of a single NETCONF session across multiple subscriptions and typical
>> > RFC6241 protocol interactions is assumed.  But we could also insert your
>> > ietf-netconf-server.yang grouping just as effectively where the leafref is seen.
>> >
>> > Anyway here are the following changes which would be made to ietf-
>> netconf-server.yang
>> >
>> >  import ietf-subscribed-notifications { prefix sn; }
>> >  import ietf-netconf-subscribed-notifications { prefix nsn; }
>> >
>> >  feature subscription-support {
>> >    description
>> >        "The 'subscription-support' feature indicates that the NETCONF server
>> >         supports configured subscriptions over call-home connections.";
>> >       reference
>> >        "RFC xxxx: Customized Subscriptions to a Publisher's Event Streams";
>> >     }
>> >
>> > augment "/sn:subscriptions/sn:subscription/sn:receivers/sn:receiver" {
>> >   if-feature "subscription-support";
>> >   when 'derived-from(../../../transport, "nsn:netconf")';
>> >   description
>> >      "This augmentation allows NETCONF specific parameters to be exposed for
>> a receiver.";
>> >    leaf netconf-endpoint {
>> >      type leafref {
>> >        path "/ncs:netconf-server/ncs:call-home/ncs:netconf-client/ncs:name";
>> >      }
>> >      description
>> >        "Remote client which need to initiate the NETCONF transport if an
>> existing
>> > NETCONF session from that client is not available.";
>> >    }
>> >  }
>> >
>> > With such a construct, it is impossible to add a leafref (or grouping) within
>> > ietf-subscribed-notifications unless ietf-netconf-server.yang exists.
>> 
>> True, and thanks for providing a concreate example.  Though I thought we
>> concluded
>> before that there might be cases where the global netconf-server isn't
>> implemented?
>> Now you're okay making that a requirement?  (I'm okay with that, if it works)
>
> I am ok with making it a requirement for drafts

Okay, assuming we resolve the "I'm not entirely sure if I understand if
what is planned is legal" issue discussed below.


> ...subsequent to the current draft-ietf-netconf-netconf-event-notifications.

This is TBD, per the discussion below, but we can try...


>   Either a revision to draft-ietf-netconf-netconf-event-notifications, or
> an update to the ietf-netconf-server.yang.

Right.   But if the dependency only goes one way, then I think the choice
is made for us already. 


>> FWIW, I think that an import statement can also assert that a dependent
>> module is
>> implemented.  For instance, in the below case, the xpath in the leafref forces
>> that the module is implemented:
>> 
>>   module ietf-netconf-subscribed-notifications {
>>     prefix nsn;
>>     import ietf-netconf-server { prefix ncs; }
>>     import ietf-subscribed-notifications { prefix sn; }
>> 
>>     augment "/sn:subscriptions/sn:subscription/sn:receivers/sn:receiver" {
>>       if-feature "subscription-support";
>>       when 'derived-from(../../../transport, "nsn:netconf")';
>>       description
>>         "This augmentation allows NETCONF specific parameters to be
>>          exposed for a receiver.";
>>       leaf netconf-endpoint {
>>         type leafref {
>>           path "/ncs:netconf-server/ncs:call-home/ncs:netconf-client/ncs:name";
>>         }
>>         description
>>           "Remote client which need to initiate the NETCONF transport if
>>            an existing NETCONF session from that client is not available.";
>>       }
>>     }
>>     ...
>>   }
>> 
>> I prefer this arrangement because it gives tangible meaning for what it means
>> to *implement* the netconf-subscribed-notifications module.
>
> I understand.  As long as we make the choice as to where to land this future
> leafref after the current draft-ietf-netconf-subscribed-notifications completes,
> I am good.

Pending the discussion below...



>> >> >> This is why I
>> >> >> was thinking before that your modules might themselves *use* the
>> >> >> *conf- server-groupings (while pruning out unneeded parts, e.g., the
>> >> >> "listen" subtree), so that it's independent of what the system has
>> >> >> implemented at the global level.
>> >> >
>> >> > If you have 500 subscriptions, you then have to populate 500 identical
>> >> groupings.
>> >>
>> >> No, you have one grouping, with 500 /netconf-server/call-home/netconf-
>> client
>> >> instances.
>> >
>> > Yes.    But I don't know why someone would voluntarily do add 500 repeated
>> > elements to a configuration datastore.
>> 
>> At first I was going to point out that, even if using to global netconf server
>> container, there would still be 500 /netconf-server/call-home/netconf-client
>> instances, but in looking ahead, I'm wondering if I misunderstand the intended
>> relationship between transports, subscriptions, and receivers.
>> 
>> If it turns out that receivers from different subscriptions can leafref the
>> same /netconf-server/call-home/netconf-client, then the 500 becomes 1, and
>> the duplicate data-entry concern goes away.
>
> Exactly.  This has always been the objective.

Okay.  Sorry for being slow to get this.  Please take a close look at SN draft
to ensure this is super clear there.



>> >> >  And yes this is possible.  But it makes the part of me which likes
>> >> > Normalized  data quite uncomfortable.
>> >> >
>> >> > But as I said before, it the WG wants such redundancy, fine.  Either
>> >> > choice need not impact decisions as part of LC.
>> >>
>> >> I don't believe that is a WG-preference thing, so much as an outcome of the
>> >> current design, which is that each receiver for each subscription has its own
>> >> state-machine and protocol messages.  There is no sharing; no two receivers
>> can
>> >> use the same RFC 6241 NETCONF session, which effectively translates to
>> each
>> >> receiver having its own /netconf-server/call-home/netconf-client instance,
>> >> right?
>> >
>> > This is incorrect.    Protocol and state-machine messages have been
>> decoupled
>> > from the transport session.
>> 
>> As mentioned above, I'm wondering if I misunderstand the intended
>> relationship
>> between transports, subscriptions, receivers, and maybe publishers too.  Can
>> you put together a diagram that describes these relationships?
>
> A configured subscription on a publisher can have many receivers.
>
> A configured subscription on a publisher may only use one type of transport (and one type of encoding).
>
> A configured receiver can receive information from multiple configured subscriptions on a single transport session from a publisher.

I'd like to have statements like these in the SN draft.  Maybe as part 
of the term definitions, but that might be too much information (busy)
for terms. The info could be sprinkled throughout the doc, but I wonder
if that might not already be the case and, if so, then it didn't work
out too well before (witness my confusion here), so perhaps some other 
section would be better?



>> > I am not sure why you think that subscriptions are unable to use a common
>> > NETCONF session?   Implementations of dynamic NETCONF subscriptions
>> have
>> > been doing this for years.    Subscription multiplexing of configured and
>> > dynamic subscriptions over a common transport is a pre-requisite for
>> > solution scalability.
>> 
>> I think because its underspecified in the SN draft, and there was confusion
>> with the address and port leafs, and ietf-netconf-subscribed-notifications
>> only defines an identity (no configuration data model).
>
> In a parallel thread to Martin, I have added a sentence aimed here.   Beyond
> that, configuration data model forces choice of the identity for the configured
> subscription.

In that thread, you wrote:

"""
I have added the following to the NETCONF-Notif document section on configured subscriptions:

"It is possible to have multiple configured subscriptions sharing a common transport to a single receiver.  The method of identifying that a receiver happens to be the same as used with another subscription is left up to implementers of this specification."
"""

This is a good sentence (assuming the discussion regarding *why* this is simpler pans out),
but shouldn't it be in the SN document (not netconf-notif)?



>> Okay, the answer is that its considered "simpler" to use a single kind
>> (not instance) of transport.  So, the outcome is, if one receiver of a
>> subscription is using a NETCONF-based transport, then all the other
>> receivers of that subscription MUST also be using a NETCONF-based
>> transport, albeit a different instance of a NETCONF-based transport
>> (as it would be redundant otherwise).  Correct?
>
> Yes
> 
>
>> Assuming this is the case, my question is, why is this "simpler"?  I mean,
>> assuming an event occurs that a subscription matches, the publisher will
>> encode a notification message to send, and then iterate over its list of
>> receivers, sending the same encoded-message to each.  But why is it less
>> simple if different transports (netconf, restconf, etc.) are used?
>
> As can be heard in the recording, and seen on dozens of WG emails, these
> issues were deeply debated.  As can been seen my slight preference actually
> was different transports.  And that is how earlier versions of the model
> covered the issue.  However the WG chose a single transport for rational
> reasons at and after IETF 100.  The issue was closed and the drafts
> updated accordingly.  
 
Eric, I'm asking for a technical answer.  In a nutshell, what are
the "rational reasons"?   Yes, I recall your having a preference for
heterogeneous transports...


>> BTW, separately, I kind of but not really understand why there is a desire
>> for the fixed encoding for all the receivers in a subscription.  I understand
>> the efficiency angle (see prev paragraph), but I get stuck on the idea that,
>> if there is a *need* to send a different encoding (e.g., "encode-json"),
>> another encoded message structure is going to have to be created anyway; it
>> seems like the same number of instructions from that perspective.  Then it
>> goes to looping over one-subscription-tree or one-tree-per-encoding.  Okay,
>> then, what makes it better? 
>
> Some implementations have claimed it is easy to bind the subscription with
> the encoding, and difficult to perform filtering before the encoding.  So
> it is better to force this separation.

Okay.  (but see next paragraph).


>> The only thing I can come up with is that it
>> might be difficult otherwise to express in YANG what encoding is being used
>> for that receiver.  For instances, if there is a leafref to /restconf-server\
>> /call-home/restconf-client, nowhere is there an "encoding" field.  Hmmm,
>> maybe the encodings a restconf server supports could be specified at a
>> higher level (e.g., /restconf-server/encodings/...), and then it would be
>> known, on a per-receiver basis, what encoding is used (netconf is always
>> xml, restconf is per configuration).  Anyway, I'm just wondering if this
>> is why the encoding for all the receivers in a subscription must be the
>> same, or is it something else?

I just sent a question to the WG regarding if ietf-restconf-server should
have a way configure which encodings it supports.  If this pans out, the
impact here is that we might want a "must" statement to ensure that the
selected encoding is supported by the leafref-ed /rcs:restconf-server/
instance.



>> In this particular fork in the thread, I think that we're discussing the merits
>> if leafref-ing vs using a grouping.  If it is the case that the same transport
>> can be used across subscriptions, then 1) it swings things back to leafref
>> approach being needed and 2) this fork in the thread is done.  [Assuming that
>> it’s a leafref, we still need to finalize if it's a leafref to the global
>> server instance or some SN-specific instance.]
>
> I believe leafref is good.  And as long as the leafref is inserted after
> the current drafts in WGLC complete, I am good.

Yes, leafref seems needed.  Whether the leafref is global vs. local, and to what
the leafref points to, are still TBD.



>> >> >> There is a difference between a server not *implementing* a ietf-*conf-
>> >> >> server module and the *conf-notif not *using* the *conf-server-grouping
>> >> >> statements.  My suggestion has been, that the *conf-notif drafts should
>> >> >> have their own lists of netconf-servers (via "uses" statements), and
>> >> >> thereby not be dependent on the existence of a global ietf-*conf-server
>> >> >> instance (which may not exist).
>> >> >
>> >> > While technically correct, there are several reasons why this is problematic.
>> >> > (1) redundancy (see the 500 above)
>> >>
>> >> This is a non-issue (see above)
>> >
>> > This is still an issue, as the drafts in WGLC support a single NETCONF session
>> > for all subscriptions and normal protocol operations.
>> 
>> As said before, sharing the same transport across subscriptions wasn't clear
>> to me before.  Still, even as 500 becomes 1, there remains the discussion if
>> the one is the global server instance or some SN-specific server instance.
>
>Same comment as above.

Which is that you're okay with the *conf-notif drafts necessitating the existence
of /*conf-server/ instances (i.e., the ietf-*conf-server module is implemented)
and, of course, you're hoping that this dependency can be introduced in some
future bis version of the *conf-notif drafts.



>> >> > (2) availability of the group means that a platform will have exposed
>> >> > *conf-server.  Explaining that a model is only available for its
>> >> > grouping would be quite a confusing deviation.
>> >>
>> >> No, it's easy, this is the difference between a module being *implemented*
>> >> or not.  The implementation status of each module is yang-library.
>> >
>> > Yes, what you say is possible.  It is also more complex.
>> 
>> Not just possible, it is actually how it happens.  The client-server modules
>> are highly sensitive to implementation status.  FWIW, I never expect the
>> ietf-*conf-client modules to ever be implemented, and the ietf-*conf-server
>> modules to be implemented "sometimes".  FWIW, the global server instances
>> we keep talking about only happen *if* the ietf-*conf-server modules are
>> implemented.
>
> I have seen implementations of YANG models without having a yang-library.
> I prefer a yang-library of course.

From a SDO perspective, yang-library is expected to be implemented (it's a
MUST in RFC 8040 and in nmda-restconf).  We should fully assume that the
server implements yang-library.  That said, if I were the implementer of
a receiver that does a dynamic subscription, I would probably write the
code to just send the establish-subscription request and check to see 
if the server returned an <rpc-error>, without first checking if the
module is listed in yang-library...


>> > So can we take out address and finally be done?   That would be a good
>> thing.
>> 
>> Yes, take out the address leaf but I think that, if we want to progress the
>> SN draft along with a transport binding definition that doesn't depend on
>> the ietf-*conf-server modules, then we might define something else like:
>> 
>>   module ietf-netconf-no-crypto-subscribed-notifications {
>>     prefix nncsn;
>>     import ietf-subscribed-notifications { prefix sn; }
>> 
>>     container implicit-netconf-receivers {
>>       list implicit-netconf-receiver {
>>         key name;
>>         leaf name { ... }
>>         leaf address { ... }
>>         leaf port { ... }
>>       }
>>     }
>>     augment "/sn:subscriptions/sn:subscription/sn:receivers/sn:receiver" {
>>       if-feature "subscription-support";
>>       when 'derived-from(../../../transport, "nsn:netconf")';
>>       leaf netconf-endpoint {
>>         type leafref {
>>           path "/nncsn:implicit-netconf-receivers/nnccs:implicit-netconf-"
>>                + "receiver/nnccs:name";
>>         }
>>       }
>>     }
>>     ...
>>   }
>
> **Martin, are you ok with this.   If you are and there are no other 
> objections, I will add this and we can be done with this thread.  Which
> would be progress.   Otherwise, let's just leave things as they are.
>
> BTW: adding back address and port also solves the "how do we have a
> common transport across multiple configured receivers".

Look at the YANG again, it first defines protocol accessible nodes for
"receivers" (i.e., distinct transports), and then it augments in a
leafref to an instance in that list.  I think this is more explicit
than rules around matching address and port values.



>> I don't quite understand how the server is supposed to know how to
>> configure the call-home parameters or the transport parameters, but
>> at least this would be on par with what you had before.
>
> Yes.

If we do it, the *conf-notif draft would have to explain such details
in text, since they'd be missing from the YANG module...



>> > The NETCONF-Notif draft needs to be implemented now for dynamic
>> subscriptions.
>> 
>> From above, and I can't ascertain why this is, when dynamic subscriptions
>> don't appear to utilize the "netconf" identity in any way...
>
> No, but non-YANG Sections 5, 7, & 8 is needed.  Plus many of the examples.

From above, it seems that we can key everything off if the *conf-notif 
module listing in yang-library is implemented.

For servers that only support NETCONF-based dynamic subscriptions (no 
configured subscriptions), then the ietf-netconf-subscribed-notifications
module can be listed in yang-library as *not implemented*.  

For servers that only support NETCONF-based configured subscriptions, 
then ietf-netconf-subscribed-notifications can be listed in yang-library
as *implemented*.

Good?



>> > An update to NETCONF-notif for configured subscriptions is possible to insert
>> > the call-home leafref (or insert new grouping).   But this update becomes
>> > unnecessary if ietf-netconf-server.yang is augmented as described above.
>> 
>> Perhaps, but it seems unnatural to do it this way.  What makes sense to me is
>> for the module that claims to be the transport-binding module to provide the
>> configuration for binding the transport.
>
> At this point we do have a relatively minor difference of option which need
> not impact the closing the current draft-ietf-netconf-netconf-event-notifications.

I assume you meant "opinion", and I agree that it's relatively minor, but I think
that you meant that it doesn't impact the closing of the SN draft, as it certainly
impacts the closing of the notif drafts, right?



>>> >> >> That said, I have to say that I'm not entirely sure if I understand
>>> >> >> if what is planned is legal.  For instance, in a normal NETCONF call
>>> >> >> -home situation, the NETCONF session begins with both sides sending
>>> >> >> <hello> messages and then the server waiting for the client to send
>>> >> >> RPCs, which might include a 5277 <create-subscription>, after which
>>> >> >> the <notifications> begin to flow.  Is this the same here, or are 
>>> >> >> you expecting the <notification> messages to start flowing 
>>> >> >> immediately?
>> >> >
>> >> > A subscription-started notification will be sent after the hellos are
>> >> > successful.  Can you point to something in RFC 6241 which says a 
>> >> > <notification> can't be sent until an RPC is sent from the client?
>> >>
>> >> It's not a very good reference, but I found this (emphasis added):
>> >>
>> >>    o  client: Invokes protocol operations on a server.  In addition, a
>> >>       client can *subscribe* to receive notifications from a server.
>> >>
>> >> We should ask the WG.  All I know is that it's always been that the
>> >> client does something to initiate server behavior.  Admittedly, this
>> >> is kind of a new thing, and it might be okay, but I think it warrants
>> >> review by others.
>> >
>> > You are welcome to make the request.
>> 
>> Eric, you are the Editor.  But beware, this could blow up and we decide to
>> drop the netconf and restconf protocols bindings entirely and only focus
>> on transport bindings for things like gRPC and udp-pub-channel.  If NC/RC
>> are needed, then the server could configure a standard call-home connection
>> (via the ietf-*conf-server modules) on which the client can start a dynamic
>> subscription.  Just thinking this might be a better win.
>
> Things are far easier with HTTP based transports, because you must get an
> explicit OK from a subscription-started before sending any <notification>.
> See RESTCONF-notif for configured subscriptions which used no RESTCONF at
> all for this function.

I'm unsure if I understand this.  Can you explain how/why this is so?

Also, going forwards, please try call out sections when you can.  It took
me awhile (too long) to see that you meant (I think) section 4.2.

BTW, in one possible outcome of the current discussions in play, is that
there may be a multiplicity of "notif" modules, such as:

  ieft-netconf-notif
  ieft-netconf-wo-crypto-notif  // better name needed
  ieft-restconf-notif
  ieft-restconf-wo-crypto-notif // better name needed
  ietf-https-notif              // unsure about this one
  ietf-grpc-notif
  ietf-udp-notif
  etc.


> Eric

Kent // contributor