Re: [Netconf] LC on subscribed-notifications-10

["Eric Voit (evoit)" <[e-mail]>]

Hi Kent,

Thanks again for the comments.  See [snip] where I removed closed items.   Everything else is in-line with <Eric3>...

From: Kent Watsen, April 17, 2018 6:54 PM

[just back from PTO]

Hi Eric,

Please see my follow-ups below: <Kent2>

Kent // as a contributor


On 4/9/18, 6:31 PM, "Eric Voit (evoit)" <[e-mail]<mailto:[e-mail]>> wrote:

Hi Kent,

Thanks for the feedback.  Look for thoughts at <Eric2> In-line...

Also everything documented below which made it into the working copy can be seen at:
https://github.com/netconf-wg/rfc5277bis/blob/master/draft-ietf-netconf-subscribed-notifications-12.txt<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_netconf-2Dwg_rfc5277bis_blob_master_draft-2Dietf-2Dnetconf-2Dsubscribed-2Dnotifications-2D12.txt&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=6hc_72tEh2fXZOBFRTk-tRPRujVYmmltu_pxFONHQCs&s=UONL_m1_J94PYNQraXzNK0Ma1G3hKFPThShVSIjd0M0&e=>





[snip]





>   I'm confused by the diagram and subtitle's use of the word "receiver",

>   when the first sentence of the paragraph above says that the SM is for

>   the publisher...



This is for the Publisher: the publisher must maintain the state of whether a receiver is currently active or suspended.



I changed the title to: "Publisher's state for a dynamic subscription" which should help here.   Other reviewers requested the addition of the word receiver to the states themselves.  This is so people could make a 1:1 correlation with the states of the configured subscription state machine.



<KENT>title is better, though maybe "Publisher's state for a receiver's dynamic subscription" would be better?  (not sure)



<Eric2> I kind of like the simplicity of the current text.  Will change if you have a very strong preference.
<Kent2> not really, but keep an eye on if anyone stumbles over this section



<Eric3> Will do...





>   Only two notifications?



Only two notifications indicate a change in the state of the subscription.

<KENT> okay, but then can you add somewhere that only two notifications are represented because they're the only ones indicating a change in the state of the subscription?



<Eric2> Text now says:



The two state change notifications "subscription-suspended" and "subscription-resumed" are shown.  These are under the control of a publisher. These are the only two state change notifications which indicate a change in state of a dynamic subscription.

 <Kent2> better, but s/only two state/only state/ - right?



<Eric3> I should have been more explicit and said these are the only two state change notifications which signal a new state for an ACTIVE dynamic subscription.  There are other state change notifications used, but they don’t indicate a state change when ACTIVE.



In any case, ‘two’ isn’t in the draft text itself, and the key question is hit within the point below.





>   Looking at the graphic, how is the reader to

>   distinguish these as notifications?



Added a * to the two notifications, and text at the bottom of the drawing which says:



* indicates a state-change-notification



<KENT> better, but somehow not satisfying…  Mentally removing these two notifications from the diagram entirely, I notice that there is no other arrow going from ACTIVE to SUSPENDED; it seems like you might need one, perhaps labeled something like "<internal state event>"?  Assuming this is done, could we then remove listing these notifications from the diagram?



<Eric2> My reading of your comment is that you don’t like the identification of the “suspend subscription” transition cause via the “subscription-suspended*” notification.   To clarify, I have removed all state change notifications from the diagram, and described them in the text below...


                        .........
                        : start :
                        :.......:
                            |
                   establish-subscription
                            |
                            |   .------modify-subscription-------.
                            v   v                                |
                      .-----------.                        .-----------.
           .--------. | receiver  |--suspend-subscription->| receiver  |
       modify-       '|  ACTIVE   |                        | SUSPENDED |
       subscription   |           |<--resume-subscription--|           |
           ---------->'-----------'                        '-----------'
                            |                                    |
                 delete/kill-subscription                   delete/kill-
                            |                               subscription
                            v                                    |
                        .........                                |
                        :  end  :<-------------------------------'
                        :.......:

          Figure 1: Publisher's state for a dynamic subscription

Of interest in this state machine are the following:
...(snip)...

   o  A publisher may choose to suspend a subscription, this is notified to a subscriber with a "subscription-suspended" state change notification.

   o  A resume subscription state change is notified to a subscriber "subscription-resumed". There are no direct external controls over resuming a subscription other than for a subscriber to attempt the modification of a subscription in a way which reduces the resources consumed.

<Kent2> er, you said that the all state change notifications were removed from the diagram, but the diagram still looks the same to me, listing "suspend-subscription" and "resume-subscription".  Also, which is it, "resume-subscription" (in the diagram) or " subscription-resumed" in the text above?

<Eric3>  I read your point <KENT> as you wanted to show the cause of the state transition (e.g., suspend subscription) rather than the resulting notification (e.g., “subscription-suspended”).  So what I was trying to do with the top part of the picture was to remove the names of the state change notifications, and just show the event which drives the subscription from one state to the other.  Otherwise the document would be confusing/overly-verbose if it tried to show both on a link (e.g., suspend subscription “subscription-suspended”) .

I can go back to the ‘*’ version of the diagram if you prefer.  Do you have a different suggestion?







[snip]



>   Are your tree diagrams dynamically-generated?  - is there any concern

>   that they are out-of-date?



Generated from Pyang.   Manually snipped from the output.  Concerns are discussed more below.   Next drafts I am certainly changing my integration environment.

<KENT> the question more regards if they've been generated (via pyang or whatever) recently…



<Eric2> With the tool Martin pointed me to for automatically generating to a fixed column width, life is much easier now.
<Kent2> so now the tree-diagrams are automatically generated?



<eric3> Every tree was automatically generated just before posting.  The only hand editing was to remove extra “|” connectors between discrete Notifications & RPCs.



[snip]



>   Also, just so I'm clear, each transport-binding needs to indicate if and

>   how the yang-data structs are returned, right?  Where is this done in

>   the netconf-notif and restconf-notif drafts?



Yes

<KENT> what about the second question?



<Eric2> In the netconf-notif draft, it is in Section 8.   The text including this is not yet published in Restconf-notif.
<Kent2> please add it to the restconf-notif doc now (strike while the iron is hot)



<Eric3> It will be my next action after this email.



>   I can't find any examples of these errors in use.  The

>   netconf-event-notifications draft only has examples for

>   the "establish-subscription-error-datastore" and

>   "modify-subscription-error-datastore" errors.



Figure 10 in the netconf-event-notifications draft works equally for subscribed-notifications, as well as yang-push.   I have identified that example in that document as being relevant to either streams or datastores with the sentence in that draft: "This subscription may have been to either a stream or a datastore."

<KENT> okay…



Here this document, I have added the sentence:



To see a NETCONF based example of an error response from above, see [I-D.draft-ietf-netconf-netconf-event-notifications], Figure 10.

<KENT> good.  Better would be to also have a reference to a RESTCONF-based example.



<eric2> Understood.  Didn’t know how to do that and not introduce a publication dependency.
<Kent2> it won't block publication if it's an Informational reference, but I continue to be uncomfortable about the restconf-notif documents not progressing.  It doesn't have to be published with these (though it would be better if it were), but it should be tracking the latest developments and what not…

<Eric3> I will do this first thing in the AM.



[snip]

>   Regarding " provides an example of such an extension" - which section?



Revised text to:



The YANG model [I-D..ietf-netconf-yang-push] Section 4.1,  provides many such extensions, this includes the augmentation of "/sn:modify-subscription/sn:input/sn:target".

<KENT> better, but:

1) I didn't review yang-push, but I hope that someone pointed out that section 4.1 needs to point to section 5 and, additionally perhaps section 5 should be moved to section 4.5…



<Eric2>   I think you are suggesting that the YANG push tree model in 4.1 needs to point to the YANG model section number.  And that perhaps the YANG model section itself shouldn’t be in an independent top level section, but rather fall into section 4.     I have no issues with that.   **Alex, do you want to update, this should be a very minor update?
<Kent2> Alex responded in another email





[snip]



>  Do the  netconf-notif and restconf-notif drafts satisfy this requirement?



Yes

<KENT> good





> where?



Netconf-notif supports interleaving of requests as described in Section 3.

<KENT> okay



Restconf-notif doesn’t need to explicitly call for pipelining support as it is a basic capability of HTTP.

<KENT> but the question isn't about pipelining.  Even NETCONF supports pipelining, something extra is needed to support "intermixing", right?



<Eric2> Yes.  And we do have that intermixing included in document requirements within this section.   Text says:



“In all cases, a single transport session MUST be capable of supporting the intermixing of RPCs and notifications from different subscriptions.”



I think that change was made after conversations with Martin, so it didn’t come back explicitly via this subthread.


<Kent2> okay, but the question isn't what's in this draft, but where in the restconf-notif draft is this requirement satisfied?

<Eric3> Per above, RESTCONF-notif is my first thing for tomorrow.   I am now feeling more comfortable I won’t be churning contexts there as the rest of the documents’ frameworks settle.



[snip]



>   Also, I'm unsure about the "subscription-state-notif" extension, how

>   is it expected to be used by a YANG processor?



Per above, it ensures that these YANG notifications if encoded in XML are not placed onto the NETCONF stream.



<KENT> actually, I thought that before it only said that the Subscription State Notifications (s2.7) were not placed into the NETCONF stream???



<Eric2> Subscription state notifications are a type of YANG notification, as they are encoded in the YANG model.  Per the London WG discussion on slide “Question 2”, I believe it easier to mark these.  See next comment below.



<Kent2> no, you're missing my point.  before you wrote that only Subscription State Notifications were not placed in the NETCONF stream, but that seems misleading if you also mean to include any notifications having the "subscription-state-notif" extension.



<Eric3> I think we can let this get picked up by the “Question 2: Subscription State Notifications” thread which Alex needs to re-frame (per your email earlier today).





The three choices seem to be:
(a) current solution

(b) hardcode the these notifications so none ever go on the NETCONF stream

(c)  make the extension “exclude-from-NETCONF-stream”.  As it is quite possible that other drafts will want to do that.



I am good with any of these.  But the first seems the cleanest, and most self contained.  Let me know it the current doesn’t work for you.



<ALEX> Just to add on:  A reason for the extension (and different solutions were discussed at different points in time) was that since this is a “meta-notification”, it should be treated differently from other notificaitons.  For example, a subscriber should receive these even if not explicitly subscribing to them – they are simply part of the “control protocol” for managing the subscriptions.   They also apply if a subscriber subscribes to something other than the NETCONF stream.

</ALEX>



<KENT> yes, Alex, part of the control protocol, this is why I'm thinking maybe Eric's choice (b) is best.  Is this being discussed elsewhere?



<Eric2> We had a discussion on this in London:

https://youtu.be/KJtg-J-6CZM?t=1963<https://urldefense.proofpoint.com/v2/url?u=https-3A__youtu.be_KJtg-2DJ-2D6CZM-3Ft-3D1963&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=6hc_72tEh2fXZOBFRTk-tRPRujVYmmltu_pxFONHQCs&s=i43hXrkycPDoO1v9EUv0DdOmQe7TkbG9rpTtams9rMI&e=>

As there was no comment in the room, I was hoping we had actually had some form of consensus between us on (a).   So I hadn’t spun up a separate question on this yet.



But it seems there is an issue.  I will open up a thread now.



<Kent2> I just responded to that "thread"



<Eric3>  Yes, let’s track there.



[snip]



When any such configured subscription receivers become ACTIVE, buffered event records (if any) will be sent immediately after the “subscription-started” notification.  The first event sent will be the most recent following the latest of four different times: the "replay-log-creation-time", "replay-log-aged-time", "replay-start-time", or the most recent publisher boot time.

<KENT> I don't understand the 2nd sentence here



<Eric2>   Rewrote to: “The leading event record sent will be the first event record subsequent to the latest of four different times: the "replay-log-creation-time", "replay-log-aged-time", "replay-start-time", or the most recent publisher boot time.”
<Kent2> Still confusing.   I think that you may need to expand this into a paragraph where you more carefully discuss the four times and how they're used here, or perhaps use an example to explain it…

<Eric3>  Added the descriptive paragraph requested in the middle of the three paragraphs below...

It is possible to place a start time on a configured subscription.  This enables streaming of logged information immediately after restart.

Replay of events records created since restart can be quite useful.  This allows event records generated before transport connectivity was supportable by a publisher to be passed to a receiver.  In addition, event records logged before restart are not sent.  This avoids the potential for accidental event record duplication.  Such duplication might otherwise be likely as a configured subscription’s identifier before and after the reboot is the same, and there may be not be evidence to a receiver that a restart has occurred.  By establishing restart as the earliest potential time for event records to be included in notification messages, a well-understood timeframe for replay is defined.

Therefore, when configured replay subscription receivers first become ACTIVE, buffered event records (if any) will be sent immediately after the "subscription-started" notification.  And the leading event record sent will be the first event record subsequent to the latest of four different times: the "replay-log-creation-time", "replay-log-aged-time", "replay-start-time", or the most recent publisher boot time.





All other replay functionality remains the same as with dynamic subscriptions as described in Section 2.4.2.1

<KENT> I'm not sure I like having to look at 2.4.2.1 and trying to figure out what this means.  Can you make this more explicit or, since 5.6 is pretty small, copy the parts into this section?



<Eric2> I initially had all the text in 2.4.2.1.  But this hid the fact that you can do replay on a configured subscription.  So your comment above lead to this section being introduced.  Which is a good thing.   But as 2.4.2.1 is not very small, to me it feels like repeating all that text here might be overkill.



<Kent2> hmmm, maybe factor the relevant text in 2.4.2.1 to yet another section and have both refer to it instead?



<Eric3> If we move the relevant Replay to a new Section just before 2.6, then we create harder difficulties.   For example, there are establish-subscription errors specific to replay in Section 2.4 -- we shouldn’t talk about these errors before replay is introduced.   And if we tried to fix this by replicating a different table for establish-subscription errors related to replay, we would confuse to readers.  I believe the current partitioning is a reasonable approach.





[snip]





>   Also,

>   does it include configured subscriptions that are currently not

>   active for whatever reason?



Yes.   First paragraph above uses the word ‘all’.

<KENT> but if not active, aka operational, why are they in the operational datastore?  This needs to be explained.



<Eric2> Two thoughts.  First, a configured subscription can be VALID without having any ACTIVE receivers. Second, the status of a configured subscription is a “config false” element which includes both the INVALID and CONCLUDED states that are not configurable.  (text below)

<Kent2> see below



Also, maybe you need to be more explicit than just having "all" …



<Eric2> You are correct, some more detail is needed.  And more description of the counters is needed.  I shook things up.  Here is what it says now:



In the operational datastore, the container "subscriptions" maintains the state of all dynamic subscriptions, as well as all configured subscriptions.  Using datastore retrieval operations, or subscribing to the "subscriptions" container via [I-D.ietf-netconf-yang-push] allows the state of subscriptions and their connectivity to receivers to be monitored.



Each subscription in the operational datastore is represented as a list element. Included in this list are event counters for each receiver, the state of each receiver, as well as the  subscription parameters currently in effect. The appearance of the leaf "configured-subscription-state" indicates that a particular subscription came into being via configuration.  This leaf also indicates if current state of that subscription is VALID,  INVALID, and CONCLUDED.



To understand the flow of event records within a subscription, there are two counters available for each receiver.  The first counter is "pushed-notifications" which shows the quantity of events actually identified for sending to a receiver.  The second counter is "excluded-notifications" which shows event records not sent to receiver.  "excluded-notifications" shows the combined results of both access control and per-subscription filtering.  For configured subscriptions, counters are reset whenever the subscription is evaluated to VALID (see (1) in Figure 8).



Dynamic subscriptions do not appear outside of the operational datastore, and are removed from the operational datastore once they expire (reaching stop-time) or when they are terminated.



<Kent2> better.  Though I wonder if it's possible for "excluded-notifications" to have an accurate count.  Assuming access-control affects the subscription request (not per-notification filtering), then this info may not be known.



<Eric2> It should be knowable.  You can count how many were sent to a receiver. You can tell how came in from event stream since subscription start.    If the number of exclusions is protected information, the “excluded-notifications” object could be access protected.



[snip]



>   The 2nd paragraph would make more sense if I was looking at a tree

>   diagram.  But then I realize that this would be the same tree-diagram

>   that should've been presented in Configured Subscriptions.



The tree is in the subscriptions container section just below.  I will gladly reference it wherever it ends up.

<KENT> you already need to be referring to it regardless.  As for where it is, see my previous comment on this topic



<Eric2> References to Figure 20 has been made.   If the tree must be moved up, it can be.   I think it fits better where it is.

<Kent2> reference by figure-number is okay, but you might want to also again which section the figure appears in.





<Eric3>  Now says:


   A tree diagram describing these parameters is shown in Figure 20
   within Section 3.3.





[snip]





>   This tree-diagram does not match what I generate.  This indicates

>   that the tree diagrams are not being dynamically-generated.  I

>   strongly suggest updating your build script to dynamically generate

>   the tree diagrams.  We cannot afford to have them be out of alignment.



At the WG request, I segmented the YANG tree into different sections.  However I do not have the tooling which automatically extracts portions of the YANG tree.



Is there a git repository which recommends a continuous integration for sub portions of a YANG tree?  For future drafts, I have certainly built a strong desire for such a continuous integration environment.



<KENT> I have my own tooling using Makefiles and shell scripts to dynamically generate and include the tree diagrams every build.  You should be looking to create similar now, for this draft (not next drafts).   Again, we cannot afford for these things to get out of alignment, and these drafts still have a way to go yet…



<Eric2> I have not seen automated tooling from pyang which pulls individual RPCs and Notification Trees into extracts.  Not finding a way to do this with –tree-path, I tried guessing.  But didn’t get there.   As the majority of my trees are RPCs and Notifications, I don’t see a fully automated solution available as yet.



<Kent2> right, this is why I wrote "and shell scripts".  I sometimes post-process the pyang tree output to e.g., cherry-pick something…





[snip]





(b) I have no insights on how to pull portions of a tree into a XML document.   Is there a tool site which provides this?

<KENT> my Makefiles call a shell script to do the insertions…



<Eric2>  My environment has certainly shown itself to be insufficient.  If WG requires Makefiles rather than what many of us use (yes, I really built most of this via NOTEPAD++, and I know there are multiple others doing this), then the WG should document expected toolsets to be used.  Note that based on my pain here that I do have my eye on an alternative tooling after these 3 drafts complete WGLC.  If there is a lull subsequent review cycles, perhaps I will convert if my experiences with the next set of drafts work.



<Kent2> The WG does not require Makefiles or any particular environment, but we do require that the drafts are accurate, and history has shown that tree-diagrams/modules/examples are commonly not accurate and yet easy to automate generation/validation for.   Most people doing this use Makefiles, and you're welcome to use or pilfer content from mine if you like (e.g., the zerotouch draft).   That said, I am working on something that might lead to tooling modifications, though I'd imagine it might take a couple years to get it to market…



<Eric3> Right now I am depending on:



pyang -f tree  ietf-subscribed-notifications.yang ietf-yang-push.yang  --tree-line-length 69



As I learn a new YANG tooling environment as I work with the notification-messages draft, I might be able to back-port my learnings with shell scripts.





The delta I see is “rw” vs “ro”.  Fixed now.   I have brought in the current tree.

<KENT> better, but not a lasting fix



<Eric2> Would the NETMOD WG be willing to put together a wiki of the development tool recommendations?    As a user, I know it would be welcomed.



<Kent2> Currently folks just copy/paste/modify with starting a new draft.  In lieu of anything else, I recommend folks copy from one of my current draft repos (as I'm always tweaking to make better).





[snip]



<KENT> better, though I'm unsure the "none" nodes need to be listed.



 <Eric2> The template text  “These are the subtrees and data nodes and their sensitivity/vulnerability” appears to make the list of all node mandatory.  As this was not your intent, I pulled the “none” out.



<Kent2> not my template, so it's not so much a question of my intent, per se.   But what I do is to call out the nodes with special considerations (i.e., having an NACM extension statement) and state that everything else isn't noteworthy.



<Eric3>  Ok,  I tweaked the template to say:

These are the subtrees and data nodes where there is a specific sensitivity/vulnerability:



[snip]





>   Re: the 6th paragraph, I'm surprised that requirements for transport-

>   bindings wasn't discussed before in its own section.  It seems like

>   a new thing here, that a receiver's transport might not be secure.

>   I'm okay with and support this, btw, as its sometimes better to

>   offload devices thru the use of a local collector node, for which

>   encryption may not be needed...



Agree with your comments.

<KENT> but where's the change?  Shouldn't this have been discussed

previously in the draft somewhere?



<Eric2> The vast majority of transport binding discussions are addressed in the transport document.  So I see this as guidance to a documenter of a transport document.  Perhaps that is unnecessary for this document, and the paragraph should be removed.  I would be fine with that.



<Kent2> wait, I don't think you can offload transport-requirements to the transport-binding documents.   I think that this document needs to define the requirements and the transport-binding documents then show how they adhere to them.   Does this make sense?



 <Eric3>With the varied transports of NETCONF, HTTP/RESTCONF, UDP, CoAP already in drafts my belief is that only a high level subset of transport requirements spanning the universe of potential transports can potentially be abstracted in this document.  The secure transport requirement is one such example, and that is a recommendation.  The Security Considerations section is a good place for that one.  Beyond the security recommendation there aren’t too many transport independent possibilities.   I did just added one new transport requirement to the very end of “Event Streams” section though (which perhaps wasn’t explicit enough elsewhere).  This requirement is:



“Event records MUST NOT be delivered to a receiver in a different order than they were placed onto an event stream.”



What other transport-independent transport requirements might there be which are not already documented?



Stepping back, I see the transport draft plus this drafts providing the aggregate set of requirements for a full solution.  And I had thought it would be up to the draft authors plus WGs to validate that the sum of the documents is sufficient.



[snip]



>   Re: 2nd to last paragraph, what is the " very-secure" tag?



Removed, and the overall points moved up into template.  As for the very-secure tag, Andy had mentioned that a few years ago.   It looks like it wasn’t standardized.

<KENT> gotcha



<Eric2>  Thanks again for your time on this.  I see these as good additions...

<Kent2> getting there!  :)



[Eric3]  Thanks again!

Eric3