Re: [Netconf] In an update, when is a delete a delete?

["Eric Voit (evoit)" <[e-mail]>]

> From: Martin Bjorklund, May 23, 2017 1:57 PM
> 
> "Eric Voit (evoit)" <[e-mail]> wrote:
> > > Martin Bjorklund, May 22, 2017 3:30 PM
> > >
> > > Alexander Clemm <[e-mail]> wrote:
> > > > Hi Martin,
> > > >
> > > > Almost overlooked your question below.  What is meant by the
> > > > filter is specified in section 3.5 of the YANG-Push document.
> > > >
> > > > "Only a single filter can be applied to a subscription at a time.
> > > > The following filter types are included in the yang-push data model:
> > > > [subtree] [xpath]"
> > >
> > > Actually, only "subtree" is defined in yang-push, "xpath" is defined
> > > in subscribed-notifications.
> >
> > At the top of yang-push page 7, xpath selection is described.  Is
> > there something you feel missing?
> 
> Yes.  First of all, the YANG module defines an identity called "xpath", based on
> "sn:filter".  So this filter has nothing to do with selecting nodes in a datastore;
> this filter is used to match against a generated notification record.

To address this we could split the xpath identity into two types: "xpath-selection" and  "xpath-boolean".  These would have different definitions, but both reference http://www.w3.org/TR/1999/REC-xpath-19991116  .  The difference is that one results in a xpath node-set expression, and the other an xpath Boolean expression.  Make sense?

> Second, the XPath filter is sorely underspecified.  The XPath context is not
> described, 

I understand and agree on your comment about the context.  The intent here is to provide equivalent capabilities of a GET.  As it would be a huge undertaking to try to consolidate an industry-wide view of the minimal xpath syntax and capabilities in networking, I am hoping this doesn't fall under the umbrella of YANG subscription.  I would be glad to support someone who wishes to take this up though.

> the expected result data type is not defined, and it is not described
> how the result is supposed to be used.

As for the result, the anydata output should be provided to the subscriber (with appropriate security applied).  They can determine how to use it.   The preferred embodiment would be to maintain a local extract of the Publisher's datastore (as defined by the filter).

> > >  [Side note - I think this is wrong, subscribed-notifications should
> > > also define "subtree".]
> > >
> > > But these filters are used by the server to decide if a certain
> > > notification that has been generated will be sent to the client or
> > > not.
> >
> > Yes, the filters in subscribed-notification are supposed to give a
> > boolean indication as to whether a specific event should traverse the
> > filter in its entirety. RFC6241 section 6 subtree filters are written
> > to provide a subset of content.  I suppose it would be possible to
> > define an event-based subtree-filter-type where a non-null result of
> > the subtree filter means that a particular event should traverse that
> > filter.  Is this what you are suggesting?
> 
> Yes.  Note that this is already provided by RFC 5277, and I have always
> assumed that this new work will provide at least the same functions as RFC
> 5277 (and more).  (But note that the XPath filter is underspecified also in RFC
> 5277...)

I also want to make sure that a non-null result from a filter allows the event to pass.  I suspect that an xpath Boolean filter could be designed to do this, but I will tweak the subscribed-notifications text so that unnecessary filtering expression complexity is not artificially required.

> > > If you want to define filters to specifify which nodes to subscribe
> > > to, I think you need to define new filters, not try to reuese these
> > > notification filters.
> >
> > Filtering syntax is hard, so we have been trying to adopt whatever is
> > available for GET.  This way we don't have to educate users on a new
> > universe of what is possible.  I fully expect that lots of learnings
> > are going to come in the industry here over time, and this will be
> > revisited in the future.
> 
> ?
> 
> I am not proposing any new filter syntax.  I am saying that the current filter
> nodes as defined in subscribed-notification cannot be used to select nodes to
> subscribe to for changes.

Understand.  Hopefully with the "xpath-selection" change proposed above, this will be covered.

Eric

> > > As for your question, I think such a filter should be defined to
> > > return a node- set to which the client subscribe to changes.  If any
> > > node (or subnode
> > > to) in
> > > this node-set changes, the notif will be sent.  Then the question
> > > about value comparision is not relevant anymore.
> >
> > Excellent, on-change should only send an update if the results of the
> > subscription filter have changed since the previous push.  It is quite
> > possible that an object has been created and then deleted since the
> > last push.
> 
> I don't understand what you're trying to say with these sentences (but since
> the first word was "Excellent" maybe it's ok ;)
> 
> 
> /martin
> 
> 
> > Representing this was the genesis of Alex's question.
> >
> > Eric
> >
> > > /martin
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > In effect, the filter specifies which data nodes to consider when
> > > > sending updates.
> > > >
> > > > ---Alex
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Martin Bjorklund [mailto:[e-mail]]
> > > > Sent: Wednesday, May 17, 2017 11:50 PM
> > > > To: Alexander Clemm <[e-mail]>
> > > > Cc: [e-mail]
> > > > Subject: Re: [Netconf] In an update, when is a delete a delete?
> > > >
> > > > Hi,
> > > >
> > > > Alexander Clemm <[e-mail]> wrote:
> > > > > Hello all,
> > > > >
> > > > > In updating the YANG-Push document
> > > > > (draft-ietf-netconf-yang-push), we have come across one issue
> > > > > that we wanted to raise with the working group.
> > > > >
> > > > > As part of an on-change subscription, update records reflect the
> > > > > type of change (e.g. whether the value of an object has changed,
> > > > > or whether an object was created or deleted); a subscription
> > > > > allows also to specify whether interested only in specific types
> > > > > of changes (for example, only creates and deleted but no value
> changes).
> > > > >
> > > > > At the same time, a subscription filter specifies which objects
> > > > > to include as part of a subscription and which not.
> > > >
> > > > Hmm, which filter are you talking about?  The only XPath filter I
> > > > find in the current set of documents is the
> > > > "ietf-subscribed-notifications:xpath" filter type (which btw is
> > > > sorely underspecified).  Section 2.2 of
> > > > draft-ietf-netconf-subscribed-notifications-02 says:
> > > >
> > > >    Events which evaluate to "true" as a
> > > >    result of the evaluation by the filter must traverse the filter in
> > > >    their entirety.
> > > >
> > > > It's not clear what this means, but my guess is that this is
> > > > supposed to work like the old RFC 5277 filters, where the filter
> > > > expression is evaluated on the notification contents, and if the
> > > > expression returns "true" (for XPath filters this means converting
> > > > the results to a boolean), then the notification is sent, otherwise not.
> > > >
> > > > But it seems you are referring to some other filter which would be
> > > > used to select a node set for which changes are reported?
> > > >
> > > > I would like to understand which filter mechanism you mean before
> > > > having an opinion in this matter.
> > > >
> > > >
> > > > /martin
> > > >
> > > >
> > > >
> > > >
> > > > > (Really, it is not
> > > > > so much of a "filter" on a stream that is generated
> > > > > independently of the filter, than it is a policy of which
> > > > > objects to include as part of subscribed update records.)
> > > > > However, a subscription filter (such as
> > > > > XPath) can be used to also specify a value filter, which will
> > > > > include or exclude objects based on their current value. This
> > > > > makes it possible to e.g. subscribe to an object "foo" but only
> > > > > if its value is 5.
> > > > >
> > > > > Now, this means that the same object could be included in one
> > > > > update, but excluded in another update, due to its value no
> > > > > longer meeting the filter criteria.  For example, if foo's value
> > > > > changes from 5 to 3 in one cycle, a periodic subscription will
> > > > > no longer include foo in its next update.  The question now
> > > > > concerns how to properly handle this in the case of an on-change
> subscription.
> > > > >
> > > > > One possibility concerns reporting the fact that "foo" no longer
> > > > > meets the subscription criteria and is no longer included in the
> > > > > update record as a "delete" event.  If foo's value again becomes "5"
> > > > > at a later point in time, that would be reported as a "create"
> > > > > event.  If foo's value changes again from 5 at a later point in
> > > > > time and then changes back to 3 before the time of the update
> > > > > (perhaps because the value changed during the dampening
> > > > > interval), it would be reported as another "delete" event
> > > > > (without ever reporting a create event).  On the other hand, if
> > > > > foo's value changed from 3 to
> > > > > 6 and back again, nothing would be reported because it did not
> > > > > meet the filter criteria at any point in time.
> > > > >
> > > > > >From the perspective of the receiver this may make sense if it
> > > > > >is synching its copy of the state.  However, from the
> > > > > >perspective of the publisher, the object was never created or
> > > > > >deleted - only its value changed, and the case when the object
> > > > > >was truly created or deleted can no longer be distinguished
> > > > > >from the case when its value
> > > changed.
> > > > > >A "create" simply means "an object now meets a filter criteria,
> > > > > >that was not reported in the previous cycle" (which does not
> > > > > >mean that the object was actually created - it may have been
> > > > > >created, or it may have simply undergone a value change).
> > > > >
> > > > > An alternative (let's call it alternative 2) is therefore to
> > > > > make a distinction between whether an object was created or
> > > > > deleted, or whether its value fell in or out of a filter range.
> > > > > This appears semantically cleaner.  However, it will require
> > > > > modifying the encoding to allow for distinction between those
> > > > > cases (currently, just plain patch encoding is used).
> > > > >
> > > > > A third alternative is to let filters select only data nodes to
> > > > > subscribe to, and separate out the value filter (or disallow it
> > > > > as a feature altogether).  This alternative has the drawback of
> > > > > being less conceptually powerful, even if it may be easier to
> implement.
> > > > >
> > > > > Thoughts?  Any preferences between 1, 2, and 3?
> > > > > --- Alex
> > > > >
> > > >
> > >
> > > _______________________________________________
> > > Netconf mailing list
> > > [e-mail]
> > > https://www.ietf.org/mailman/listinfo/netconf
> >