Re: [Netconf] In an update, when is a delete a delete?

[Martin Bjorklund <[e-mail]>]

"Eric Voit (evoit)" <[e-mail]> wrote:
> > From: Martin Bjorklund, May 23, 2017 1:57 PM
> > 
> > "Eric Voit (evoit)" <[e-mail]> wrote:
> > > > Martin Bjorklund, May 22, 2017 3:30 PM
> > > >
> > > > Alexander Clemm <[e-mail]> wrote:
> > > > > Hi Martin,
> > > > >
> > > > > Almost overlooked your question below.  What is meant by the
> > > > > filter is specified in section 3.5 of the YANG-Push document.
> > > > >
> > > > > "Only a single filter can be applied to a subscription at a time.
> > > > > The following filter types are included in the yang-push data model:
> > > > > [subtree] [xpath]"
> > > >
> > > > Actually, only "subtree" is defined in yang-push, "xpath" is defined
> > > > in subscribed-notifications.
> > >
> > > At the top of yang-push page 7, xpath selection is described.  Is
> > > there something you feel missing?
> > 
> > Yes.  First of all, the YANG module defines an identity called
> > "xpath", based on
> > "sn:filter".  So this filter has nothing to do with selecting nodes in
> > a datastore;
> > this filter is used to match against a generated notification record.
> 
> To address this we could split the xpath identity into two types:
> "xpath-selection" and "xpath-boolean".  These would have different
> definitions, but both reference
> http://www.w3.org/TR/1999/REC-xpath-19991116 .  The difference is that
> one results in a xpath node-set expression, and the other an xpath
> Boolean expression.  Make sense?

Not really.  In subscribed-notifications you have defined a generic
filter mechanism, that is used to block/allow the sending of generated
notifications to a subscriber.  This filter mechanism cannot be used
to select nodes to subscribe to for changes in a datastore.  You need
to define a separate mechansim for that in yang-push.  (Maybe not
even call it "filter", but perhaps "selection").

Some comments on subscribed-notifications: This generic mechanism
allows various filter syntaxes.  This generic mechanism needs to
explain what is required by a filter syntax definition (an identity,
evaluation rules).  In section 2.2 the document says that two filter
syntaxes are supported, but only one is defined (xpath).  It needs to
explain that a filter is supposed to return true or false (this part
RFC 5277 got right, see section 3.6).  Also, the current module has
the filter in an anyxml node; it is not clear how an XPath expression
is encoded in anyxml.

> > Second, the XPath filter is sorely underspecified.  The XPath context
> > is not
> > described, 
> 
> I understand and agree on your comment about the context.  The intent
> here is to provide equivalent capabilities of a GET.

I understand that.

> As it would be a
> huge undertaking to try to consolidate an industry-wide view of the
> minimal xpath syntax and capabilities in networking

Ehh... yes?  What does this have to do with specifiying the XPath
context?

> , I am hoping this
> doesn't fall under the umbrella of YANG subscription.  I would be glad
> to support someone who wishes to take this up though.
> 
> > the expected result data type is not defined, and it is not described
> > how the result is supposed to be used.
> 
> As for the result, the anydata output should be provided to the
> subscriber (with appropriate security applied).

I was referring to the output of the filter evaluation.

> They can determine
> how to use it.  The preferred embodiment would be to maintain a local
> extract of the Publisher's datastore (as defined by the filter).
> 
> > > >  [Side note - I think this is wrong, subscribed-notifications should
> > > > also define "subtree".]
> > > >
> > > > But these filters are used by the server to decide if a certain
> > > > notification that has been generated will be sent to the client or
> > > > not.
> > >
> > > Yes, the filters in subscribed-notification are supposed to give a
> > > boolean indication as to whether a specific event should traverse the
> > > filter in its entirety. RFC6241 section 6 subtree filters are written
> > > to provide a subset of content.  I suppose it would be possible to
> > > define an event-based subtree-filter-type where a non-null result of
> > > the subtree filter means that a particular event should traverse that
> > > filter.  Is this what you are suggesting?
> > 
> > Yes.  Note that this is already provided by RFC 5277, and I have
> > always
> > assumed that this new work will provide at least the same functions as
> > RFC
> > 5277 (and more).  (But note that the XPath filter is underspecified
> > also in RFC
> > 5277...)
> 
> I also want to make sure that a non-null result from a filter allows
> the event to pass.  I suspect that an xpath Boolean filter could be
> designed to do this, but I will tweak the subscribed-notifications
> text so that unnecessary filtering expression complexity is not
> artificially required.

Please make sure you understand how subtree filters and XPath filters
work in RFC 5277.  There is nothing wrong with that functionality.

> > > > If you want to define filters to specifify which nodes to subscribe
> > > > to, I think you need to define new filters, not try to reuese these
> > > > notification filters.
> > >
> > > Filtering syntax is hard, so we have been trying to adopt whatever is
> > > available for GET.  This way we don't have to educate users on a new
> > > universe of what is possible.  I fully expect that lots of learnings
> > > are going to come in the industry here over time, and this will be
> > > revisited in the future.
> > 
> > ?
> > 
> > I am not proposing any new filter syntax.  I am saying that the
> > current filter
> > nodes as defined in subscribed-notification cannot be used to select
> > nodes to
> > subscribe to for changes.
> 
> Understand.  Hopefully with the "xpath-selection" change proposed
> above, this will be covered.

No, see above.


/martin



> 
> Eric
> 
> > > > As for your question, I think such a filter should be defined to
> > > > return a node- set to which the client subscribe to changes.  If any
> > > > node (or subnode
> > > > to) in
> > > > this node-set changes, the notif will be sent.  Then the question
> > > > about value comparision is not relevant anymore.
> > >
> > > Excellent, on-change should only send an update if the results of the
> > > subscription filter have changed since the previous push.  It is quite
> > > possible that an object has been created and then deleted since the
> > > last push.
> > 
> > I don't understand what you're trying to say with these sentences (but
> > since
> > the first word was "Excellent" maybe it's ok ;)
> > 
> > 
> > /martin
> > 
> > 
> > > Representing this was the genesis of Alex's question.
> > >
> > > Eric
> > >
> > > > /martin
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > >
> > > > > In effect, the filter specifies which data nodes to consider when
> > > > > sending updates.
> > > > >
> > > > > ---Alex
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Martin Bjorklund [mailto:[e-mail]]
> > > > > Sent: Wednesday, May 17, 2017 11:50 PM
> > > > > To: Alexander Clemm <[e-mail]>
> > > > > Cc: [e-mail]
> > > > > Subject: Re: [Netconf] In an update, when is a delete a delete?
> > > > >
> > > > > Hi,
> > > > >
> > > > > Alexander Clemm <[e-mail]> wrote:
> > > > > > Hello all,
> > > > > >
> > > > > > In updating the YANG-Push document
> > > > > > (draft-ietf-netconf-yang-push), we have come across one issue
> > > > > > that we wanted to raise with the working group.
> > > > > >
> > > > > > As part of an on-change subscription, update records reflect the
> > > > > > type of change (e.g. whether the value of an object has changed,
> > > > > > or whether an object was created or deleted); a subscription
> > > > > > allows also to specify whether interested only in specific types
> > > > > > of changes (for example, only creates and deleted but no value
> > changes).
> > > > > >
> > > > > > At the same time, a subscription filter specifies which objects
> > > > > > to include as part of a subscription and which not.
> > > > >
> > > > > Hmm, which filter are you talking about?  The only XPath filter I
> > > > > find in the current set of documents is the
> > > > > "ietf-subscribed-notifications:xpath" filter type (which btw is
> > > > > sorely underspecified).  Section 2.2 of
> > > > > draft-ietf-netconf-subscribed-notifications-02 says:
> > > > >
> > > > >    Events which evaluate to "true" as a
> > > > >    result of the evaluation by the filter must traverse the filter in
> > > > >    their entirety.
> > > > >
> > > > > It's not clear what this means, but my guess is that this is
> > > > > supposed to work like the old RFC 5277 filters, where the filter
> > > > > expression is evaluated on the notification contents, and if the
> > > > > expression returns "true" (for XPath filters this means converting
> > > > > the results to a boolean), then the notification is sent, otherwise
> > > > > not.
> > > > >
> > > > > But it seems you are referring to some other filter which would be
> > > > > used to select a node set for which changes are reported?
> > > > >
> > > > > I would like to understand which filter mechanism you mean before
> > > > > having an opinion in this matter.
> > > > >
> > > > >
> > > > > /martin
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > (Really, it is not
> > > > > > so much of a "filter" on a stream that is generated
> > > > > > independently of the filter, than it is a policy of which
> > > > > > objects to include as part of subscribed update records.)
> > > > > > However, a subscription filter (such as
> > > > > > XPath) can be used to also specify a value filter, which will
> > > > > > include or exclude objects based on their current value. This
> > > > > > makes it possible to e.g. subscribe to an object "foo" but only
> > > > > > if its value is 5.
> > > > > >
> > > > > > Now, this means that the same object could be included in one
> > > > > > update, but excluded in another update, due to its value no
> > > > > > longer meeting the filter criteria.  For example, if foo's value
> > > > > > changes from 5 to 3 in one cycle, a periodic subscription will
> > > > > > no longer include foo in its next update.  The question now
> > > > > > concerns how to properly handle this in the case of an on-change
> > subscription.
> > > > > >
> > > > > > One possibility concerns reporting the fact that "foo" no longer
> > > > > > meets the subscription criteria and is no longer included in the
> > > > > > update record as a "delete" event.  If foo's value again becomes "5"
> > > > > > at a later point in time, that would be reported as a "create"
> > > > > > event.  If foo's value changes again from 5 at a later point in
> > > > > > time and then changes back to 3 before the time of the update
> > > > > > (perhaps because the value changed during the dampening
> > > > > > interval), it would be reported as another "delete" event
> > > > > > (without ever reporting a create event).  On the other hand, if
> > > > > > foo's value changed from 3 to
> > > > > > 6 and back again, nothing would be reported because it did not
> > > > > > meet the filter criteria at any point in time.
> > > > > >
> > > > > > >From the perspective of the receiver this may make sense if it
> > > > > > >is synching its copy of the state.  However, from the
> > > > > > >perspective of the publisher, the object was never created or
> > > > > > >deleted - only its value changed, and the case when the object
> > > > > > >was truly created or deleted can no longer be distinguished
> > > > > > >from the case when its value
> > > > changed.
> > > > > > >A "create" simply means "an object now meets a filter criteria,
> > > > > > >that was not reported in the previous cycle" (which does not
> > > > > > >mean that the object was actually created - it may have been
> > > > > > >created, or it may have simply undergone a value change).
> > > > > >
> > > > > > An alternative (let's call it alternative 2) is therefore to
> > > > > > make a distinction between whether an object was created or
> > > > > > deleted, or whether its value fell in or out of a filter range.
> > > > > > This appears semantically cleaner.  However, it will require
> > > > > > modifying the encoding to allow for distinction between those
> > > > > > cases (currently, just plain patch encoding is used).
> > > > > >
> > > > > > A third alternative is to let filters select only data nodes to
> > > > > > subscribe to, and separate out the value filter (or disallow it
> > > > > > as a feature altogether).  This alternative has the drawback of
> > > > > > being less conceptually powerful, even if it may be easier to
> > implement.
> > > > > >
> > > > > > Thoughts?  Any preferences between 1, 2, and 3?
> > > > > > --- Alex
> > > > > >
> > > > >
> > > >
> > > > _______________________________________________
> > > > Netconf mailing list
> > > > [e-mail]
> > > > https://www.ietf.org/mailman/listinfo/netconf
> > >
>