Re: [Netconf] In an update, when is a delete a delete?

[Andy Bierman <[e-mail]>]

On Tue, May 23, 2017 at 3:16 PM, Eric Voit (evoit) <[e-mail]> wrote:

> > From: Martin Bjorklund, May 23, 2017 1:57 PM
> >
> > "Eric Voit (evoit)" <[e-mail]> wrote:
> > > > Martin Bjorklund, May 22, 2017 3:30 PM
> > > >
> > > > Alexander Clemm <[e-mail]> wrote:
> > > > > Hi Martin,
> > > > >
> > > > > Almost overlooked your question below.  What is meant by the
> > > > > filter is specified in section 3.5 of the YANG-Push document.
> > > > >
> > > > > "Only a single filter can be applied to a subscription at a time.
> > > > > The following filter types are included in the yang-push data
> model:
> > > > > [subtree] [xpath]"
> > > >
> > > > Actually, only "subtree" is defined in yang-push, "xpath" is defined
> > > > in subscribed-notifications.
> > >
> > > At the top of yang-push page 7, xpath selection is described.  Is
> > > there something you feel missing?
> >
> > Yes.  First of all, the YANG module defines an identity called "xpath",
> based on
> > "sn:filter".  So this filter has nothing to do with selecting nodes in a
> datastore;
> > this filter is used to match against a generated notification record.
>
> To address this we could split the xpath identity into two types:
> "xpath-selection" and  "xpath-boolean".  These would have different
> definitions, but both reference http://www.w3.org/TR/1999/REC-
> xpath-19991116  .  The difference is that one results in a xpath node-set
> expression, and the other an xpath Boolean expression.  Make sense?
>
> > Second, the XPath filter is sorely underspecified.  The XPath context is
> not
> > described,
>
> I understand and agree on your comment about the context.  The intent here
> is to provide equivalent capabilities of a GET.  As it would be a huge
> undertaking to try to consolidate an industry-wide view of the minimal
> xpath syntax and capabilities in networking, I am hoping this doesn't fall
> under the umbrella of YANG subscription.  I would be glad to support
> someone who wishes to take this up though.
>
>

subtree and XPath filtering are implemented in various ways which could all
be considered valid.
The node-set result is not the same across server implementations.

For example:

   <filter>
     <foo />
     <foo>
       <bar>1</bar>
     </foo>
  </filter>

Does the server return the entire /foo subtree or return the entire /foo
subtree
followed by another subtree for /foo[bar=1]?  (yes. that's the problem)
XPath has many more ways to overlap node-sets.

Should we even document all the under-implemented servers that do not
support the
complete XPath syntax? Probably not, but it would be worthwhile to have an
"XPath-core"
that is widely supported.

I agree that YANG Push is not the correct document for this sort of work.

Perhaps YANG Push filtering should be 2 phases: (1) select + (2) test),
similar to how XSLT works.

(1) select can be simple and static, like a node-instance-identifier
expression from NACM
(2) test can be an XPath boolean expression run against each node selected
in (1),
      using the selected node as the context node


Andy



> > the expected result data type is not defined, and it is not described
> > how the result is supposed to be used.
>
> As for the result, the anydata output should be provided to the subscriber
> (with appropriate security applied).  They can determine how to use it.
>  The preferred embodiment would be to maintain a local extract of the
> Publisher's datastore (as defined by the filter).
>
> > > >  [Side note - I think this is wrong, subscribed-notifications should
> > > > also define "subtree".]
> > > >
> > > > But these filters are used by the server to decide if a certain
> > > > notification that has been generated will be sent to the client or
> > > > not.
> > >
> > > Yes, the filters in subscribed-notification are supposed to give a
> > > boolean indication as to whether a specific event should traverse the
> > > filter in its entirety. RFC6241 section 6 subtree filters are written
> > > to provide a subset of content.  I suppose it would be possible to
> > > define an event-based subtree-filter-type where a non-null result of
> > > the subtree filter means that a particular event should traverse that
> > > filter.  Is this what you are suggesting?
> >
> > Yes.  Note that this is already provided by RFC 5277, and I have always
> > assumed that this new work will provide at least the same functions as
> RFC
> > 5277 (and more).  (But note that the XPath filter is underspecified also
> in RFC
> > 5277...)
>
> I also want to make sure that a non-null result from a filter allows the
> event to pass.  I suspect that an xpath Boolean filter could be designed to
> do this, but I will tweak the subscribed-notifications text so that
> unnecessary filtering expression complexity is not artificially required.
>
> > > > If you want to define filters to specifify which nodes to subscribe
> > > > to, I think you need to define new filters, not try to reuese these
> > > > notification filters.
> > >
> > > Filtering syntax is hard, so we have been trying to adopt whatever is
> > > available for GET.  This way we don't have to educate users on a new
> > > universe of what is possible.  I fully expect that lots of learnings
> > > are going to come in the industry here over time, and this will be
> > > revisited in the future.
> >
> > ?
> >
> > I am not proposing any new filter syntax.  I am saying that the current
> filter
> > nodes as defined in subscribed-notification cannot be used to select
> nodes to
> > subscribe to for changes.
>
> Understand.  Hopefully with the "xpath-selection" change proposed above,
> this will be covered.
>
> Eric
>
> > > > As for your question, I think such a filter should be defined to
> > > > return a node- set to which the client subscribe to changes.  If any
> > > > node (or subnode
> > > > to) in
> > > > this node-set changes, the notif will be sent.  Then the question
> > > > about value comparision is not relevant anymore.
> > >
> > > Excellent, on-change should only send an update if the results of the
> > > subscription filter have changed since the previous push.  It is quite
> > > possible that an object has been created and then deleted since the
> > > last push.
> >
> > I don't understand what you're trying to say with these sentences (but
> since
> > the first word was "Excellent" maybe it's ok ;)
> >
> >
> > /martin
> >
> >
> > > Representing this was the genesis of Alex's question.
> > >
> > > Eric
> > >
> > > > /martin
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > >
> > > > > In effect, the filter specifies which data nodes to consider when
> > > > > sending updates.
> > > > >
> > > > > ---Alex
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Martin Bjorklund [mailto:[e-mail]]
> > > > > Sent: Wednesday, May 17, 2017 11:50 PM
> > > > > To: Alexander Clemm <[e-mail]>
> > > > > Cc: [e-mail]
> > > > > Subject: Re: [Netconf] In an update, when is a delete a delete?
> > > > >
> > > > > Hi,
> > > > >
> > > > > Alexander Clemm <[e-mail]> wrote:
> > > > > > Hello all,
> > > > > >
> > > > > > In updating the YANG-Push document
> > > > > > (draft-ietf-netconf-yang-push), we have come across one issue
> > > > > > that we wanted to raise with the working group.
> > > > > >
> > > > > > As part of an on-change subscription, update records reflect the
> > > > > > type of change (e.g. whether the value of an object has changed,
> > > > > > or whether an object was created or deleted); a subscription
> > > > > > allows also to specify whether interested only in specific types
> > > > > > of changes (for example, only creates and deleted but no value
> > changes).
> > > > > >
> > > > > > At the same time, a subscription filter specifies which objects
> > > > > > to include as part of a subscription and which not.
> > > > >
> > > > > Hmm, which filter are you talking about?  The only XPath filter I
> > > > > find in the current set of documents is the
> > > > > "ietf-subscribed-notifications:xpath" filter type (which btw is
> > > > > sorely underspecified).  Section 2.2 of
> > > > > draft-ietf-netconf-subscribed-notifications-02 says:
> > > > >
> > > > >    Events which evaluate to "true" as a
> > > > >    result of the evaluation by the filter must traverse the filter
> in
> > > > >    their entirety.
> > > > >
> > > > > It's not clear what this means, but my guess is that this is
> > > > > supposed to work like the old RFC 5277 filters, where the filter
> > > > > expression is evaluated on the notification contents, and if the
> > > > > expression returns "true" (for XPath filters this means converting
> > > > > the results to a boolean), then the notification is sent,
> otherwise not.
> > > > >
> > > > > But it seems you are referring to some other filter which would be
> > > > > used to select a node set for which changes are reported?
> > > > >
> > > > > I would like to understand which filter mechanism you mean before
> > > > > having an opinion in this matter.
> > > > >
> > > > >
> > > > > /martin
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > (Really, it is not
> > > > > > so much of a "filter" on a stream that is generated
> > > > > > independently of the filter, than it is a policy of which
> > > > > > objects to include as part of subscribed update records.)
> > > > > > However, a subscription filter (such as
> > > > > > XPath) can be used to also specify a value filter, which will
> > > > > > include or exclude objects based on their current value. This
> > > > > > makes it possible to e.g. subscribe to an object "foo" but only
> > > > > > if its value is 5.
> > > > > >
> > > > > > Now, this means that the same object could be included in one
> > > > > > update, but excluded in another update, due to its value no
> > > > > > longer meeting the filter criteria.  For example, if foo's value
> > > > > > changes from 5 to 3 in one cycle, a periodic subscription will
> > > > > > no longer include foo in its next update.  The question now
> > > > > > concerns how to properly handle this in the case of an on-change
> > subscription.
> > > > > >
> > > > > > One possibility concerns reporting the fact that "foo" no longer
> > > > > > meets the subscription criteria and is no longer included in the
> > > > > > update record as a "delete" event.  If foo's value again becomes
> "5"
> > > > > > at a later point in time, that would be reported as a "create"
> > > > > > event.  If foo's value changes again from 5 at a later point in
> > > > > > time and then changes back to 3 before the time of the update
> > > > > > (perhaps because the value changed during the dampening
> > > > > > interval), it would be reported as another "delete" event
> > > > > > (without ever reporting a create event).  On the other hand, if
> > > > > > foo's value changed from 3 to
> > > > > > 6 and back again, nothing would be reported because it did not
> > > > > > meet the filter criteria at any point in time.
> > > > > >
> > > > > > >From the perspective of the receiver this may make sense if it
> > > > > > >is synching its copy of the state.  However, from the
> > > > > > >perspective of the publisher, the object was never created or
> > > > > > >deleted - only its value changed, and the case when the object
> > > > > > >was truly created or deleted can no longer be distinguished
> > > > > > >from the case when its value
> > > > changed.
> > > > > > >A "create" simply means "an object now meets a filter criteria,
> > > > > > >that was not reported in the previous cycle" (which does not
> > > > > > >mean that the object was actually created - it may have been
> > > > > > >created, or it may have simply undergone a value change).
> > > > > >
> > > > > > An alternative (let's call it alternative 2) is therefore to
> > > > > > make a distinction between whether an object was created or
> > > > > > deleted, or whether its value fell in or out of a filter range.
> > > > > > This appears semantically cleaner.  However, it will require
> > > > > > modifying the encoding to allow for distinction between those
> > > > > > cases (currently, just plain patch encoding is used).
> > > > > >
> > > > > > A third alternative is to let filters select only data nodes to
> > > > > > subscribe to, and separate out the value filter (or disallow it
> > > > > > as a feature altogether).  This alternative has the drawback of
> > > > > > being less conceptually powerful, even if it may be easier to
> > implement.
> > > > > >
> > > > > > Thoughts?  Any preferences between 1, 2, and 3?
> > > > > > --- Alex
> > > > > >
> > > > >
> > > >
> > > > _______________________________________________
> > > > Netconf mailing list
> > > > [e-mail]
> > > > https://www.ietf.org/mailman/listinfo/netconf
> > >
>
> _______________________________________________
> Netconf mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/netconf
>