Re: [Netconf] In an update, when is a delete a delete?

[Andy Bierman <[e-mail]>]

Hi,

Let's compare to what 5277 already has defined.
I agree with Martin that new functionality should be defined with
new terms and new specs.

First, RFC 5277 allows the event-type to be filtered.
This is basic functionality every server should support.
(e.g., select netconf-config-change, ignore netconf-session-start)
This event type test is not separate from the event content filter.
Both are boolean tests.

Notification filters for YANG Push need to select the event-types used,
such as push-change-update.  It would be nice if this was handled
automatically,
instead of listing them in the <establish-subscription>.

Traditional notification filters would require that the structure of an
event such
as push-change-update be known to the developer in full detail.
IMO this is a bad approach.  A new filter type that selects data nodes
would be much better.  The current proposal is easy to under-specify
and (IMO) nearly impossible for operators to use.


Andy



On Wed, May 24, 2017 at 3:25 PM, Alexander Clemm <[e-mail]
> wrote:

> Let me briefly summarize where I think we are. The thread has clearly
> moved on from the initial issue of how to distinguish between causes for
> updates to no longer include a given data node (because it was deleted, or
> because it changed its value to no longer match a filter) to other issues.
>
> For those other issues, I think there are in fact two separate items that
> we are trying to discuss at the same time, which are really orthogonal to
> one another:
>
> - The first item concerns the concept of a "filter" vs a "selector".  A
> filter is what gets specified for any subscription to notifications, which
> defines which notifications a subscriber wants to receive.  The filter is
> applied to the notification as a whole, i.e. either the notification is
> delivered or it is not.  It does not apply to subsets of contents within
> the notification.  A selector, on the other hand, is used to specify
> updates of which data nodes to include in a YANG-push subscription.  The
> same update notification could include updates of several data nodes, hence
> a filter applied to the notification as a whole would be inappropriate here
> - the semantics is slightly different: as a subscriber, "of  which data
> nodes would you like to receive updates", not "which notifications would
> you like to receive".
>
> Filters and selectors can be specified using the same syntax.  We are
> faced with a choice between specifying a single construct as part of a
> subscription, which is treated as a filter (in case of a "regular"
> subscription for notification updates) or as a selector (in case of a
> subscription to datastore updates) , or having separate objects, i.e.
> adding a separate "selector" construct for YANG-push.  Using a single
> object amounts to overloading.  It is more compact but with arguably a more
> complex semantics.  Using separate object results in a model that is more
> verbose, but has arguably simpler semantics.
>
> - The second item concerns the issue of whether the filter/selector
> construct has a dynamic type or a static type.  This is where the issue of
> identity vs case statement etc comes in.  In case of a dynamic type, we use
> two objects:  One object of a generic type (anydata) holds the
> filter/selector construct itself, the second object specifies how to
> interpret it, i.e. which type it is.  (In case of "overloading, we can also
> make explicit the distinction whether it is a filter or a selector).  That
> second object is an identityref, referencing one of the identities that
> designates the filter/selector type.  In case of a static type, we use a
> case statement (to distinguish which specific type it is).
>
> The argument that Eric is making is that we should have a single
> overloaded object that can serve as a filter or a selector depending on the
> context and whether it is used in a notification subscription or a
> YANG-push subscription, and that we use a dynamic type including an
> identityref that indicates whether the object serves as a selector or a
> filter.  From my perspective, I feel that not overloading may be
> conceptually a bit "cleaner", but at the end of the day I am fine either
> way.  And I am not entirely sure, Martin, what you are proposing.  Either
> way, we should document the issue and our choice clearly.
>
> --- Alex
>
>
> -----Original Message-----
> From: Martin Bjorklund [mailto:[e-mail]]
> Sent: Wednesday, May 24, 2017 12:36 AM
> To: [e-mail]
> Cc: Alexander Clemm <[e-mail]>; [e-mail]
> Subject: Re: [Netconf] In an update, when is a delete a delete?
>
> "Eric Voit (evoit)" <[e-mail]> wrote:
> > > From: Martin Bjorklund, May 23, 2017 1:57 PM
> > >
> > > "Eric Voit (evoit)" <[e-mail]> wrote:
> > > > > Martin Bjorklund, May 22, 2017 3:30 PM
> > > > >
> > > > > Alexander Clemm <[e-mail]> wrote:
> > > > > > Hi Martin,
> > > > > >
> > > > > > Almost overlooked your question below.  What is meant by the
> > > > > > filter is specified in section 3.5 of the YANG-Push document.
> > > > > >
> > > > > > "Only a single filter can be applied to a subscription at a time.
> > > > > > The following filter types are included in the yang-push data
> model:
> > > > > > [subtree] [xpath]"
> > > > >
> > > > > Actually, only "subtree" is defined in yang-push, "xpath" is
> > > > > defined in subscribed-notifications.
> > > >
> > > > At the top of yang-push page 7, xpath selection is described.  Is
> > > > there something you feel missing?
> > >
> > > Yes.  First of all, the YANG module defines an identity called
> > > "xpath", based on "sn:filter".  So this filter has nothing to do
> > > with selecting nodes in a datastore; this filter is used to match
> > > against a generated notification record.
> >
> > To address this we could split the xpath identity into two types:
> > "xpath-selection" and "xpath-boolean".  These would have different
> > definitions, but both reference
> > http://www.w3.org/TR/1999/REC-xpath-19991116 .  The difference is that
> > one results in a xpath node-set expression, and the other an xpath
> > Boolean expression.  Make sense?
>
> Not really.  In subscribed-notifications you have defined a generic filter
> mechanism, that is used to block/allow the sending of generated
> notifications to a subscriber.  This filter mechanism cannot be used to
> select nodes to subscribe to for changes in a datastore.  You need to
> define a separate mechansim for that in yang-push.  (Maybe not even call it
> "filter", but perhaps "selection").
>
> Some comments on subscribed-notifications: This generic mechanism allows
> various filter syntaxes.  This generic mechanism needs to explain what is
> required by a filter syntax definition (an identity, evaluation rules).  In
> section 2.2 the document says that two filter syntaxes are supported, but
> only one is defined (xpath).  It needs to explain that a filter is supposed
> to return true or false (this part RFC 5277 got right, see section 3.6).
> Also, the current module has the filter in an anyxml node; it is not clear
> how an XPath expression is encoded in anyxml.
>
> > > Second, the XPath filter is sorely underspecified.  The XPath
> > > context is not described,
> >
> > I understand and agree on your comment about the context.  The intent
> > here is to provide equivalent capabilities of a GET.
>
> I understand that.
>
> > As it would be a
> > huge undertaking to try to consolidate an industry-wide view of the
> > minimal xpath syntax and capabilities in networking
>
> Ehh... yes?  What does this have to do with specifiying the XPath context?
>
> > , I am hoping this
> > doesn't fall under the umbrella of YANG subscription.  I would be glad
> > to support someone who wishes to take this up though.
> >
> > > the expected result data type is not defined, and it is not
> > > described how the result is supposed to be used.
> >
> > As for the result, the anydata output should be provided to the
> > subscriber (with appropriate security applied).
>
> I was referring to the output of the filter evaluation.
>
> > They can determine
> > how to use it.  The preferred embodiment would be to maintain a local
> > extract of the Publisher's datastore (as defined by the filter).
> >
> > > > >  [Side note - I think this is wrong, subscribed-notifications
> > > > > should also define "subtree".]
> > > > >
> > > > > But these filters are used by the server to decide if a certain
> > > > > notification that has been generated will be sent to the client
> > > > > or not.
> > > >
> > > > Yes, the filters in subscribed-notification are supposed to give a
> > > > boolean indication as to whether a specific event should traverse
> > > > the filter in its entirety. RFC6241 section 6 subtree filters are
> > > > written to provide a subset of content.  I suppose it would be
> > > > possible to define an event-based subtree-filter-type where a
> > > > non-null result of the subtree filter means that a particular
> > > > event should traverse that filter.  Is this what you are suggesting?
> > >
> > > Yes.  Note that this is already provided by RFC 5277, and I have
> > > always assumed that this new work will provide at least the same
> > > functions as RFC
> > > 5277 (and more).  (But note that the XPath filter is underspecified
> > > also in RFC
> > > 5277...)
> >
> > I also want to make sure that a non-null result from a filter allows
> > the event to pass.  I suspect that an xpath Boolean filter could be
> > designed to do this, but I will tweak the subscribed-notifications
> > text so that unnecessary filtering expression complexity is not
> > artificially required.
>
> Please make sure you understand how subtree filters and XPath filters work
> in RFC 5277.  There is nothing wrong with that functionality.
>
> > > > > If you want to define filters to specifify which nodes to
> > > > > subscribe to, I think you need to define new filters, not try to
> > > > > reuese these notification filters.
> > > >
> > > > Filtering syntax is hard, so we have been trying to adopt whatever
> > > > is available for GET.  This way we don't have to educate users on
> > > > a new universe of what is possible.  I fully expect that lots of
> > > > learnings are going to come in the industry here over time, and
> > > > this will be revisited in the future.
> > >
> > > ?
> > >
> > > I am not proposing any new filter syntax.  I am saying that the
> > > current filter nodes as defined in subscribed-notification cannot be
> > > used to select nodes to subscribe to for changes.
> >
> > Understand.  Hopefully with the "xpath-selection" change proposed
> > above, this will be covered.
>
> No, see above.
>
>
> /martin
>
>
>
> >
> > Eric
> >
> > > > > As for your question, I think such a filter should be defined to
> > > > > return a node- set to which the client subscribe to changes.  If
> > > > > any node (or subnode
> > > > > to) in
> > > > > this node-set changes, the notif will be sent.  Then the
> > > > > question about value comparision is not relevant anymore.
> > > >
> > > > Excellent, on-change should only send an update if the results of
> > > > the subscription filter have changed since the previous push.  It
> > > > is quite possible that an object has been created and then deleted
> > > > since the last push.
> > >
> > > I don't understand what you're trying to say with these sentences
> > > (but since the first word was "Excellent" maybe it's ok ;)
> > >
> > >
> > > /martin
> > >
> > >
> > > > Representing this was the genesis of Alex's question.
> > > >
> > > > Eric
> > > >
> > > > > /martin
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > > In effect, the filter specifies which data nodes to consider
> > > > > > when sending updates.
> > > > > >
> > > > > > ---Alex
> > > > > >
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Martin Bjorklund [mailto:[e-mail]]
> > > > > > Sent: Wednesday, May 17, 2017 11:50 PM
> > > > > > To: Alexander Clemm <[e-mail]>
> > > > > > Cc: [e-mail]
> > > > > > Subject: Re: [Netconf] In an update, when is a delete a delete?
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Alexander Clemm <[e-mail]> wrote:
> > > > > > > Hello all,
> > > > > > >
> > > > > > > In updating the YANG-Push document
> > > > > > > (draft-ietf-netconf-yang-push), we have come across one
> > > > > > > issue that we wanted to raise with the working group.
> > > > > > >
> > > > > > > As part of an on-change subscription, update records reflect
> > > > > > > the type of change (e.g. whether the value of an object has
> > > > > > > changed, or whether an object was created or deleted); a
> > > > > > > subscription allows also to specify whether interested only
> > > > > > > in specific types of changes (for example, only creates and
> > > > > > > deleted but no value
> > > changes).
> > > > > > >
> > > > > > > At the same time, a subscription filter specifies which
> > > > > > > objects to include as part of a subscription and which not.
> > > > > >
> > > > > > Hmm, which filter are you talking about?  The only XPath
> > > > > > filter I find in the current set of documents is the
> > > > > > "ietf-subscribed-notifications:xpath" filter type (which btw
> > > > > > is sorely underspecified).  Section 2.2 of
> > > > > > draft-ietf-netconf-subscribed-notifications-02 says:
> > > > > >
> > > > > >    Events which evaluate to "true" as a
> > > > > >    result of the evaluation by the filter must traverse the
> filter in
> > > > > >    their entirety.
> > > > > >
> > > > > > It's not clear what this means, but my guess is that this is
> > > > > > supposed to work like the old RFC 5277 filters, where the
> > > > > > filter expression is evaluated on the notification contents,
> > > > > > and if the expression returns "true" (for XPath filters this
> > > > > > means converting the results to a boolean), then the
> > > > > > notification is sent, otherwise not.
> > > > > >
> > > > > > But it seems you are referring to some other filter which
> > > > > > would be used to select a node set for which changes are
> reported?
> > > > > >
> > > > > > I would like to understand which filter mechanism you mean
> > > > > > before having an opinion in this matter.
> > > > > >
> > > > > >
> > > > > > /martin
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > (Really, it is not
> > > > > > > so much of a "filter" on a stream that is generated
> > > > > > > independently of the filter, than it is a policy of which
> > > > > > > objects to include as part of subscribed update records.)
> > > > > > > However, a subscription filter (such as
> > > > > > > XPath) can be used to also specify a value filter, which
> > > > > > > will include or exclude objects based on their current
> > > > > > > value. This makes it possible to e.g. subscribe to an object
> > > > > > > "foo" but only if its value is 5.
> > > > > > >
> > > > > > > Now, this means that the same object could be included in
> > > > > > > one update, but excluded in another update, due to its value
> > > > > > > no longer meeting the filter criteria.  For example, if
> > > > > > > foo's value changes from 5 to 3 in one cycle, a periodic
> > > > > > > subscription will no longer include foo in its next update.
> > > > > > > The question now concerns how to properly handle this in the
> > > > > > > case of an on-change
> > > subscription.
> > > > > > >
> > > > > > > One possibility concerns reporting the fact that "foo" no
> > > > > > > longer meets the subscription criteria and is no longer
> > > > > > > included in the update record as a "delete" event.  If foo's
> value again becomes "5"
> > > > > > > at a later point in time, that would be reported as a "create"
> > > > > > > event.  If foo's value changes again from 5 at a later point
> > > > > > > in time and then changes back to 3 before the time of the
> > > > > > > update (perhaps because the value changed during the
> > > > > > > dampening interval), it would be reported as another
> > > > > > > "delete" event (without ever reporting a create event).  On
> > > > > > > the other hand, if foo's value changed from 3 to
> > > > > > > 6 and back again, nothing would be reported because it did
> > > > > > > not meet the filter criteria at any point in time.
> > > > > > >
> > > > > > > >From the perspective of the receiver this may make sense if
> > > > > > > >it is synching its copy of the state.  However, from the
> > > > > > > >perspective of the publisher, the object was never created
> > > > > > > >or deleted - only its value changed, and the case when the
> > > > > > > >object was truly created or deleted can no longer be
> > > > > > > >distinguished from the case when its value
> > > > > changed.
> > > > > > > >A "create" simply means "an object now meets a filter
> > > > > > > >criteria, that was not reported in the previous cycle"
> > > > > > > >(which does not mean that the object was actually created -
> > > > > > > >it may have been created, or it may have simply undergone a
> value change).
> > > > > > >
> > > > > > > An alternative (let's call it alternative 2) is therefore to
> > > > > > > make a distinction between whether an object was created or
> > > > > > > deleted, or whether its value fell in or out of a filter range.
> > > > > > > This appears semantically cleaner.  However, it will require
> > > > > > > modifying the encoding to allow for distinction between
> > > > > > > those cases (currently, just plain patch encoding is used).
> > > > > > >
> > > > > > > A third alternative is to let filters select only data nodes
> > > > > > > to subscribe to, and separate out the value filter (or
> > > > > > > disallow it as a feature altogether).  This alternative has
> > > > > > > the drawback of being less conceptually powerful, even if it
> > > > > > > may be easier to
> > > implement.
> > > > > > >
> > > > > > > Thoughts?  Any preferences between 1, 2, and 3?
> > > > > > > --- Alex
> > > > > > >
> > > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Netconf mailing list
> > > > > [e-mail]
> > > > > https://www.ietf.org/mailman/listinfo/netconf
> > > >
> >
>
> _______________________________________________
> Netconf mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/netconf
>