Re: [Netconf] In an update, when is a delete a delete?

Hi Martin, 
Thank you for your response.  
Two more comments inline, <ALEX> (including a response to Eric)
--- Alex

-----Original Message-----
From: Eric Voit (evoit) [mailto:evoit@cisco.com] 
Sent: Tuesday, May 23, 2017 9:04 AM
To: Martin Bjorklund <mbj@tail-f.com>; Alexander Clemm <alexander.clemm@huawei.com>
Cc: netconf@ietf.org
Subject: RE: [Netconf] In an update, when is a delete a delete?

> Martin Bjorklund, May 22, 2017 3:30 PM
> 
> Alexander Clemm <alexander.clemm@huawei.com> wrote:
> > Hi Martin,
> >
> > Almost overlooked your question below.  What is meant by the filter 
> > is specified in section 3.5 of the YANG-Push document.
> >
> > "Only a single filter can be applied to a subscription at a time.  
> > The following filter types are included in the yang-push data model:
> > [subtree] [xpath]"
> 
> Actually, only "subtree" is defined in yang-push, "xpath" is defined 
> in subscribed-notifications.

At the top of yang-push page 7, xpath selection is described.  Is there something you feel missing?

>  [Side note - I think this is wrong, subscribed-notifications should 
> also define "subtree".]
>
> But these filters are used by the server to decide if a certain 
> notification that has been generated will be sent to the client or not.

Yes, the filters in subscribed-notification are supposed to give a boolean indication as to whether a specific event should traverse the filter in its entirety. RFC6241 section 6 subtree filters are written to provide a subset of content.   I suppose it would be possible to define an event-based subtree-filter-type where a non-null result of the subtree filter means that a particular event should traverse that filter.  Is this what you are suggesting?

> If you want to define filters to specifify which nodes to subscribe 
> to, I think you need to define new filters, not try to reuese these notification filters.

Filtering syntax is hard, so we have been trying to adopt whatever is available for GET.  This way we don't have to educate users on a new universe of what is possible.  I fully expect that lots of learnings are going to come in the industry here over time, and this will be revisited in the future.

<ALEX> I think the difference in semantics between a filter that filters on the contents that is included in a notification (and applies to the notification as a whole), and a filter that specifies updates of which data nodes to include in an update notification, is actually key.  (Arguably, the second filter defines which content to generate in the first place.)  

We will review the text to make sure the distinction is clear.  That said, we do want to use the same filter syntax constructs for both, even if they are used slightly differently when used as a notification filter vs a filter that specifies which updates to include. 
</ALEX>

> As for your question, I think such a filter should be defined to 
> return a node- set to which the client subscribe to changes.  If any 
> node (or subnode to) in this node-set changes, the notif will be sent.  
> Then the question about value comparision is not relevant anymore.

Excellent, on-change should only send an update if the results of the subscription filter have changed since the previous push.   It is quite possible that an object has been created and then deleted since the last push.  Representing this was the genesis of Alex's question.

<ALEX>I interpret the statement differently:  It is not about whether the results of the subscription filter have changed, but if any nodes in the node-set that had been subscribed to changed.   In other words, I take it as an endorsement of option 3, not of option 1.  
--- Alex
</ALEX>

Eric

> /martin
> 
> 
> 
> 
> 
> >
> > In effect, the filter specifies which data nodes to consider when 
> > sending updates.
> >
> > ---Alex
> >
> >
> > -----Original Message-----
> > From: Martin Bjorklund [mailto:mbj@tail-f.com]
> > Sent: Wednesday, May 17, 2017 11:50 PM
> > To: Alexander Clemm <alexander.clemm@huawei.com>
> > Cc: netconf@ietf.org
> > Subject: Re: [Netconf] In an update, when is a delete a delete?
> >
> > Hi,
> >
> > Alexander Clemm <alexander.clemm@huawei.com> wrote:
> > > Hello all,
> > >
> > > In updating the YANG-Push document (draft-ietf-netconf-yang-push), 
> > > we have come across one issue that we wanted to raise with the 
> > > working group.
> > >
> > > As part of an on-change subscription, update records reflect the 
> > > type of change (e.g. whether the value of an object has changed, 
> > > or whether an object was created or deleted); a subscription 
> > > allows also to specify whether interested only in specific types 
> > > of changes (for example, only creates and deleted but no value changes).
> > >
> > > At the same time, a subscription filter specifies which objects to 
> > > include as part of a subscription and which not.
> >
> > Hmm, which filter are you talking about?  The only XPath filter I 
> > find in the current set of documents is the 
> > "ietf-subscribed-notifications:xpath" filter type (which btw is 
> > sorely underspecified).  Section 2.2 of
> > draft-ietf-netconf-subscribed-notifications-02 says:
> >
> >    Events which evaluate to "true" as a
> >    result of the evaluation by the filter must traverse the filter in
> >    their entirety.
> >
> > It's not clear what this means, but my guess is that this is 
> > supposed to work like the old RFC 5277 filters, where the filter 
> > expression is evaluated on the notification contents, and if the 
> > expression returns "true" (for XPath filters this means converting 
> > the results to a boolean), then the notification is sent, otherwise not.
> >
> > But it seems you are referring to some other filter which would be 
> > used to select a node set for which changes are reported?
> >
> > I would like to understand which filter mechanism you mean before 
> > having an opinion in this matter.
> >
> >
> > /martin
> >
> >
> >
> >
> > > (Really, it is not
> > > so much of a "filter" on a stream that is generated independently 
> > > of the filter, than it is a policy of which objects to include as 
> > > part of subscribed update records.)  However, a subscription 
> > > filter (such as
> > > XPath) can be used to also specify a value filter, which will 
> > > include or exclude objects based on their current value. This 
> > > makes it possible to e.g. subscribe to an object "foo" but only if 
> > > its value is 5.
> > >
> > > Now, this means that the same object could be included in one 
> > > update, but excluded in another update, due to its value no longer 
> > > meeting the filter criteria.  For example, if foo's value changes 
> > > from 5 to 3 in one cycle, a periodic subscription will no longer 
> > > include foo in its next update.  The question now concerns how to 
> > > properly handle this in the case of an on-change subscription.
> > >
> > > One possibility concerns reporting the fact that "foo" no longer 
> > > meets the subscription criteria and is no longer included in the 
> > > update record as a "delete" event.  If foo's value again becomes "5"
> > > at a later point in time, that would be reported as a "create"
> > > event.  If foo's value changes again from 5 at a later point in 
> > > time and then changes back to 3 before the time of the update 
> > > (perhaps because the value changed during the dampening interval), 
> > > it would be reported as another "delete" event (without ever 
> > > reporting a create event).  On the other hand, if foo's value 
> > > changed from 3 to
> > > 6 and back again, nothing would be reported because it did not 
> > > meet the filter criteria at any point in time.
> > >
> > > >From the perspective of the receiver this may make sense if it is 
> > > >synching its copy of the state.  However, from the perspective of 
> > > >the publisher, the object was never created or deleted - only its 
> > > >value changed, and the case when the object was truly created or 
> > > >deleted can no longer be distinguished from the case when its 
> > > >value
> changed.
> > > >A "create" simply means "an object now meets a filter criteria, 
> > > >that was not reported in the previous cycle" (which does not mean 
> > > >that the object was actually created - it may have been created, 
> > > >or it may have simply undergone a value change).
> > >
> > > An alternative (let's call it alternative 2) is therefore to make 
> > > a distinction between whether an object was created or deleted, or 
> > > whether its value fell in or out of a filter range.  This appears 
> > > semantically cleaner.  However, it will require modifying the 
> > > encoding to allow for distinction between those cases (currently, 
> > > just plain patch encoding is used).
> > >
> > > A third alternative is to let filters select only data nodes to 
> > > subscribe to, and separate out the value filter (or disallow it as 
> > > a feature altogether).  This alternative has the drawback of being 
> > > less conceptually powerful, even if it may be easier to implement.
> > >
> > > Thoughts?  Any preferences between 1, 2, and 3?
> > > --- Alex
> > >
> >
> 
> _______________________________________________
> Netconf mailing list
> Netconf@ietf.org
> https://www.ietf.org/mailman/listinfo/netconf