Re: [Netconf] In an update, when is a delete a delete?
Martin Bjorklund <mbj@tail-f.com> Wed, 24 May 2017 07:42 UTC
Return-Path: <mbj@tail-f.com>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 314BF1270A0 for <netconf@ietfa.amsl.com>; Wed, 24 May 2017 00:42:23 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.901
X-Spam-Level:
X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L7s9UMZyHo_Y for <netconf@ietfa.amsl.com>; Wed, 24 May 2017 00:42:20 -0700 (PDT)
Received: from mail.tail-f.com (mail.tail-f.com [46.21.102.45]) by ietfa.amsl.com (Postfix) with ESMTP id 7F7C31201FA for <netconf@ietf.org>; Wed, 24 May 2017 00:42:20 -0700 (PDT)
Received: from localhost (unknown [173.38.220.40]) by mail.tail-f.com (Postfix) with ESMTPSA id 72C351AE046D; Wed, 24 May 2017 09:42:19 +0200 (CEST)
Date: Wed, 24 May 2017 09:42:35 +0200
Message-Id: <20170524.094235.889675806701058038.mbj@tail-f.com>
To: andy@yumaworks.com
Cc: evoit@cisco.com, netconf@ietf.org
From: Martin Bjorklund <mbj@tail-f.com>
In-Reply-To: <CABCOCHRx6XNK9Ay9hzj3eL-HD9Exj0nk=J5g06Y__NZKLyzQ7A@mail.gmail.com>
References: <20170523.195720.334918098247517091.mbj@tail-f.com> <98ef4c64e750467ca9a35b66b359dc8d@XCH-RTP-013.cisco.com> <CABCOCHRx6XNK9Ay9hzj3eL-HD9Exj0nk=J5g06Y__NZKLyzQ7A@mail.gmail.com>
X-Mailer: Mew version 6.7 on Emacs 24.5 / Mule 6.0 (HANACHIRUSATO)
Mime-Version: 1.0
Content-Type: Text/Plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/XKmu991_5gDa5OIgz5MepnT05Os>
Subject: Re: [Netconf] In an update, when is a delete a delete?
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Network Configuration WG mailing list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 24 May 2017 07:42:23 -0000
Andy Bierman <andy@yumaworks.com> wrote: > On Tue, May 23, 2017 at 3:16 PM, Eric Voit (evoit) <evoit@cisco.com> wrote: > > > > From: Martin Bjorklund, May 23, 2017 1:57 PM > > > > > > "Eric Voit (evoit)" <evoit@cisco.com> wrote: > > > > > Martin Bjorklund, May 22, 2017 3:30 PM > > > > > > > > > > Alexander Clemm <alexander.clemm@huawei.com> wrote: > > > > > > Hi Martin, > > > > > > > > > > > > Almost overlooked your question below. What is meant by the > > > > > > filter is specified in section 3.5 of the YANG-Push document. > > > > > > > > > > > > "Only a single filter can be applied to a subscription at a time. > > > > > > The following filter types are included in the yang-push data > > model: > > > > > > [subtree] [xpath]" > > > > > > > > > > Actually, only "subtree" is defined in yang-push, "xpath" is defined > > > > > in subscribed-notifications. > > > > > > > > At the top of yang-push page 7, xpath selection is described. Is > > > > there something you feel missing? > > > > > > Yes. First of all, the YANG module defines an identity called "xpath", > > based on > > > "sn:filter". So this filter has nothing to do with selecting nodes in a > > datastore; > > > this filter is used to match against a generated notification record. > > > > To address this we could split the xpath identity into two types: > > "xpath-selection" and "xpath-boolean". These would have different > > definitions, but both reference http://www.w3.org/TR/1999/REC- > > xpath-19991116 . The difference is that one results in a xpath node-set > > expression, and the other an xpath Boolean expression. Make sense? > > > > > Second, the XPath filter is sorely underspecified. The XPath context is > > not > > > described, > > > > I understand and agree on your comment about the context. The intent here > > is to provide equivalent capabilities of a GET. As it would be a huge > > undertaking to try to consolidate an industry-wide view of the minimal > > xpath syntax and capabilities in networking, I am hoping this doesn't fall > > under the umbrella of YANG subscription. I would be glad to support > > someone who wishes to take this up though. > > > > > > subtree and XPath filtering are implemented in various ways which could all > be considered valid. > The node-set result is not the same across server implementations. > > For example: > > <filter> > <foo /> > <foo> > <bar>1</bar> > </foo> > </filter> > > Does the server return the entire /foo subtree or return the entire /foo > subtree > followed by another subtree for /foo[bar=1]? (yes. that's the problem) > XPath has many more ways to overlap node-sets. Actually, for XPath the situation is better, since the XPath evaluation will return a node set (which does not contain duplicates, regardless of the expression), and the resulting node set is returned (in the case of get). As for subtree filter, RFC 6241 says: Specific data instances are not duplicated in the response in the event that the request contains multiple filter subtree expressions that select the same data. > Should we even document all the under-implemented servers that do not > support the > complete XPath syntax? Probably not, but it would be worthwhile to have an > "XPath-core" > that is widely supported. > > I agree that YANG Push is not the correct document for this sort of work. > > Perhaps YANG Push filtering should be 2 phases: (1) select + (2) test), > similar to how XSLT works. > (1) select can be simple and static, like a node-instance-identifier > expression from NACM > (2) test can be an XPath boolean expression run against each node selected > in (1), > using the selected node as the context node I wonder if (2) is even needed... /martin > > > Andy > > > > > > the expected result data type is not defined, and it is not described > > > how the result is supposed to be used. > > > > As for the result, the anydata output should be provided to the subscriber > > (with appropriate security applied). They can determine how to use it. > > The preferred embodiment would be to maintain a local extract of the > > Publisher's datastore (as defined by the filter). > > > > > > > [Side note - I think this is wrong, subscribed-notifications should > > > > > also define "subtree".] > > > > > > > > > > But these filters are used by the server to decide if a certain > > > > > notification that has been generated will be sent to the client or > > > > > not. > > > > > > > > Yes, the filters in subscribed-notification are supposed to give a > > > > boolean indication as to whether a specific event should traverse the > > > > filter in its entirety. RFC6241 section 6 subtree filters are written > > > > to provide a subset of content. I suppose it would be possible to > > > > define an event-based subtree-filter-type where a non-null result of > > > > the subtree filter means that a particular event should traverse that > > > > filter. Is this what you are suggesting? > > > > > > Yes. Note that this is already provided by RFC 5277, and I have always > > > assumed that this new work will provide at least the same functions as > > RFC > > > 5277 (and more). (But note that the XPath filter is underspecified also > > in RFC > > > 5277...) > > > > I also want to make sure that a non-null result from a filter allows the > > event to pass. I suspect that an xpath Boolean filter could be designed to > > do this, but I will tweak the subscribed-notifications text so that > > unnecessary filtering expression complexity is not artificially required. > > > > > > > If you want to define filters to specifify which nodes to subscribe > > > > > to, I think you need to define new filters, not try to reuese these > > > > > notification filters. > > > > > > > > Filtering syntax is hard, so we have been trying to adopt whatever is > > > > available for GET. This way we don't have to educate users on a new > > > > universe of what is possible. I fully expect that lots of learnings > > > > are going to come in the industry here over time, and this will be > > > > revisited in the future. > > > > > > ? > > > > > > I am not proposing any new filter syntax. I am saying that the current > > filter > > > nodes as defined in subscribed-notification cannot be used to select > > nodes to > > > subscribe to for changes. > > > > Understand. Hopefully with the "xpath-selection" change proposed above, > > this will be covered. > > > > Eric > > > > > > > As for your question, I think such a filter should be defined to > > > > > return a node- set to which the client subscribe to changes. If any > > > > > node (or subnode > > > > > to) in > > > > > this node-set changes, the notif will be sent. Then the question > > > > > about value comparision is not relevant anymore. > > > > > > > > Excellent, on-change should only send an update if the results of the > > > > subscription filter have changed since the previous push. It is quite > > > > possible that an object has been created and then deleted since the > > > > last push. > > > > > > I don't understand what you're trying to say with these sentences (but > > since > > > the first word was "Excellent" maybe it's ok ;) > > > > > > > > > /martin > > > > > > > > > > Representing this was the genesis of Alex's question. > > > > > > > > Eric > > > > > > > > > /martin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In effect, the filter specifies which data nodes to consider when > > > > > > sending updates. > > > > > > > > > > > > ---Alex > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Martin Bjorklund [mailto:mbj@tail-f.com] > > > > > > Sent: Wednesday, May 17, 2017 11:50 PM > > > > > > To: Alexander Clemm <alexander.clemm@huawei.com> > > > > > > Cc: netconf@ietf.org > > > > > > Subject: Re: [Netconf] In an update, when is a delete a delete? > > > > > > > > > > > > Hi, > > > > > > > > > > > > Alexander Clemm <alexander.clemm@huawei.com> wrote: > > > > > > > Hello all, > > > > > > > > > > > > > > In updating the YANG-Push document > > > > > > > (draft-ietf-netconf-yang-push), we have come across one issue > > > > > > > that we wanted to raise with the working group. > > > > > > > > > > > > > > As part of an on-change subscription, update records reflect the > > > > > > > type of change (e.g. whether the value of an object has changed, > > > > > > > or whether an object was created or deleted); a subscription > > > > > > > allows also to specify whether interested only in specific types > > > > > > > of changes (for example, only creates and deleted but no value > > > changes). > > > > > > > > > > > > > > At the same time, a subscription filter specifies which objects > > > > > > > to include as part of a subscription and which not. > > > > > > > > > > > > Hmm, which filter are you talking about? The only XPath filter I > > > > > > find in the current set of documents is the > > > > > > "ietf-subscribed-notifications:xpath" filter type (which btw is > > > > > > sorely underspecified). Section 2.2 of > > > > > > draft-ietf-netconf-subscribed-notifications-02 says: > > > > > > > > > > > > Events which evaluate to "true" as a > > > > > > result of the evaluation by the filter must traverse the filter > > in > > > > > > their entirety. > > > > > > > > > > > > It's not clear what this means, but my guess is that this is > > > > > > supposed to work like the old RFC 5277 filters, where the filter > > > > > > expression is evaluated on the notification contents, and if the > > > > > > expression returns "true" (for XPath filters this means converting > > > > > > the results to a boolean), then the notification is sent, > > otherwise not. > > > > > > > > > > > > But it seems you are referring to some other filter which would be > > > > > > used to select a node set for which changes are reported? > > > > > > > > > > > > I would like to understand which filter mechanism you mean before > > > > > > having an opinion in this matter. > > > > > > > > > > > > > > > > > > /martin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > (Really, it is not > > > > > > > so much of a "filter" on a stream that is generated > > > > > > > independently of the filter, than it is a policy of which > > > > > > > objects to include as part of subscribed update records.) > > > > > > > However, a subscription filter (such as > > > > > > > XPath) can be used to also specify a value filter, which will > > > > > > > include or exclude objects based on their current value. This > > > > > > > makes it possible to e.g. subscribe to an object "foo" but only > > > > > > > if its value is 5. > > > > > > > > > > > > > > Now, this means that the same object could be included in one > > > > > > > update, but excluded in another update, due to its value no > > > > > > > longer meeting the filter criteria. For example, if foo's value > > > > > > > changes from 5 to 3 in one cycle, a periodic subscription will > > > > > > > no longer include foo in its next update. The question now > > > > > > > concerns how to properly handle this in the case of an on-change > > > subscription. > > > > > > > > > > > > > > One possibility concerns reporting the fact that "foo" no longer > > > > > > > meets the subscription criteria and is no longer included in the > > > > > > > update record as a "delete" event. If foo's value again becomes > > "5" > > > > > > > at a later point in time, that would be reported as a "create" > > > > > > > event. If foo's value changes again from 5 at a later point in > > > > > > > time and then changes back to 3 before the time of the update > > > > > > > (perhaps because the value changed during the dampening > > > > > > > interval), it would be reported as another "delete" event > > > > > > > (without ever reporting a create event). On the other hand, if > > > > > > > foo's value changed from 3 to > > > > > > > 6 and back again, nothing would be reported because it did not > > > > > > > meet the filter criteria at any point in time. > > > > > > > > > > > > > > >From the perspective of the receiver this may make sense if it > > > > > > > >is synching its copy of the state. However, from the > > > > > > > >perspective of the publisher, the object was never created or > > > > > > > >deleted - only its value changed, and the case when the object > > > > > > > >was truly created or deleted can no longer be distinguished > > > > > > > >from the case when its value > > > > > changed. > > > > > > > >A "create" simply means "an object now meets a filter criteria, > > > > > > > >that was not reported in the previous cycle" (which does not > > > > > > > >mean that the object was actually created - it may have been > > > > > > > >created, or it may have simply undergone a value change). > > > > > > > > > > > > > > An alternative (let's call it alternative 2) is therefore to > > > > > > > make a distinction between whether an object was created or > > > > > > > deleted, or whether its value fell in or out of a filter range. > > > > > > > This appears semantically cleaner. However, it will require > > > > > > > modifying the encoding to allow for distinction between those > > > > > > > cases (currently, just plain patch encoding is used). > > > > > > > > > > > > > > A third alternative is to let filters select only data nodes to > > > > > > > subscribe to, and separate out the value filter (or disallow it > > > > > > > as a feature altogether). This alternative has the drawback of > > > > > > > being less conceptually powerful, even if it may be easier to > > > implement. > > > > > > > > > > > > > > Thoughts? Any preferences between 1, 2, and 3? > > > > > > > --- Alex > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Netconf mailing list > > > > > Netconf@ietf.org > > > > > https://www.ietf.org/mailman/listinfo/netconf > > > > > > > > _______________________________________________ > > Netconf mailing list > > Netconf@ietf.org > > https://www.ietf.org/mailman/listinfo/netconf > >
- [Netconf] In an update, when is a delete a delete? Alexander Clemm
- Re: [Netconf] In an update, when is a delete a de… Igor Bryskin
- Re: [Netconf] In an update, when is a delete a de… Randy Presuhn
- Re: [Netconf] In an update, when is a delete a de… Martin Bjorklund
- Re: [Netconf] In an update, when is a delete a de… Andy Bierman
- Re: [Netconf] In an update, when is a delete a de… Eric Voit (evoit)
- Re: [Netconf] In an update, when is a delete a de… Eric Voit (evoit)
- Re: [Netconf] In an update, when is a delete a de… Kent Watsen
- Re: [Netconf] In an update, when is a delete a de… Andy Bierman
- Re: [Netconf] In an update, when is a delete a de… Alexander Clemm
- Re: [Netconf] In an update, when is a delete a de… Andy Bierman
- Re: [Netconf] In an update, when is a delete a de… Alexander Clemm
- Re: [Netconf] In an update, when is a delete a de… Andy Bierman
- Re: [Netconf] In an update, when is a delete a de… Alexander Clemm
- Re: [Netconf] In an update, when is a delete a de… Andy Bierman
- Re: [Netconf] In an update, when is a delete a de… Juergen Schoenwaelder
- Re: [Netconf] In an update, when is a delete a de… Andy Bierman
- Re: [Netconf] In an update, when is a delete a de… Juergen Schoenwaelder
- Re: [Netconf] In an update, when is a delete a de… Andy Bierman
- Re: [Netconf] In an update, when is a delete a de… Juergen Schoenwaelder
- Re: [Netconf] In an update, when is a delete a de… Alexander Clemm
- Re: [Netconf] In an update, when is a delete a de… Martin Bjorklund
- Re: [Netconf] In an update, when is a delete a de… Eric Voit (evoit)
- Re: [Netconf] In an update, when is a delete a de… Alexander Clemm
- Re: [Netconf] In an update, when is a delete a de… Martin Bjorklund
- Re: [Netconf] In an update, when is a delete a de… Eric Voit (evoit)
- Re: [Netconf] In an update, when is a delete a de… Andy Bierman
- Re: [Netconf] In an update, when is a delete a de… Martin Bjorklund
- Re: [Netconf] In an update, when is a delete a de… Martin Bjorklund
- Re: [Netconf] In an update, when is a delete a de… Alexander Clemm
- Re: [Netconf] In an update, when is a delete a de… Andy Bierman
- Re: [Netconf] In an update, when is a delete a de… Eric Voit (evoit)
- Re: [Netconf] In an update, when is a delete a de… Martin Bjorklund
- Re: [Netconf] In an update, when is a delete a de… Robert Wilton
- Re: [Netconf] In an update, when is a delete a de… Martin Bjorklund
- Re: [Netconf] In an update, when is a delete a de… Eric Voit (evoit)
- Re: [Netconf] In an update, when is a delete a de… Martin Bjorklund
- Re: [Netconf] In an update, when is a delete a de… Eric Voit (evoit)
- Re: [Netconf] In an update, when is a delete a de… Martin Bjorklund
- Re: [Netconf] In an update, when is a delete a de… Martin Bjorklund
- Re: [Netconf] In an update, when is a delete a de… Eric Voit (evoit)
- Re: [Netconf] In an update, when is a delete a de… Martin Bjorklund
- Re: [Netconf] In an update, when is a delete a de… Eric Voit (evoit)
- Re: [Netconf] In an update, when is a delete a de… Alexander Clemm