Re: [netconf] get-data origin filters
Martin Bjorklund <mbj@tail-f.com> Mon, 07 October 2019 19:04 UTC
Return-Path: <mbj@tail-f.com>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 279CE12008A for <netconf@ietfa.amsl.com>; Mon, 7 Oct 2019 12:04:07 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.89
X-Spam-Level:
X-Spam-Status: No, score=-1.89 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_FILL_THIS_FORM_SHORT=0.01] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fSItKG81f8FO for <netconf@ietfa.amsl.com>; Mon, 7 Oct 2019 12:04:04 -0700 (PDT)
Received: from mail.tail-f.com (mail.tail-f.com [46.21.102.45]) by ietfa.amsl.com (Postfix) with ESMTP id 054D61200E9 for <netconf@ietf.org>; Mon, 7 Oct 2019 12:04:03 -0700 (PDT)
Received: from localhost (h-4-44.A165.priv.bahnhof.se [158.174.4.44]) by mail.tail-f.com (Postfix) with ESMTPSA id 221F61AE018A; Mon, 7 Oct 2019 21:04:02 +0200 (CEST)
Date: Mon, 07 Oct 2019 21:04:01 +0200
Message-Id: <20191007.210401.1161655948996670044.mbj@tail-f.com>
To: andy@yumaworks.com
Cc: rwilton@cisco.com, netconf@ietf.org
From: Martin Bjorklund <mbj@tail-f.com>
In-Reply-To: <CABCOCHTrMg-zA_3GrdxQC7dMLhBgCw5L04JjYcRbeU5V+0K7dw@mail.gmail.com>
References: <CABCOCHRS=J24hFth=OS2RNrE6WErSovpaCyQ9KP1Q3J_HYn7aw@mail.gmail.com> <20191007.193909.1640793317135302246.mbj@tail-f.com> <CABCOCHTrMg-zA_3GrdxQC7dMLhBgCw5L04JjYcRbeU5V+0K7dw@mail.gmail.com>
X-Mailer: Mew version 6.7 on Emacs 25.2 / Mule 6.0 (HANACHIRUSATO)
Mime-Version: 1.0
Content-Type: Text/Plain; charset="utf-8"
Content-Transfer-Encoding: base64
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/nes2WR34YWRSFUCubJrVoCrj4Ok>
Subject: Re: [netconf] get-data origin filters
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 07 Oct 2019 19:04:07 -0000
Andy Bierman <andy@yumaworks.com> wrote: > On Mon, Oct 7, 2019 at 10:39 AM Martin Bjorklund <mbj@tail-f.com> wrote: > > > Andy Bierman <andy@yumaworks.com> wrote: > > > On Mon, Oct 7, 2019 at 7:36 AM Rob Wilton (rwilton) <rwilton@cisco.com> > > > wrote: > > > > > > > Hi Andy, > > > > > > > > > > > > > > > > Don’t all the filters effectively work this way? > > > > > > > > > > > > > I do not see the text that explains origin-filter and > > negated-origin-filter > > > working the way Martin > > > describes it. These filters do not say anywhere to select a node because > > > it has descendants > > > that match the origin filters. It says very clearly that the filter test > > > is on the specified node. > > > > Let's simplify the example somewhat. Suppose the client sends: > > > > <get-data> > > <datastore>ds:operational</datastore> > > <origin-filter>or:learned</origin-filter> > > <with-origin/> > > </get-data> > > > > Are you saying that b/c the top-level node doesn't have or:learned, > > this filter will return nothing? > > > > > > Yes -- the origin for addresses is "intended". > The text clearly says the origin-filter has to match the origin annotation > for the node. > The annotation for "address" can have only 1 value. > > I don't think this is implementable so I will just use > deviation(not-supported) and ignore it Fine; up to you! But the question is if the spec is unclear or not. From what I can tell so far, it seems it is not incorrect, but can probably be clarified, perhaps along the lines of Rob's explanation. /martin > > > Andy > > It seems you are assuming that the server first matches the top-level > > nodes, and if they match, then match descendants and so one. The text > > doesn't say that, however. It says: > > > > A configuration node matches the filter if its > > 'origin' annotation is derived from or equal to any of > > the given filter values. > > > > and: > > > > Any ancestor nodes (including list keys) of nodes selected by > > the filters are included in the response. > > > > So I think it is pretty clear that the origin filter matches the > > zipcode nodes in the example, and then the ancestors are included in > > the reply. > > > > > > > It also says the origin is derived from the origin annotation for that > > node. > > > Since only 1 instance of the origin annotation is allowed per node, there > > > is no way to tag > > > a node with multiple origins. > > > > Agreed. > > > > > > /martin > > > > > > > If implementation is too complex then people will just leave it out (w/ a > > > deviation). > > > It is unlikely that the instrumentation knows at any given instant all > > the > > > origin values > > > of all the descendant dynamic data at the instant the <get-data> request > > is > > > processed. > > > > > > > > > > > > > > > > > > > > They select a subset of the nodes to include in the response, and must > > > > also include all ancestor nodes and required list keys to the selected > > > > nodes, regardless of whether those ancestor/key nodes were also > > selected by > > > > the query. > > > > > > > > > > > > > > > > > > Yes. Understood. > > > Still does not explain how a filter for the list node selects descendant > > > nodes that match the origin filters. > > > > > > > > > > > > > E.g. a “config false” filter will still return “config true” nodes if > > they > > > > are ancestors or list keys to a descendant config false node. The same > > > > logic applies for xpath and origin filters as well. > > > > > > > > > > > > > > > > > > No they won't. > > > Where is that text? > > > > > > get-data config=filter=false > > > > > > This starts from top-level YANG nodes. > > > If the top-level YANG node is not config=false then the server will not > > > keep looking for descendants that match. > > > > > > > > > > > > > Thanks, > > > > Rob > > > > > > > > > > > > > > Andy > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *From:* netconf <netconf-bounces@ietf.org> *On Behalf Of *Andy Bierman > > > > *Sent:* 07 October 2019 15:12 > > > > *To:* Martin Bjorklund <mbj@tail-f.com> > > > > *Cc:* Netconf <netconf@ietf.org> > > > > *Subject:* Re: [netconf] get-data origin filters > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Oct 7, 2019 at 12:43 AM Martin Bjorklund <mbj@tail-f.com> > > wrote: > > > > > > > > Andy Bierman <andy@yumaworks.com> wrote: > > > > > On Sun, Oct 6, 2019 at 8:32 AM Martin Bjorklund <mbj@tail-f.com> > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > Andy Bierman <andy@yumaworks.com> wrote: > > > > > > > Hi, > > > > > > > > > > > > > > I am trying to figure out how to use the origin-filter and > > > > > > > negated-origin-filter > > > > > > > in the <get-data> operation in RFC 8526. > > > > > > > > > > > > > > > > > > > > > leaf-list origin-filter { > > > > > > > type or:origin-ref; > > > > > > > description > > > > > > > "Filter based on the 'origin' annotation. A > > > > > > > configuration node matches the filter if its > > 'origin' > > > > > > > annotation is derived from or equal to any of the > > > > given > > > > > > > filter values."; > > > > > > > } > > > > > > > > > > > > > > > > > > > > > These filters seem kind of worthless if implemented according to > > the > > > > > > text. > > > > > > > Consider a simple example where there is 1 learned leaf within a > > > > list: > > > > > > > > > > > > > > module: address > > > > > > > +--rw addresses > > > > > > > +--rw address* [last-name first-name] > > > > > > > +--rw last-name string > > > > > > > +--rw first-name string > > > > > > > +--rw street? string > > > > > > > +--rw city? string > > > > > > > +--rw zipcode? string > > > > > > > +--rw phone* [phone-type] > > > > > > > +--rw phone-type enumeration > > > > > > > +--rw phone-number string > > > > > > > > > > > > > > Let's say the "zipcode" field is learned in <operational> > > > > > > > (e.g. ZIP code lookup expands missing or 5 digit zipcode to full > > 9 > > > > digit > > > > > > > zipcode). > > > > > > > So /addresses and /addresses/address have origin "intended". > > > > > > > Only the /addresses/address/zipcode leaf has origin "learned". > > > > > > > > > > > > > > So how does origin-filter=learned find all the learned leafs? > > > > > > > > > > > > Perhaps I don't understand your question; IMO you give the answer > > to > > > > > > this question below: > > > > > > > > > > > > > What filters are required to return only the learned entries + > > > > ancestors > > > > > > + > > > > > > > ancestor-or-self keys? Seems like this filter mechanism has to > > be > > > > used > > > > > > > to retrieve the exact leaf that might be learned, and the client > > > > > > > needs to know in advance all the possible nodes that might be > > > > learned. > > > > > > > > > > > > > > Want to be able to retrieve an ancestor that is intended and > > still > > > > find > > > > > > the > > > > > > > learned entries > > > > > > > > > > > > > > get-data xpath-filter=/addresses/address > > origin-filtter=learned > > > > > > > > > > > > ... here. So this request will return: > > > > > > > > > > > > <addresses or:origin="or:intended"> > > > > > > <address> > > > > > > <last-name>...</last-name> > > > > > > <first-name>...</first-name> > > > > > > <zipcode or:origin="or:learned">...</zipcode> > > > > > > </address> > > > > > > ... > > > > > > </addresses> > > > > > > > > > > > > > > > > > I do not interpret the text the same way as you. > > > > > > > > Does this mean that you think that the reply is different from what I > > > > show above? If so, what would it be, and why? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Explain how the list address node has origin "learned". > > > > > > > > > > > > > > > > The filter is for /addresses/address and only origin=learned. > > > > > > > > How does the list node have origin=learned? > > > > > > > > It can only have 1 value. > > > > > > > > It has child nodes with both intended and learned as origin. > > > > > > > > I do no understand how the origin=learned matched this node. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The content returned > > > > > > > > > > by get-data must satisfy all filters, i.e., the filter > > > > > criteria are logically ANDed. > > > > > > > > > > > > > > > leaf-list origin-filter { > > > > > type or:origin-ref; > > > > > description > > > > > "Filter based on the 'origin' annotation. A > > > > > configuration node matches the filter if its 'origin' > > > > > annotation is derived from or equal to any of the > > given > > > > > filter values."; > > > > > } > > > > > > > > > > > > > > > Configuration nodes that do not have an > > > > > 'origin' annotation are treated as if they have the > > > > > 'origin' annotation 'or:unknown'. > > > > > > > > > > > > > > > > > > > > > The draft shows an example where both "intended" and "system" are > > given > > > > > > > as filters. This will work but will include all the "intended" > > > > leafs as > > > > > > > well. > > > > > > > What if a "learned" node is within a "system" node within an > > > > "intended" > > > > > > > node? > > > > > > > > > > > > This works as well. Note that the get-data description says: > > > > > > > > > > > > Any ancestor nodes (including list keys) of nodes > > selected by > > > > > > the filters are included in the response. > > > > > > > > > > > > > > > > > > > > > > > > > > > > The issue is how the /iaddresses and /addresses/address nodes match > > the > > > > > origin "learned". > > > > > > > > They don't, but they are included b/c of the quoted text above (i.e.: > > > > Any ancestor nodes (including list keys) of nodes selected by > > > > the filters are included in the response.) > > > > > > > > > > > > > > > > > > > > > > > > No. > > > > > > > > > > > > > > > > If the filter was for /addresses/address/zipcode then maybe that text > > > > applies. > > > > > > > > It is still unclear that the XPath is fully processed and then the > > > > origin-filter is processed. > > > > > > > > The RFC just says they are ANDed together. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The leafs in list "address" are a mixture of "intended" and "learned" > > > > > origin. > > > > > The text clearly says that a node has a single origin property, > > coupled > > > > to > > > > > the annotation. > > > > > > > > > > Issue 1: mixed origin descendant nodes > > > > > So how does a search on /addresses/address match > > origin-filter=learned? > > > > > I cannot find any text that says what the origin of a list or > > P-container > > > > > is if it > > > > > contains nodes of mixed origin. > > > > > > > > See above. > > > > > > > > > > > > > > > > No text above explains how the list origin is tagged if it has multiple > > > > types of child nodes. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Issue 2: NP-containers > > > > > > > > > > Also from RFC 8342: > > > > > > > > > > The origin applies to all configuration nodes except non-presence > > > > > containers. > > > > > > > > > > > > > > > What if the top-level node is an NP-container in this case. > > > > > I thought the top-level node MUST have an origin attribute. > > > > > > > > > > The text is not clear how NP-containers are handled. > > > > > Do they have an origin attribute? If not then RFC 8526 says they have > > > > > origin "unknown". > > > > > Is the intent that NP-containers always pass the origin-filter tests > > > > (test > > > > > skipped)? > > > > > > > > No, since they don't have an origin value they will not be selected by > > > > the filter. But an NP-container will be included in the reply if it > > > > is the ancestor of a node that is selected by the filter. > > > > > > > > > > > > > > > > The RFC text does not really say that. > > > > > > > > Since it is very difficult to know if a data node 5 layers deep is > > going > > > > to match, > > > > > > > > implementing these filters according to this vague interpretation is > > > > unlikely. > > > > > > > > > > > > > > > > > > > > /martin > > > > > > > > > > > > > > > > Andy > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /martin > > > > > > > > > > > > > > > > > Andy > > > > > > > > > > > > > > > > > > > > > > > Seems like the client needs to know a lot about the server > > > > implementation > > > > > > > details > > > > > > > in order to use the origin filters. > > > > > > > > > > > > > > > > > > > > > Andy > > > > > > > > > > > > > > > >
- [netconf] get-data origin filters Andy Bierman
- Re: [netconf] get-data origin filters Martin Bjorklund
- Re: [netconf] get-data origin filters Andy Bierman
- Re: [netconf] get-data origin filters Martin Bjorklund
- Re: [netconf] get-data origin filters Andy Bierman
- Re: [netconf] get-data origin filters Rob Wilton (rwilton)
- Re: [netconf] get-data origin filters Andy Bierman
- Re: [netconf] get-data origin filters Rob Wilton (rwilton)
- Re: [netconf] get-data origin filters Andy Bierman
- Re: [netconf] get-data origin filters Martin Bjorklund
- Re: [netconf] get-data origin filters Andy Bierman
- Re: [netconf] get-data origin filters Martin Bjorklund