Re: [netconf] get-data origin filters

Andy Bierman <andy@yumaworks.com> Mon, 07 October 2019 17:50 UTC

Return-Path: <andy@yumaworks.com>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E05A512010F for <netconf@ietfa.amsl.com>; Mon, 7 Oct 2019 10:50:34 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.889
X-Spam-Level:
X-Spam-Status: No, score=-1.889 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_FILL_THIS_FORM_SHORT=0.01] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=yumaworks-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KegbhF72KFYp for <netconf@ietfa.amsl.com>; Mon, 7 Oct 2019 10:50:31 -0700 (PDT)
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 68B7012088E for <netconf@ietf.org>; Mon, 7 Oct 2019 10:50:30 -0700 (PDT)
Received: by mail-lf1-x134.google.com with SMTP id r2so9897336lfn.8 for <netconf@ietf.org>; Mon, 07 Oct 2019 10:50:30 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yumaworks-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mPYbuyzaot960tCqRnmoRRA/eWBMjFzpN+aGSJSoea0=; b=J5qxL9ernBX3eYa/qWtvYAkGln7jrQ9ZUYJYnjOxms20xTfcC9a7bHb6kekCY9t4Z4 ABVzYgS5xUN9JWVIB3zoiR8+CnNvB29JtR1edMsk+9Z65KhWR1Z+fjDxbnE20yqbEDQM pchDogHzktNi94ZNGe83MIos1CeLHpm7PVpxcDVTf3jHAgYMFlnMc93/6KmQiHreaHif yqDIt75IADxyY4fKOSjc+pUoUsbg+QODQ+/tpDRHPgp7dKEdps01ey/Ke5s4uABGPycv jRw3bIAmkB7ZzTfAtTqKFGZ5Slb8n6V3nEzfurOS/Ezj1P0Nk93mjsxWj0ADff54CsjC d+4Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mPYbuyzaot960tCqRnmoRRA/eWBMjFzpN+aGSJSoea0=; b=G7ARhZ47cyP2y507/ARjW4cF7379TW6fXt2XQ9gx9rfq+LkEhqS4RRyZItI5l64StG fAUAydz6OlmRX7uu2J8ELl4Uy2oTrysWEOEr8Aco5Oi5NxZY01CbOCDdIgoslgwKRn9d VNDGPYfG+XqsfsSCiZyxy+VjC7KESDzeXSOnQl0FarKs9qV0i46jqu3BsLXZH4mX8jfG QFMkixE64Oft0oHR5BDyWAwKX+LUddxQUtyQffZFr2a3jIgP1b28XHADOezA3GgjZ6O+ 9XEkrpTcLPQ/yjcINc0gVWNCkIt9eW+/YmbzX+dTS1QkZlflpCVqvLo8S3e4Ci6AC1AD f0Hg==
X-Gm-Message-State: APjAAAVgcPr1QlMoo7gVpkR/tX1ywFrErvYPl5FeAdydbzSgu2BZ5rvU nkaiR7H18Z0PDQwWrSiME9GDpPbSN84MKb7DCyd35w==
X-Google-Smtp-Source: APXvYqzXDZpa7qY8qLjWpmtrfeVezRXe/8TbHOVYWWrC117FBZcC6/+qDGvrmNQCkiFqmpsTeo4EKB7aeg3Bw9SI4P4=
X-Received: by 2002:a19:7610:: with SMTP id c16mr14681656lff.51.1570470628398; Mon, 07 Oct 2019 10:50:28 -0700 (PDT)
MIME-Version: 1.0
References: <CABCOCHSMRrL4VR7eR8sQCtMnmg5=EE0d8g37Vr956vkUtVTBQA@mail.gmail.com> <MN2PR11MB4366BB8F556DE7DC866FE27BB59B0@MN2PR11MB4366.namprd11.prod.outlook.com> <CABCOCHRS=J24hFth=OS2RNrE6WErSovpaCyQ9KP1Q3J_HYn7aw@mail.gmail.com> <20191007.193909.1640793317135302246.mbj@tail-f.com>
In-Reply-To: <20191007.193909.1640793317135302246.mbj@tail-f.com>
From: Andy Bierman <andy@yumaworks.com>
Date: Mon, 07 Oct 2019 10:50:17 -0700
Message-ID: <CABCOCHTrMg-zA_3GrdxQC7dMLhBgCw5L04JjYcRbeU5V+0K7dw@mail.gmail.com>
To: Martin Bjorklund <mbj@tail-f.com>
Cc: Robert Wilton <rwilton@cisco.com>, Netconf <netconf@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000efb1b7059455afb6"
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/_9LDg-VAhG0ACehWROt0WPmqEXc>
Subject: Re: [netconf] get-data origin filters
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 07 Oct 2019 17:50:35 -0000

On Mon, Oct 7, 2019 at 10:39 AM Martin Bjorklund <mbj@tail-f.com> wrote:

> Andy Bierman <andy@yumaworks.com> wrote:
> > On Mon, Oct 7, 2019 at 7:36 AM Rob Wilton (rwilton) <rwilton@cisco.com>
> > wrote:
> >
> > > Hi Andy,
> > >
> > >
> > >
> > > Don’t all the filters effectively work this way?
> > >
> >
> >
> > I do not see the text that explains origin-filter and
> negated-origin-filter
> > working the way Martin
> > describes it.  These filters do not say anywhere to select a node because
> > it has descendants
> > that match the origin filters.  It says very clearly that the filter test
> > is on the specified node.
>
> Let's simplify the example somewhat.  Suppose the client sends:
>
>   <get-data>
>     <datastore>ds:operational</datastore>
>     <origin-filter>or:learned</origin-filter>
>     <with-origin/>
>   </get-data>
>
> Are you saying that b/c the top-level node doesn't have or:learned,
> this filter will return nothing?
>
>

Yes -- the origin for addresses is "intended".
The text clearly says the origin-filter has to match the origin annotation
for the node.
The annotation for "address" can have only 1 value.

I don't think this is implementable so I will just use
deviation(not-supported) and ignore it


Andy

It seems you are assuming that the server first matches the top-level
> nodes, and if they match, then match descendants and so one.  The text
> doesn't say that, however.  It says:
>
>                 A configuration node matches the filter if its
>                 'origin' annotation is derived from or equal to any of
>                 the given filter values.
>
> and:
>
>           Any ancestor nodes (including list keys) of nodes selected by
>           the filters are included in the response.
>
> So I think it is pretty clear that the origin filter matches the
> zipcode nodes in the example, and then the ancestors are included in
> the reply.
>
>
> > It also says the origin is derived from the origin annotation for that
> node.
> > Since only 1 instance of the origin annotation is allowed per node, there
> > is no way to tag
> > a node with multiple origins.
>
> Agreed.
>
>
> /martin
>
>
> > If implementation is too complex then people will just leave it out (w/ a
> > deviation).
> > It is unlikely that the instrumentation knows at any given instant all
> the
> > origin values
> > of all the descendant dynamic data at the instant the <get-data> request
> is
> > processed.
> >
> >
> >
> >
> > >
> > > They select a subset of the nodes to include in the response, and must
> > > also include all ancestor nodes and required list keys to the selected
> > > nodes, regardless of whether those ancestor/key nodes were also
> selected by
> > > the query.
> > >
> > >
> > >
> >
> > Yes. Understood.
> > Still does not explain how a filter for the list node selects descendant
> > nodes that match the origin filters.
> >
> >
> >
> > > E.g. a “config false” filter will still return “config true” nodes if
> they
> > > are ancestors or list keys to a descendant config false node.  The same
> > > logic applies for xpath and origin filters as well.
> > >
> > >
> > >
> >
> > No they won't.
> > Where is that text?
> >
> >     get-data config=filter=false
> >
> > This starts from top-level YANG nodes.
> > If the top-level YANG node is not config=false then the server will not
> > keep looking for descendants that match.
> >
> >
> >
> > > Thanks,
> > > Rob
> > >
> > >
> >
> > Andy
> >
> >
> >
> > >
> > >
> > >
> > >
> > > *From:* netconf <netconf-bounces@ietf.org> *On Behalf Of *Andy Bierman
> > > *Sent:* 07 October 2019 15:12
> > > *To:* Martin Bjorklund <mbj@tail-f.com>
> > > *Cc:* Netconf <netconf@ietf.org>
> > > *Subject:* Re: [netconf] get-data origin filters
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Oct 7, 2019 at 12:43 AM Martin Bjorklund <mbj@tail-f.com>
> wrote:
> > >
> > > Andy Bierman <andy@yumaworks.com> wrote:
> > > > On Sun, Oct 6, 2019 at 8:32 AM Martin Bjorklund <mbj@tail-f.com>
> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Andy Bierman <andy@yumaworks.com> wrote:
> > > > > > Hi,
> > > > > >
> > > > > > I am trying to figure out how to use the origin-filter and
> > > > > > negated-origin-filter
> > > > > > in the <get-data> operation in RFC 8526.
> > > > > >
> > > > > >
> > > > > >           leaf-list origin-filter {
> > > > > >              type or:origin-ref;
> > > > > >              description
> > > > > >                "Filter based on the 'origin' annotation.  A
> > > > > >                 configuration node matches the filter if its
> 'origin'
> > > > > >                 annotation is derived from or equal to any of the
> > > given
> > > > > >                 filter values.";
> > > > > >            }
> > > > > >
> > > > > >
> > > > > > These filters seem kind of worthless if implemented according to
> the
> > > > > text.
> > > > > > Consider a simple example where there is 1 learned leaf within a
> > > list:
> > > > > >
> > > > > > module: address
> > > > > >   +--rw addresses
> > > > > >      +--rw address* [last-name first-name]
> > > > > >         +--rw last-name     string
> > > > > >         +--rw first-name    string
> > > > > >         +--rw street?       string
> > > > > >         +--rw city?         string
> > > > > >         +--rw zipcode?      string
> > > > > >         +--rw phone* [phone-type]
> > > > > >            +--rw phone-type      enumeration
> > > > > >            +--rw phone-number    string
> > > > > >
> > > > > > Let's say the "zipcode" field is learned in <operational>
> > > > > > (e.g. ZIP code lookup expands missing or 5 digit zipcode to full
> 9
> > > digit
> > > > > > zipcode).
> > > > > > So /addresses and /addresses/address have origin "intended".
> > > > > > Only the /addresses/address/zipcode leaf has origin "learned".
> > > > > >
> > > > > > So how does origin-filter=learned find all the learned leafs?
> > > > >
> > > > > Perhaps I don't understand your question; IMO you give the answer
> to
> > > > > this question below:
> > > > >
> > > > > > What filters are required to return only the learned entries +
> > > ancestors
> > > > > +
> > > > > > ancestor-or-self keys?  Seems like this filter mechanism has to
> be
> > > used
> > > > > > to retrieve the exact leaf that might be learned, and the client
> > > > > > needs to know in advance all the possible nodes that might be
> > > learned.
> > > > > >
> > > > > > Want to be able to retrieve an ancestor that is intended and
> still
> > > find
> > > > > the
> > > > > > learned entries
> > > > > >
> > > > > >    get-data xpath-filter=/addresses/address
> origin-filtter=learned
> > > > >
> > > > > ... here.  So this request will return:
> > > > >
> > > > >    <addresses or:origin="or:intended">
> > > > >      <address>
> > > > >        <last-name>...</last-name>
> > > > >        <first-name>...</first-name>
> > > > >        <zipcode or:origin="or:learned">...</zipcode>
> > > > >      </address>
> > > > >      ...
> > > > >    </addresses>
> > > > >
> > > > >
> > > > I do not interpret the text the same way as you.
> > >
> > > Does this mean that you think that the reply is different from what I
> > > show above?  If so, what would it be, and why?
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > Explain how the list address node has origin "learned".
> > >
> > >
> > >
> > > The filter is for /addresses/address and only origin=learned.
> > >
> > > How does the list node have origin=learned?
> > >
> > > It can only have 1 value.
> > >
> > > It has child nodes with both intended and learned as origin.
> > >
> > > I do no understand how the origin=learned matched this node.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > >                      The content returned
> > > >
> > > >           by get-data must satisfy all filters, i.e., the filter
> > > >           criteria are logically ANDed.
> > > >
> > > >
> > > >           leaf-list origin-filter {
> > > >              type or:origin-ref;
> > > >              description
> > > >                "Filter based on the 'origin' annotation.  A
> > > >                 configuration node matches the filter if its 'origin'
> > > >                 annotation is derived from or equal to any of the
> given
> > > >                 filter values.";
> > > >            }
> > > >
> > > >
> > > >               Configuration nodes that do not have an
> > > >               'origin' annotation are treated as if they have the
> > > >               'origin' annotation 'or:unknown'.
> > > >
> > > >
> > > >
> > > > > The draft shows an example where both "intended" and "system" are
> given
> > > > > > as filters.  This will work but will include all the "intended"
> > > leafs as
> > > > > > well.
> > > > > > What if a "learned" node is within a "system" node within an
> > > "intended"
> > > > > > node?
> > > > >
> > > > > This works as well.  Note that the get-data description says:
> > > > >
> > > > >           Any ancestor nodes (including list keys) of nodes
> selected by
> > > > >           the filters are included in the response.
> > > > >
> > > > >
> > > > >
> > > >
> > > > The issue is how the /iaddresses and /addresses/address nodes match
> the
> > > > origin "learned".
> > >
> > > They don't, but they are included b/c of the quoted text above (i.e.:
> > >       Any ancestor nodes (including list keys) of nodes selected by
> > >       the filters are included in the response.)
> > >
> > >
> > >
> > >
> > >
> > > No.
> > >
> > >
> > >
> > > If the filter was for /addresses/address/zipcode then maybe that text
> > > applies.
> > >
> > > It is still unclear that the XPath is fully processed and then the
> > > origin-filter is processed.
> > >
> > > The RFC just says they are ANDed together.
> > >
> > >
> > >
> > >
> > >
> > >
> > > > The leafs in list "address" are a mixture of "intended" and "learned"
> > > > origin.
> > > > The text clearly says that a node has a single origin property,
> coupled
> > > to
> > > > the annotation.
> > > >
> > > > Issue 1: mixed origin descendant nodes
> > > > So how does a search on /addresses/address match
> origin-filter=learned?
> > > > I cannot find any text that says what the origin of a list or
> P-container
> > > > is if it
> > > > contains nodes of mixed origin.
> > >
> > > See above.
> > >
> > >
> > >
> > > No text above explains how the list origin is tagged if it has multiple
> > > types of child nodes.
> > >
> > >
> > >
> > >
> > >
> > >
> > > > Issue 2: NP-containers
> > > >
> > > > Also from RFC 8342:
> > > >
> > > >    The origin applies to all configuration nodes except non-presence
> > > >    containers.
> > > >
> > > >
> > > > What if the top-level node is an NP-container in this case.
> > > > I thought the top-level node MUST have an origin attribute.
> > > >
> > > > The text is not clear how NP-containers are handled.
> > > > Do they have an origin attribute? If not then RFC 8526 says they have
> > > > origin "unknown".
> > > > Is the intent that NP-containers always pass the origin-filter tests
> > > (test
> > > > skipped)?
> > >
> > > No, since they don't have an origin value they will not be selected by
> > > the filter.  But an NP-container will be included in the reply if it
> > > is the ancestor of a node that is selected by the filter.
> > >
> > >
> > >
> > > The RFC text does not really say that.
> > >
> > > Since it is very difficult to know if a data node 5 layers deep is
> going
> > > to match,
> > >
> > > implementing these filters according to this vague interpretation is
> > > unlikely.
> > >
> > >
> > >
> > >
> > > /martin
> > >
> > >
> > >
> > > Andy
> > >
> > >
> > >
> > >
> > >
> > > >
> > > >
> > > >
> > > > /martin
> > > > >
> > > > >
> > > > Andy
> > > >
> > > >
> > > > >
> > > > > > Seems like the client needs to know a lot about the server
> > > implementation
> > > > > > details
> > > > > > in order to use the origin filters.
> > > > > >
> > > > > >
> > > > > > Andy
> > > > >
> > >
> > >
>