Re: [netconf] get-data origin filters

["Rob Wilton (rwilton)" <[e-mail]>]

My understanding of the intention of the way the filters are logically meant to work are:


  1.  Construct the full response to the request (as if no filters are to be returned):
  2.  Restrict the response, so that the selected elements match any subtree filter.
  3.  Restrict the response, so that the selected elements match any xpath filter.
  4.  Restrict the response, so that the selected elements match any config true/false filter.
  5.  Restrict the response, so that the selected elements match any origin filter.
  6.  Constrain to the requested depth
  7.  Add in required ancestors and list keys.
  8.  Return the result.


I don’t see text in the filters that states that if a node is filtered then all of its descendants are automatically filtered as well.  I think that you are assuming this behaviour.

A node always only has a single origin, although it could change.  E.g. if a system configured was explicitly configured, then it would make sense to change its origin to configured because it would exist regardless of whether it was originally added by the system.

Thanks,
Rob


From: Andy Bierman <[e-mail]>
Sent: 07 October 2019 15:52
To: Rob Wilton (rwilton) <[e-mail]>
Cc: Martin Bjorklund <[e-mail]>; Netconf <[e-mail]>
Subject: Re: [netconf] get-data origin filters



On Mon, Oct 7, 2019 at 7:36 AM Rob Wilton (rwilton) <[e-mail]<mailto:[e-mail]>> wrote:
Hi Andy,

Don’t all the filters effectively work this way?


I do not see the text that explains origin-filter and negated-origin-filter working the way Martin
describes it.  These filters do not say anywhere to select a node because it has descendants
that match the origin filters.  It says very clearly that the filter test is on the specified node.
It also says the origin is derived from the origin annotation for that node.
Since only 1 instance of the origin annotation is allowed per node, there is no way to tag
a node with multiple origins.

If implementation is too complex then people will just leave it out (w/ a deviation).
It is unlikely that the instrumentation knows at any given instant all the origin values
of all the descendant dynamic data at the instant the <get-data> request is processed.




They select a subset of the nodes to include in the response, and must also include all ancestor nodes and required list keys to the selected nodes, regardless of whether those ancestor/key nodes were also selected by the query.


Yes. Understood.
Still does not explain how a filter for the list node selects descendant nodes that match the origin filters.


E.g. a “config false” filter will still return “config true” nodes if they are ancestors or list keys to a descendant config false node.  The same logic applies for xpath and origin filters as well.


No they won't.
Where is that text?

    get-data config=filter=false

This starts from top-level YANG nodes.
If the top-level YANG node is not config=false then the server will not keep looking for descendants that match.


Thanks,
Rob


Andy




From: netconf <[e-mail]<mailto:[e-mail]>> On Behalf Of Andy Bierman
Sent: 07 October 2019 15:12
To: Martin Bjorklund <[e-mail]<mailto:[e-mail]>>
Cc: Netconf <[e-mail]<mailto:[e-mail]>>
Subject: Re: [netconf] get-data origin filters



On Mon, Oct 7, 2019 at 12:43 AM Martin Bjorklund <[e-mail]<mailto:[e-mail]>> wrote:
Andy Bierman <[e-mail]<mailto:[e-mail]>> wrote:
> On Sun, Oct 6, 2019 at 8:32 AM Martin Bjorklund <[e-mail]<mailto:[e-mail]>> wrote:
>
> > Hi,
> >
> > Andy Bierman <[e-mail]<mailto:[e-mail]>> wrote:
> > > Hi,
> > >
> > > I am trying to figure out how to use the origin-filter and
> > > negated-origin-filter
> > > in the <get-data> operation in RFC 8526.
> > >
> > >
> > >           leaf-list origin-filter {
> > >              type or:origin-ref;
> > >              description
> > >                "Filter based on the 'origin' annotation.  A
> > >                 configuration node matches the filter if its 'origin'
> > >                 annotation is derived from or equal to any of the given
> > >                 filter values.";
> > >            }
> > >
> > >
> > > These filters seem kind of worthless if implemented according to the
> > text.
> > > Consider a simple example where there is 1 learned leaf within a list:
> > >
> > > module: address
> > >   +--rw addresses
> > >      +--rw address* [last-name first-name]
> > >         +--rw last-name     string
> > >         +--rw first-name    string
> > >         +--rw street?       string
> > >         +--rw city?         string
> > >         +--rw zipcode?      string
> > >         +--rw phone* [phone-type]
> > >            +--rw phone-type      enumeration
> > >            +--rw phone-number    string
> > >
> > > Let's say the "zipcode" field is learned in <operational>
> > > (e.g. ZIP code lookup expands missing or 5 digit zipcode to full 9 digit
> > > zipcode).
> > > So /addresses and /addresses/address have origin "intended".
> > > Only the /addresses/address/zipcode leaf has origin "learned".
> > >
> > > So how does origin-filter=learned find all the learned leafs?
> >
> > Perhaps I don't understand your question; IMO you give the answer to
> > this question below:
> >
> > > What filters are required to return only the learned entries + ancestors
> > +
> > > ancestor-or-self keys?  Seems like this filter mechanism has to be used
> > > to retrieve the exact leaf that might be learned, and the client
> > > needs to know in advance all the possible nodes that might be learned.
> > >
> > > Want to be able to retrieve an ancestor that is intended and still find
> > the
> > > learned entries
> > >
> > >    get-data xpath-filter=/addresses/address origin-filtter=learned
> >
> > ... here.  So this request will return:
> >
> >    <addresses or:origin="or:intended">
> >      <address>
> >        <last-name>...</last-name>
> >        <first-name>...</first-name>
> >        <zipcode or:origin="or:learned">...</zipcode>
> >      </address>
> >      ...
> >    </addresses>
> >
> >
> I do not interpret the text the same way as you.

Does this mean that you think that the reply is different from what I
show above?  If so, what would it be, and why?



Explain how the list address node has origin "learned".

The filter is for /addresses/address and only origin=learned.
How does the list node have origin=learned?
It can only have 1 value.
It has child nodes with both intended and learned as origin.
I do no understand how the origin=learned matched this node.




>
>                      The content returned
>
>           by get-data must satisfy all filters, i.e., the filter
>           criteria are logically ANDed.
>
>
>           leaf-list origin-filter {
>              type or:origin-ref;
>              description
>                "Filter based on the 'origin' annotation.  A
>                 configuration node matches the filter if its 'origin'
>                 annotation is derived from or equal to any of the given
>                 filter values.";
>            }
>
>
>               Configuration nodes that do not have an
>               'origin' annotation are treated as if they have the
>               'origin' annotation 'or:unknown'.
>
>
>
> > The draft shows an example where both "intended" and "system" are given
> > > as filters.  This will work but will include all the "intended" leafs as
> > > well.
> > > What if a "learned" node is within a "system" node within an "intended"
> > > node?
> >
> > This works as well.  Note that the get-data description says:
> >
> >           Any ancestor nodes (including list keys) of nodes selected by
> >           the filters are included in the response.
> >
> >
> >
>
> The issue is how the /iaddresses and /addresses/address nodes match the
> origin "learned".

They don't, but they are included b/c of the quoted text above (i.e.:
      Any ancestor nodes (including list keys) of nodes selected by
      the filters are included in the response.)


No.

If the filter was for /addresses/address/zipcode then maybe that text applies.
It is still unclear that the XPath is fully processed and then the origin-filter is processed.
The RFC just says they are ANDed together.



> The leafs in list "address" are a mixture of "intended" and "learned"
> origin.
> The text clearly says that a node has a single origin property, coupled to
> the annotation.
>
> Issue 1: mixed origin descendant nodes
> So how does a search on /addresses/address match origin-filter=learned?
> I cannot find any text that says what the origin of a list or P-container
> is if it
> contains nodes of mixed origin.

See above.

No text above explains how the list origin is tagged if it has multiple types of child nodes.



> Issue 2: NP-containers
>
> Also from RFC 8342:
>
>    The origin applies to all configuration nodes except non-presence
>    containers.
>
>
> What if the top-level node is an NP-container in this case.
> I thought the top-level node MUST have an origin attribute.
>
> The text is not clear how NP-containers are handled.
> Do they have an origin attribute? If not then RFC 8526 says they have
> origin "unknown".
> Is the intent that NP-containers always pass the origin-filter tests (test
> skipped)?

No, since they don't have an origin value they will not be selected by
the filter.  But an NP-container will be included in the reply if it
is the ancestor of a node that is selected by the filter.

The RFC text does not really say that.
Since it is very difficult to know if a data node 5 layers deep is going to match,
implementing these filters according to this vague interpretation is unlikely.


/martin

Andy



>
>
>
> /martin
> >
> >
> Andy
>
>
> >
> > > Seems like the client needs to know a lot about the server implementation
> > > details
> > > in order to use the origin filters.
> > >
> > >
> > > Andy
> >