Re: [netconf] get-data origin filters

[Martin Bjorklund <[e-mail]>]

Andy Bierman <[e-mail]> wrote:
> On Mon, Oct 7, 2019 at 7:36 AM Rob Wilton (rwilton) <[e-mail]>
> wrote:
> 
> > Hi Andy,
> >
> >
> >
> > Don’t all the filters effectively work this way?
> >
> 
> 
> I do not see the text that explains origin-filter and negated-origin-filter
> working the way Martin
> describes it.  These filters do not say anywhere to select a node because
> it has descendants
> that match the origin filters.  It says very clearly that the filter test
> is on the specified node.

Let's simplify the example somewhat.  Suppose the client sends:

  <get-data>
    <datastore>ds:operational</datastore>
    <origin-filter>or:learned</origin-filter>
    <with-origin/>
  </get-data>

Are you saying that b/c the top-level node doesn't have or:learned,
this filter will return nothing?

It seems you are assuming that the server first matches the top-level
nodes, and if they match, then match descendants and so one.  The text
doesn't say that, however.  It says:

                A configuration node matches the filter if its
                'origin' annotation is derived from or equal to any of
                the given filter values.

and:

          Any ancestor nodes (including list keys) of nodes selected by
          the filters are included in the response.

So I think it is pretty clear that the origin filter matches the
zipcode nodes in the example, and then the ancestors are included in
the reply.


> It also says the origin is derived from the origin annotation for that node.
> Since only 1 instance of the origin annotation is allowed per node, there
> is no way to tag
> a node with multiple origins.

Agreed.


/martin


> If implementation is too complex then people will just leave it out (w/ a
> deviation).
> It is unlikely that the instrumentation knows at any given instant all the
> origin values
> of all the descendant dynamic data at the instant the <get-data> request is
> processed.
> 
> 
> 
> 
> >
> > They select a subset of the nodes to include in the response, and must
> > also include all ancestor nodes and required list keys to the selected
> > nodes, regardless of whether those ancestor/key nodes were also selected by
> > the query.
> >
> >
> >
> 
> Yes. Understood.
> Still does not explain how a filter for the list node selects descendant
> nodes that match the origin filters.
> 
> 
> 
> > E.g. a “config false” filter will still return “config true” nodes if they
> > are ancestors or list keys to a descendant config false node.  The same
> > logic applies for xpath and origin filters as well.
> >
> >
> >
> 
> No they won't.
> Where is that text?
> 
>     get-data config=filter=false
> 
> This starts from top-level YANG nodes.
> If the top-level YANG node is not config=false then the server will not
> keep looking for descendants that match.
> 
> 
> 
> > Thanks,
> > Rob
> >
> >
> 
> Andy
> 
> 
> 
> >
> >
> >
> >
> > *From:* netconf <[e-mail]> *On Behalf Of *Andy Bierman
> > *Sent:* 07 October 2019 15:12
> > *To:* Martin Bjorklund <[e-mail]>
> > *Cc:* Netconf <[e-mail]>
> > *Subject:* Re: [netconf] get-data origin filters
> >
> >
> >
> >
> >
> >
> >
> > On Mon, Oct 7, 2019 at 12:43 AM Martin Bjorklund <[e-mail]> wrote:
> >
> > Andy Bierman <[e-mail]> wrote:
> > > On Sun, Oct 6, 2019 at 8:32 AM Martin Bjorklund <[e-mail]> wrote:
> > >
> > > > Hi,
> > > >
> > > > Andy Bierman <[e-mail]> wrote:
> > > > > Hi,
> > > > >
> > > > > I am trying to figure out how to use the origin-filter and
> > > > > negated-origin-filter
> > > > > in the <get-data> operation in RFC 8526.
> > > > >
> > > > >
> > > > >           leaf-list origin-filter {
> > > > >              type or:origin-ref;
> > > > >              description
> > > > >                "Filter based on the 'origin' annotation.  A
> > > > >                 configuration node matches the filter if its 'origin'
> > > > >                 annotation is derived from or equal to any of the
> > given
> > > > >                 filter values.";
> > > > >            }
> > > > >
> > > > >
> > > > > These filters seem kind of worthless if implemented according to the
> > > > text.
> > > > > Consider a simple example where there is 1 learned leaf within a
> > list:
> > > > >
> > > > > module: address
> > > > >   +--rw addresses
> > > > >      +--rw address* [last-name first-name]
> > > > >         +--rw last-name     string
> > > > >         +--rw first-name    string
> > > > >         +--rw street?       string
> > > > >         +--rw city?         string
> > > > >         +--rw zipcode?      string
> > > > >         +--rw phone* [phone-type]
> > > > >            +--rw phone-type      enumeration
> > > > >            +--rw phone-number    string
> > > > >
> > > > > Let's say the "zipcode" field is learned in <operational>
> > > > > (e.g. ZIP code lookup expands missing or 5 digit zipcode to full 9
> > digit
> > > > > zipcode).
> > > > > So /addresses and /addresses/address have origin "intended".
> > > > > Only the /addresses/address/zipcode leaf has origin "learned".
> > > > >
> > > > > So how does origin-filter=learned find all the learned leafs?
> > > >
> > > > Perhaps I don't understand your question; IMO you give the answer to
> > > > this question below:
> > > >
> > > > > What filters are required to return only the learned entries +
> > ancestors
> > > > +
> > > > > ancestor-or-self keys?  Seems like this filter mechanism has to be
> > used
> > > > > to retrieve the exact leaf that might be learned, and the client
> > > > > needs to know in advance all the possible nodes that might be
> > learned.
> > > > >
> > > > > Want to be able to retrieve an ancestor that is intended and still
> > find
> > > > the
> > > > > learned entries
> > > > >
> > > > >    get-data xpath-filter=/addresses/address origin-filtter=learned
> > > >
> > > > ... here.  So this request will return:
> > > >
> > > >    <addresses or:origin="or:intended">
> > > >      <address>
> > > >        <last-name>...</last-name>
> > > >        <first-name>...</first-name>
> > > >        <zipcode or:origin="or:learned">...</zipcode>
> > > >      </address>
> > > >      ...
> > > >    </addresses>
> > > >
> > > >
> > > I do not interpret the text the same way as you.
> >
> > Does this mean that you think that the reply is different from what I
> > show above?  If so, what would it be, and why?
> >
> >
> >
> >
> >
> >
> >
> > Explain how the list address node has origin "learned".
> >
> >
> >
> > The filter is for /addresses/address and only origin=learned.
> >
> > How does the list node have origin=learned?
> >
> > It can only have 1 value.
> >
> > It has child nodes with both intended and learned as origin.
> >
> > I do no understand how the origin=learned matched this node.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > >
> > >                      The content returned
> > >
> > >           by get-data must satisfy all filters, i.e., the filter
> > >           criteria are logically ANDed.
> > >
> > >
> > >           leaf-list origin-filter {
> > >              type or:origin-ref;
> > >              description
> > >                "Filter based on the 'origin' annotation.  A
> > >                 configuration node matches the filter if its 'origin'
> > >                 annotation is derived from or equal to any of the given
> > >                 filter values.";
> > >            }
> > >
> > >
> > >               Configuration nodes that do not have an
> > >               'origin' annotation are treated as if they have the
> > >               'origin' annotation 'or:unknown'.
> > >
> > >
> > >
> > > > The draft shows an example where both "intended" and "system" are given
> > > > > as filters.  This will work but will include all the "intended"
> > leafs as
> > > > > well.
> > > > > What if a "learned" node is within a "system" node within an
> > "intended"
> > > > > node?
> > > >
> > > > This works as well.  Note that the get-data description says:
> > > >
> > > >           Any ancestor nodes (including list keys) of nodes selected by
> > > >           the filters are included in the response.
> > > >
> > > >
> > > >
> > >
> > > The issue is how the /iaddresses and /addresses/address nodes match the
> > > origin "learned".
> >
> > They don't, but they are included b/c of the quoted text above (i.e.:
> >       Any ancestor nodes (including list keys) of nodes selected by
> >       the filters are included in the response.)
> >
> >
> >
> >
> >
> > No.
> >
> >
> >
> > If the filter was for /addresses/address/zipcode then maybe that text
> > applies.
> >
> > It is still unclear that the XPath is fully processed and then the
> > origin-filter is processed.
> >
> > The RFC just says they are ANDed together.
> >
> >
> >
> >
> >
> >
> > > The leafs in list "address" are a mixture of "intended" and "learned"
> > > origin.
> > > The text clearly says that a node has a single origin property, coupled
> > to
> > > the annotation.
> > >
> > > Issue 1: mixed origin descendant nodes
> > > So how does a search on /addresses/address match origin-filter=learned?
> > > I cannot find any text that says what the origin of a list or P-container
> > > is if it
> > > contains nodes of mixed origin.
> >
> > See above.
> >
> >
> >
> > No text above explains how the list origin is tagged if it has multiple
> > types of child nodes.
> >
> >
> >
> >
> >
> >
> > > Issue 2: NP-containers
> > >
> > > Also from RFC 8342:
> > >
> > >    The origin applies to all configuration nodes except non-presence
> > >    containers.
> > >
> > >
> > > What if the top-level node is an NP-container in this case.
> > > I thought the top-level node MUST have an origin attribute.
> > >
> > > The text is not clear how NP-containers are handled.
> > > Do they have an origin attribute? If not then RFC 8526 says they have
> > > origin "unknown".
> > > Is the intent that NP-containers always pass the origin-filter tests
> > (test
> > > skipped)?
> >
> > No, since they don't have an origin value they will not be selected by
> > the filter.  But an NP-container will be included in the reply if it
> > is the ancestor of a node that is selected by the filter.
> >
> >
> >
> > The RFC text does not really say that.
> >
> > Since it is very difficult to know if a data node 5 layers deep is going
> > to match,
> >
> > implementing these filters according to this vague interpretation is
> > unlikely.
> >
> >
> >
> >
> > /martin
> >
> >
> >
> > Andy
> >
> >
> >
> >
> >
> > >
> > >
> > >
> > > /martin
> > > >
> > > >
> > > Andy
> > >
> > >
> > > >
> > > > > Seems like the client needs to know a lot about the server
> > implementation
> > > > > details
> > > > > in order to use the origin filters.
> > > > >
> > > > >
> > > > > Andy
> > > >
> >
> >