Re: [netmod] [Netconf] Retrieving Information Pointed by leafref

[Robert Wilton <[e-mail]>]

Hi Xufeng,

Just a note for clarity: My combined proposal using subtree filtering 
and XPath isn't a valid NETCONF filter today, I was suggesting it as a 
potential future enhancement to NETCONF.  My reading of the NETCONF 
draft is that an XPath filter can only be specified at the top level.

Thanks,
Rob


On 11/10/2017 19:43, Xufeng Liu wrote:
>
> Hi Rob,
>
> Thanks for the proposal. I think that the subtree does help, to a 
> certain extend. The approach is worth mentioning to the implementers, 
> though the remaining XPath is still complicated and could be worse for 
> a more complex model. The good thing about the approach is that this 
> is what we already have today. I agree with Tom on that NETCONF should 
> have a better solution, but that may require protocol changes and need 
> to cover more generic cases as Per described.
>
> Thanks,
>
> - Xufeng
>
> *From:*Netconf [mailto:[e-mail]] *On Behalf Of *Robert 
> Wilton
> *Sent:* Wednesday, October 11, 2017 6:49 AM
> *To:* t.petch <[e-mail]>; Igor Bryskin 
> <[e-mail]>; [e-mail]; [e-mail]
> *Subject:* Re: [Netconf] [netmod] Retrieving Information Pointed by 
> leafref
>
> I've also been thinking about this problem a bit more :-)
>
> The XPath solution works, but the expression isn't particularly nice 
> to write, and I suspect that implementations may struggle to implement 
> it efficiently (if they support XPath filtering at all).
>
> A nicer solution here might be to allow the XPath filters to be 
> combined with a subtree filter along the lines of a more advanced type 
> of "Attribute Match Expression" sec 6.2.2 of rfc6241.
>
> E.g. rather than this XPath filter:
>
> /te:te/te:tunnels/te:tunnel[te:name='foo'] |
> /te:te/te:connections/te:connection[te:name=/te:te/te:tunnels/te:tunnel[te:name='foo']/te:connections/te:connection/te:name]
>
> Here is example of what a subtree filter combined with an XPath filter 
> could potentially look like (which of course isn't valid NETCONF/YANG 
> today):
>
> <filter type="subtree-xpath">
>    <te:te xmlns:te="...">
>      <te:tunnels te:name="foo"/>
>      <te:connections>
>        <filter type="xpath" select="te:name = ../te:tunnels/te:tunnel[te:name='foo']/te:connections/te:connection/te:name)"/>
>      </te:connections>
>    </te:te>
> </filter>
>
>
> Any opinions on whether this would be beneficial or is this just 
> reinventing the wheel?
>
> Thanks,
> Rob
>
> On 11/10/2017 10:41, t.petch wrote:
>
>     Igor
>
>     Thinking laterally, this is a problem that DNS encountered a few decades
>
>     ago and solved, by allowing the server to include additional information
>
>     not specifically requested that the server can see is going to be needed
>
>     for the next step, so if the client asks only about a CNAME, then the
>
>     server can provide the relevant IP address as well.
>
>     I suspect that the current rules for Netconf do not allow the server to
>
>     send anything not explicitly requested, which is a shame (IMO).
>
>     The DNS approach works very well, in fact I do not think we would
>
>     survive without it.
>
>     Tom Petch
>
>     ----- Original Message -----
>
>     From: "Igor Bryskin"<[e-mail]> <mailto:[e-mail]>
>
>     Sent: Tuesday, October 10, 2017 2:35 PM
>
>     Hi Rob,
>
>     This helps a lot. What you wrote will work.
>
>     The only difference is that if we would have the "joimt with" clause as
>
>     we proposed, the server would be able to tailor the te-tunnel
>
>     presentation to the client's requirements, e.g. substituting the
>
>     connection pointers with connection bodies, while, according to your
>
>     suggestion, the server will provide the te-tunnel body as is, and then
>
>     augment it with the cobbection information, thus, leaving
>
>     for the client to "shuffle " the received data. But I do agree, this
>
>     would be a minor inconvinience for the client, the important thing is
>
>     that the client will get all the data in one piece.
>
>     Thanks a lot,
>
>     Igor
>
>     c
>
>     From:Robert Wilton
>
>     To:Igor Bryskin,
>
>     Cc:Per Hedeland,[e-mail],[e-mail]
>     <mailto:[e-mail],[e-mail]>,
>
>     Date:2017-10-10 06:41:04
>
>     Subject:Re: [netmod] [Netconf] Retrieving Information Pointed by leafref
>
>     Hi Igor,
>
>     On 09/10/2017 23:11, Igor Bryskin wrote:
>
>         Hi Per,
>
>         This is a good news, but, please, help us out.
>
>         Consider, we have a node - "te-tunnel" - which among other attributes
>
>     has two key leafref lists:
>
>         1) each member of the 1st list points to a "connection" supporting the
>
>     te-tunnel. All connections supporting all te-tunnels are stored in a
>
>     single list of connections.
>
>         2) each member of the 2nd list points to a supporting "te-tunnel" -
>
>     the te-tunnel in question depends on. All te=tunnels including the
>
>     te-tunnel in question, are stored in a single list of te-tunnels.
>
>         The question: how the client can retrieve via a single request all
>
>     attributes of the te-tunnel in question along with all parameters of all
>
>     connections supporting the te-tunnel, but with just pointers to
>
>     supporting te-tunnels (so that the interested client can use the
>
>     pointers to retrieve full data via subsequent separate requests) ?
>
>     I think that it might be something like this (for tunnel name foo):
>
>         /te/tunnels/tunnel[name='foo'] |
>
>     /te/connections/connection[name=/te/tunnels/tunnel[name='foo']/connectio
>
>     ns/connection/name]
>
>     E.g. in English, this should equate to something like:
>
>     Return all information for tunnel foo AND ALSO
>
>     Return all information for all connections where the connection name
>
>     matches one of the connections listed in tunnel foo.
>
>         Likewise, how the client can ask for full data of the te-tunnel and
>
>     all supporting te-tunnels and just pointers for supporting connections?
>
>     If my xpath above is right, then this would be something roughly like
>
>     this:
>
>         /te/tunnels/tunnel[name='foo'] |
>
>     /te/tunnels/tunnel[name=/te/tunnels/tunnel[name='foo']/supporting-tunnel
>
>     s/supporting-tunnel/name]
>
>     I'm an XPath novice, so the expressions might be wrong.
>
>     https://www.freeformatter.com/xpath-tester.html  might be useful. E.g. if
>
>     you can construct a simple XML instance tree of your data, you could
>
>     validate whether the XPath expression works.
>
>     I hope that this is of some help,
>
>     Rob
>
>         I really appreciate your help,
>
>         Igor
>
>         -----Original Message-----
>
>         From: Per Hedeland [mailto:[e-mail]]
>
>         Sent: Monday, October 09, 2017 5:21 PM
>
>         To: Igor Bryskin
>
>         Cc:[e-mail] <mailto:[e-mail]>;[e-mail] <mailto:[e-mail]>;[e-mail] <mailto:[e-mail]>;
>
>     [e-mail] <mailto:[e-mail]>
>
>         Subject: Re: [Netconf] [netmod] Retrieving Information Pointed by
>
>     leafref
>
>         Just to be clear: what we're suggesting is that you can use the
>
>         already-existing standard NETCONF XPath capability to achieve the
>
>     desired
>
>         result - seehttps://tools.ietf.org/html/rfc6241#section-8.9
>
>         --Per
>
>         On 2017-10-09 21:52, Igor Bryskin wrote:
>
>             I agree. For example, a leafref may point not to a singls entity, but
>
>     to a list of entities, and the client might want to expand all of them
>
>     into the joint get response.
>
>             Igor
>
>             *From:*Per Hedeland
>
>             *To:*Martin Bjorklund,
>
>             *Cc:*Igor
>
>     Bryskin,[e-mail],[e-mail],[e-mail]
>     <mailto:[e-mail],[e-mail],[e-mail]>,
>
>             *Date:*2017-10-09 15:12:22
>
>             *Subject:*Re: [Netconf] [netmod] Retrieving Information Pointed by
>
>     leafref
>
>             On 2017-10-09 19:13, Martin Bjorklund wrote:
>
>                 Igor Bryskin<[e-mail]> <mailto:[e-mail]>  wrote:
>
>                     Hi Per,
>
>                     Basically, what we need is a way for a client to request something
>
>                     like this:
>
>                     get <XPath> joint with <XPath1, XPath2, ..., XPathn>
>
>                 ... which is what Per's expression does!  Note that "|" in XPath
>
>     means
>
>                 "union".
>
>                 But as Per explained, it only works in some cases (when the leafref
>
>                 acts a "single pointer").
>
>             Well, that particular expression works only in that case - but since
>
>     it
>
>             is effectively the client that (perhaps based on the data model)
>
>     decides
>
>             what the leafref-leafs "mean" (in this case the single key of a
>
>     single
>
>             list), other cases can be handled the same way. E.g. multiple
>
>             leafref-to-key leafs that together give the keys of a multi-key list
>
>             just amounts to a slightly hairier XPath filter...
>
>             --Per
>
>                     with a server interpreting the request as follows:
>
>                     if a node pointed by XPath contains a pointer (e.g. key leafref)
>
>                     matching one of the XPath from the "joint with" list, then the
>
>     server
>
>                     must provide the entire body of the node pointed by the pointer,
>
>                     otherwise, just the pointer (as it happens today, that is, when no
>
>                     "joint with" list specified).
>
>                     We think that this would allow for the client to optimize the
>
>     number
>
>                     of request-response iterations depending on application/use case.
>
>                     Regards,
>
>                     Igor
>
>                 /martin
>
>                     -----Original Message-----
>
>                     From: Per Hedeland [mailto:[e-mail]]
>
>                     Sent: Monday, October 09, 2017 12:06 PM
>
>                     To: Xufeng Liu
>
>                     Cc: Igor Bryskin;[e-mail] <mailto:[e-mail]>;[e-mail] <mailto:[e-mail]>
>
>                     Subject: Re: [Netconf] [netmod] Retrieving Information Pointed by
>
>                     leafref
>
>                     I understand your use case, but a leaf of type leafref does not in
>
>                     general identify a single node in the data tree - the leafref path
>
>                     could
>
>                     be for a non-key leaf, and/or the path could traverse list nodes,
>
>                     and/or
>
>                     the "target" list could have multiple keys and thus multiple
>
>                     leafref-leafs be required to identify a specific list entry.
>
>                     Thus it seems to me that your use case is not a reasonable basis
>
>     for a
>
>                     new protocol operation. My XPath foo isn't very good either, but I
>
>     do
>
>                     believe Robert's suggestion of using an XPath filter could be a way
>
>                     forward. I *think* the filter expression would be something along
>
>     the
>
>                     lines of
>
>                         /te/tunnels/tunnel[name='foo'] |
>
>     /te/explicit-paths/explicit-path[name=/te/tunnels/tunnel[name='foo']/pat
>
>     hs/path/explicit-path]
>
>                     --Per
>
>                     On 2017-10-09 15:42, Xufeng Liu wrote:
>
>                         Hi Per,
>
>                         *From:* Igor Bryskin [mailto:[e-mail]]
>
>                         *Sent:* Sunday, October 8, 2017 7:04 PM
>
>                         *To:* Igor Bryskin<[e-mail]>
>                         <mailto:[e-mail]>;[e-mail] <mailto:[e-mail]>;
>
>                         *[e-mail]
>                         <mailto:*[e-mail]>
>
>                         *Cc:*[e-mail] <mailto:[e-mail]>;[e-mail] <mailto:[e-mail]>
>
>                         *Subject:* Re: [Netconf] [netmod] Retrieving Information Pointed
>
>     by
>
>                         *leafref
>
>                         Hi Joel,
>
>                         Thanks, I think I didnt explain our problem correctly.
>
>                         In our case we have a leafref pointing to a te tunnel name, which
>
>                         happens to be a key to lookup the (axilary) tunnel.  We need a way
>
>     to
>
>                         include the entire tunnel body (not just a name) into the get
>
>                         response. This is to optimize the number of iterations between the
>
>                         client and the server. As Xufeng put it something similar to SQL
>
>     join,
>
>                         Igor
>
>                         *From:*Igor Bryskin
>
>                         *To:*[e-mail],[e-mail]
>                         <mailto:[e-mail]>,
>
>                         *Cc:*[e-mail],[e-mail] <mailto:[e-mail]>,
>
>                         *Date:*2017-10-08 17:36:47
>
>                         *Subject:*Re: [Netconf] [netmod] Retrieving Information Pointed by
>
>                         *leafref
>
>                         Hi Per,
>
>                         In a nutshell we would lika for a netconf client to have a way to
>
>                         instruct the server on whether in response to the get request the
>
>                         server needs to provide the entire body of a datastore node
>
>     pointed
>
>                         to by a leafref or just a pointer to said node, so that the node's
>
>                         body could be retrieved by a subsequent separate request. This is
>
>                         requested by implementors who want to optimise rhe number of
>
>                         interactions between a client and its server.
>
>                         Cheers,
>
>                         Igor
>
>                         *From:*Per Hedeland
>
>                         *To:*Xufeng Liu,
>
>                         *Cc:*[e-mail],'NetMod WG',
>
>                         *Date:*2017-10-08 14:01:27
>
>                         *Subject:*Re: [Netconf] [netmod] Retrieving Information Pointed by
>
>                         *leafref
>
>                         On 2017-10-06 23:11, Xufeng Liu wrote:
>
>                             During the design team discussion for TE and MPLS YANG modeling,
>
>     we
>
>                             have received a request from implementers: How to minimize the
>
>     number
>
>                             of NETCONF/RESTCONF RPCs to improve operation efficiency?
>
>                             Especially for the case when the operator or client software
>
>     needs to
>
>                             retrieve the object contents pointed by a leafref.
>
>                             For example, given the following simplified TE tunnel model,
>
>                             +--rw te
>
>                                     +--rw explicit-paths
>
>                                     |  +--rw explicit-path* [name]
>
>                                     |     +--rw name                      string
>
>                                     |        +--rw explicit-route-object* [index]
>
>                                     |           +--rw index                   uint32
>
>                                     |           +--rw explicit-route-usage?   identityref
>
>                                     +--rw tunnels
>
>                                     |  +--rw tunnel* [name]
>
>                                     |  |  +--rw name                   string
>
>                                     |  |  +--rw paths
>
>                                     |  |  |  +--rw path* [name]
>
>                             |  |  |     +--rw explicit-path?  ->
>
>                             |  |  |     ../../../../../explicit-paths/explicit-path/name
>
>                             when the client tries to retrieve a tunnels information based on
>
>     the
>
>                             tunnel name, the get operation returns a list of leafrefs
>
>     pointing
>
>                             to the paths of the tunnel.
>
>                         Sorry, I'm afraid I don't follow. Can you explain exactly what
>
>     your
>
>                         "get" request is (protocol and payload), and where the "list of
>
>                         leafref's" (whatever that may be) occurs in the reply?
>
>                         */[Xufeng] The get operation is the NETCONF/RESTCON <get>
>
>     protocol
>
>                         *operation, or the <get-data> operation described in
>
>                         *https://tools.ietf.org/html/draft-dsdt-nmda-netconf-01  and the
>
>     GET
>
>                         *operations
>
>                         on {+restconf}/ds/<datastore> described in
>
>                         https://tools.ietf.org/html/draft-ietf-netconf-nmda-restconf-00./*
>
>                         */ /*
>
>                         */We have a list of leafref values because in this example model,
>
>     each
>
>                         *tunnel contains a list of paths, each of them contains a leafref.
>
>     The
>
>                         *get returns a value for each instance of such a leafref,
>
>                         which (as a string value) will be used as a constraint (foreign
>
>     key)
>
>                         to retrieve the instance of an explicit-path in the model above./*
>
>                         JFYI, in case there is some fundamental misunderstanding here: a
>
>     leaf
>
>                         of
>
>                         type leafref has a single value - *one* of those that satisfy the
>
>                         leafref
>
>                         constraint, in case there are multiple "candidates".
>
>                         --Per
>
>                             The client needs to issue at
>
>                             least one more get operation to retrieve the path information
>
>     about
>
>                             the given tunnel. The request is to combine these two operations
>
>     into
>
>                             one.
>
>                             In the RDBMS SQL world, join can be used when SQL select is
>
>                             performed, but NETCONF/YANG currently does not have this
>
>     capability.
>
>                             Wed like to ask whether such a request is considered reasonable.
>
>                             If the request is reasonable, the next question is how to
>
>                             proceed. This seems to be a protocol issue rather than YANG
>
>     modeling
>
>                             issue. Is it acceptable to add a new operation to achieve such a
>
>                             <get-data> operation with expanded leafrefs?
>
>                             Comments and suggestions are appreciated.
>
>                             Thanks,
>
>                             - Xufeng
>
>                             _______________________________________________
>
>                             netmod mailing list
>
>                             [e-mail] <mailto:[e-mail]>  <mailto:[e-mail]>
>                             <mailto:[e-mail]>
>
>                             https://www.ietf.org/mailman/listinfo/netmod
>
>                         _______________________________________________
>
>                         Netconf mailing list
>
>                         [e-mail] <mailto:[e-mail]>  <mailto:[e-mail]>
>                         <mailto:[e-mail]>
>
>                         https://www.ietf.org/mailman/listinfo/netconf
>
>                     _______________________________________________
>
>                     Netconf mailing list
>
>                     [e-mail] <mailto:[e-mail]>
>
>                     https://www.ietf.org/mailman/listinfo/netconf
>
>         _______________________________________________
>
>         netmod mailing list
>
>         [e-mail] <mailto:[e-mail]>
>
>         https://www.ietf.org/mailman/listinfo/netmod
>
>         .
>
>     ------------------------------------------------------------------------
>
>     --------
>
>         _______________________________________________
>
>         netmod mailing list
>
>         [e-mail] <mailto:[e-mail]>
>
>         https://www.ietf.org/mailman/listinfo/netmod
>
>     .
>