Re: [netmod] [Netconf] Retrieving Information Pointed by leafref

Robert Wilton <rwilton@cisco.com> Wed, 11 October 2017 10:49 UTC

Return-Path: <rwilton@cisco.com>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 328F8133341; Wed, 11 Oct 2017 03:49:15 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.5
X-Spam-Level:
X-Spam-Status: No, score=-14.5 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hbHlX7lkPr4d; Wed, 11 Oct 2017 03:49:12 -0700 (PDT)
Received: from aer-iport-2.cisco.com (aer-iport-2.cisco.com [173.38.203.52]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1CD2513308D; Wed, 11 Oct 2017 03:49:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=47111; q=dns/txt; s=iport; t=1507718951; x=1508928551; h=subject:to:references:from:message-id:date:mime-version: in-reply-to; bh=bf4/PYXG9GB3XMxu3dorvn87flQI0wLHIuuBdZsk6nk=; b=OVuGAwBjP63b/7Sp4QzqDnxEuY//+DN07gAeHpJ/g3OgFpjjGi1IhXpK P9SKFgHRSnbi0h+dqfiHOluQPpO0bUnctjUdqpqC5g0DLn3hIJApMRQyd l5kD17KMeO6twusO9yU6XKdmNK0PzqqCBYWWmRPTNyFyXUOAf8qJOMRcJ Y=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0B0AQD29d1Z/xbLJq1UChkBAQEBAQEBA?= =?us-ascii?q?QEBAQcBAQEBAYJvP4ERbieDeosTkCp5h0yNaoIPAwoYAQqBXoJrTwKFGRcBAgE?= =?us-ascii?q?BAQEBAQFrKIUdAQEBBAEBGAkERxcECxEDAQEBASABBgMCAiEGHwkIBgEMBgIBA?= =?us-ascii?q?RYBiXEDFRCpCoFtOieHHg2DYgEBAQEBAQEBAQEBAQEBAQEBAQEBARgFgy2DU4F?= =?us-ascii?q?qK4JJNYJegXsVNoJzgmEFihMIhyaHHYgiPIdeiBKEeYIUhXODWocujHuBCYdfg?= =?us-ascii?q?TkhAjSBDjIhCB0VSYUaHIFoPzYBAQGHECyCFQEBAQ?=
X-IronPort-AV: E=Sophos;i="5.43,361,1503360000"; d="scan'208,217";a="655386848"
Received: from aer-iport-nat.cisco.com (HELO aer-core-3.cisco.com) ([173.38.203.22]) by aer-iport-2.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Oct 2017 10:49:08 +0000
Received: from [10.63.23.63] (dhcp-ensft1-uk-vla370-10-63-23-63.cisco.com [10.63.23.63]) by aer-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id v9BAn7cA019888; Wed, 11 Oct 2017 10:49:07 GMT
To: "t.petch" <ietfc@btconnect.com>, Igor Bryskin <Igor.Bryskin@huawei.com>, netconf@ietf.org, netmod@ietf.org
References: <049501d34104$6aa46670$3fed3350$@gmail.com> <59DB9E54.8080805@tail-f.com> <0C72C38E7EBC34499E8A9E7DD007863909CDB234@SJCEML702-CHM.china.huawei.com> <20171009.191347.1897981146275128665.mbj@tail-f.com> <6f8eb6ff-8fc5-4be3-d582-b188bd2337a6@tail-f.com> <etPan.59dbd366.8bfdc1a.12f7@localhost> <a1af1cd1-9a61-9d1c-49d3-f1e031525f0a@tail-f.com> <0C72C38E7EBC34499E8A9E7DD007863909CDB9E2@SJCEML702-CHM.china.huawei.com> <42819484-f9b5-4f06-dd58-23d9bc8c1ecc@cisco.com> <etPan.59dccc8e.149bf998.1428@localhost> <02aa01d34275$192f1840$4001a8c0@gateway.2wire.net>
From: Robert Wilton <rwilton@cisco.com>
Message-ID: <d391c56c-cdeb-179d-8fbb-2f62d53d727a@cisco.com>
Date: Wed, 11 Oct 2017 11:49:07 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0
MIME-Version: 1.0
In-Reply-To: <02aa01d34275$192f1840$4001a8c0@gateway.2wire.net>
Content-Type: multipart/alternative; boundary="------------160EB91B6F901AE456FBD2D0"
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/mWhzZs8y2u5YITggEjCxSGMc9HQ>
Subject: Re: [netmod] [Netconf] Retrieving Information Pointed by leafref
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Oct 2017 10:49:15 -0000

I've also been thinking about this problem a bit more :-)

The XPath solution works, but the expression isn't particularly nice to 
write, and I suspect that implementations may struggle to implement it 
efficiently (if they support XPath filtering at all).

A nicer solution here might be to allow the XPath filters to be combined 
with a subtree filter along the lines of a more advanced type of 
"Attribute Match Expression" sec 6.2.2 of rfc6241.

E.g. rather than this XPath filter:

/te:te/te:tunnels/te:tunnel[te:name='foo'] |
/te:te/te:connections/te:connection[te:name=/te:te/te:tunnels/te:tunnel[te:name='foo']/te:connections/te:connection/te:name]

Here is example of what a subtree filter combined with an XPath filter 
could potentially look like (which of course isn't valid NETCONF/YANG 
today):

<filter type="subtree-xpath">
   <te:te xmlns:te="...">
     <te:tunnels te:name="foo"/>
     <te:connections>
       <filter type="xpath" select="te:name = ../te:tunnels/te:tunnel[te:name='foo']/te:connections/te:connection/te:name)"/>
     </te:connections>
   </te:te>
</filter>


Any opinions on whether this would be beneficial or is this just 
reinventing the wheel?

Thanks,
Rob


On 11/10/2017 10:41, t.petch wrote:
> Igor
>
> Thinking laterally, this is a problem that DNS encountered a few decades
> ago and solved, by allowing the server to include additional information
> not specifically requested that the server can see is going to be needed
> for the next step, so if the client asks only about a CNAME, then the
> server can provide the relevant IP address as well.
>
> I suspect that the current rules for Netconf do not allow the server to
> send anything not explicitly requested, which is a shame (IMO).
>
> The DNS approach works very well, in fact I do not think we would
> survive without it.
>
> Tom Petch
>
> ----- Original Message -----
> From: "Igor Bryskin" <Igor.Bryskin@huawei.com>
> Sent: Tuesday, October 10, 2017 2:35 PM
>
> Hi Rob,
>
> This helps a lot. What you wrote will work.
>
> The only difference is that if we would have the "joimt with" clause as
> we proposed, the server would be able to tailor the te-tunnel
> presentation to the client's requirements, e.g. substituting the
> connection pointers with connection bodies, while, according to your
> suggestion, the server will provide the te-tunnel body as is, and then
> augment it with the cobbection information, thus, leaving
> for the client to "shuffle " the received data. But I do agree, this
> would be a minor inconvinience for the client, the important thing is
> that the client will get all the data in one piece.
>
> Thanks a lot,
> Igor
>
> c
>
> From:Robert Wilton
> To:Igor Bryskin,
> Cc:Per Hedeland,netmod@ietf.org,netconf@ietf.org,
> Date:2017-10-10 06:41:04
> Subject:Re: [netmod] [Netconf] Retrieving Information Pointed by leafref
>
> Hi Igor,
>
> On 09/10/2017 23:11, Igor Bryskin wrote:
>> Hi Per,
>>
>> This is a good news, but, please, help us out.
>> Consider, we have a node - "te-tunnel" - which among other attributes
> has two key leafref lists:
>> 1) each member of the 1st list points to a "connection" supporting the
> te-tunnel. All connections supporting all te-tunnels are stored in a
> single list of connections.
>> 2) each member of the 2nd list points to a supporting "te-tunnel" -
> the te-tunnel in question depends on. All te=tunnels including the
> te-tunnel in question, are stored in a single list of te-tunnels.
>> The question: how the client can retrieve via a single request all
> attributes of the te-tunnel in question along with all parameters of all
> connections supporting the te-tunnel, but with just pointers to
> supporting te-tunnels (so that the interested client can use the
> pointers to retrieve full data via subsequent separate requests) ?
> I think that it might be something like this (for tunnel name foo):
>
>     /te/tunnels/tunnel[name='foo'] |
>
> /te/connections/connection[name=/te/tunnels/tunnel[name='foo']/connectio
> ns/connection/name]
>
> E.g. in English, this should equate to something like:
>
> Return all information for tunnel foo AND ALSO
> Return all information for all connections where the connection name
> matches one of the connections listed in tunnel foo.
>
>> Likewise, how the client can ask for full data of the te-tunnel and
> all supporting te-tunnels and just pointers for supporting connections?
> If my xpath above is right, then this would be something roughly like
> this:
>
>     /te/tunnels/tunnel[name='foo'] |
>
> /te/tunnels/tunnel[name=/te/tunnels/tunnel[name='foo']/supporting-tunnel
> s/supporting-tunnel/name]
>
>
> I'm an XPath novice, so the expressions might be wrong.
>
> https://www.freeformatter.com/xpath-tester.html might be useful. E.g. if
> you can construct a simple XML instance tree of your data, you could
> validate whether the XPath expression works.
>
> I hope that this is of some help,
> Rob
>
>
>> I really appreciate your help,
>>
>> Igor
>>
>>
>> -----Original Message-----
>> From: Per Hedeland [mailto:per@tail-f.com]
>> Sent: Monday, October 09, 2017 5:21 PM
>> To: Igor Bryskin
>> Cc: mbj@tail-f.com; xufeng.liu.ietf@gmail.com; netconf@ietf.org;
> netmod@ietf.org
>> Subject: Re: [Netconf] [netmod] Retrieving Information Pointed by
> leafref
>> Just to be clear: what we're suggesting is that you can use the
>> already-existing standard NETCONF XPath capability to achieve the
> desired
>> result - see https://tools.ietf.org/html/rfc6241#section-8.9
>>
>> --Per
>>
>> On 2017-10-09 21:52, Igor Bryskin wrote:
>>> I agree. For example, a leafref may point not to a singls entity, but
> to a list of entities, and the client might want to expand all of them
> into the joint get response.
>>> Igor
>>>
>>> *From:*Per Hedeland
>>> *To:*Martin Bjorklund,
>>> *Cc:*Igor
> Bryskin,xufeng.liu.ietf@gmail.com,netconf@ietf.org,netmod@ietf.org,
>>> *Date:*2017-10-09 15:12:22
>>> *Subject:*Re: [Netconf] [netmod] Retrieving Information Pointed by
> leafref
>>> On 2017-10-09 19:13, Martin Bjorklund wrote:
>>>> Igor Bryskin <Igor.Bryskin@huawei.com> wrote:
>>>>> Hi Per,
>>>>>
>>>>> Basically, what we need is a way for a client to request something
>>>>> like this:
>>>>>
>>>>> get <XPath> joint with <XPath1, XPath2, ..., XPathn>
>>>> ... which is what Per's expression does!  Note that "|" in XPath
> means
>>>> "union".
>>>>
>>>> But as Per explained, it only works in some cases (when the leafref
>>>> acts a "single pointer").
>>> Well, that particular expression works only in that case - but since
> it
>>> is effectively the client that (perhaps based on the data model)
> decides
>>> what the leafref-leafs "mean" (in this case the single key of a
> single
>>> list), other cases can be handled the same way. E.g. multiple
>>> leafref-to-key leafs that together give the keys of a multi-key list
>>> just amounts to a slightly hairier XPath filter...
>>>
>>> --Per
>>>
>>>>> with a server interpreting the request as follows:
>>>>> if a node pointed by XPath contains a pointer (e.g. key leafref)
>>>>> matching one of the XPath from the "joint with" list, then the
> server
>>>>> must provide the entire body of the node pointed by the pointer,
>>>>> otherwise, just the pointer (as it happens today, that is, when no
>>>>> "joint with" list specified).
>>>>>
>>>>> We think that this would allow for the client to optimize the
> number
>>>>> of request-response iterations depending on application/use case.
>>>>>
>>>>> Regards,
>>>>> Igor
>>>>
>>>> /martin
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Per Hedeland [mailto:per@tail-f.com]
>>>>> Sent: Monday, October 09, 2017 12:06 PM
>>>>> To: Xufeng Liu
>>>>> Cc: Igor Bryskin; netconf@ietf.org; netmod@ietf.org
>>>>> Subject: Re: [Netconf] [netmod] Retrieving Information Pointed by
>>>>> leafref
>>>>>
>>>>> I understand your use case, but a leaf of type leafref does not in
>>>>> general identify a single node in the data tree - the leafref path
>>>>> could
>>>>> be for a non-key leaf, and/or the path could traverse list nodes,
>>>>> and/or
>>>>> the "target" list could have multiple keys and thus multiple
>>>>> leafref-leafs be required to identify a specific list entry.
>>>>>
>>>>> Thus it seems to me that your use case is not a reasonable basis
> for a
>>>>> new protocol operation. My XPath foo isn't very good either, but I
> do
>>>>> believe Robert's suggestion of using an XPath filter could be a way
>>>>> forward. I *think* the filter expression would be something along
> the
>>>>> lines of
>>>>>
>>>>>     /te/tunnels/tunnel[name='foo'] |
>>>>>
> /te/explicit-paths/explicit-path[name=/te/tunnels/tunnel[name='foo']/pat
> hs/path/explicit-path]
>>>>> --Per
>>>>>
>>>>> On 2017-10-09 15:42, Xufeng Liu wrote:
>>>>>> Hi Per,
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* Igor Bryskin [mailto:Igor.Bryskin@huawei.com]
>>>>>> *Sent:* Sunday, October 8, 2017 7:04 PM
>>>>>> *To:* Igor Bryskin <Igor.Bryskin@huawei.com>om>; per@tail-f.com;
>>>>>> *xufeng.liu.ietf@gmail.com
>>>>>> *Cc:* netconf@ietf.org; netmod@ietf.org
>>>>>> *Subject:* Re: [Netconf] [netmod] Retrieving Information Pointed
> by
>>>>>> *leafref
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi Joel,
>>>>>>
>>>>>> Thanks, I think I didnt explain our problem correctly.
>>>>>>
>>>>>> In our case we have a leafref pointing to a te tunnel name, which
>>>>>> happens to be a key to lookup the (axilary) tunnel.  We need a way
> to
>>>>>> include the entire tunnel body (not just a name) into the get
>>>>>> response. This is to optimize the number of iterations between the
>>>>>> client and the server. As Xufeng put it something similar to SQL
> join,
>>>>>> Igor
>>>>>>
>>>>>> *From:*Igor Bryskin
>>>>>>
>>>>>> *To:*per@tail-f.com,xufeng.liu.ietf@gmail.com,
>>>>>>
>>>>>> *Cc:*netconf@ietf.org,netmod@ietf.org,
>>>>>>
>>>>>> *Date:*2017-10-08 17:36:47
>>>>>>
>>>>>> *Subject:*Re: [Netconf] [netmod] Retrieving Information Pointed by
>>>>>> *leafref
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi Per,
>>>>>>
>>>>>> In a nutshell we would lika for a netconf client to have a way to
>>>>>> instruct the server on whether in response to the get request the
>>>>>> server needs to provide the entire body of a datastore node
> pointed
>>>>>> to by a leafref or just a pointer to said node, so that the node's
>>>>>> body could be retrieved by a subsequent separate request. This is
>>>>>> requested by implementors who want to optimise rhe number of
>>>>>> interactions between a client and its server.
>>>>>>
>>>>>> Cheers,
>>>>>> Igor
>>>>>>
>>>>>> *From:*Per Hedeland
>>>>>>
>>>>>> *To:*Xufeng Liu,
>>>>>>
>>>>>> *Cc:*netconf@ietf.org,'NetMod WG',
>>>>>>
>>>>>> *Date:*2017-10-08 14:01:27
>>>>>>
>>>>>> *Subject:*Re: [Netconf] [netmod] Retrieving Information Pointed by
>>>>>> *leafref
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2017-10-06 23:11, Xufeng Liu wrote:
>>>>>>> During the design team discussion for TE and MPLS YANG modeling,
> we
>>>>>>> have received a request from implementers: How to minimize the
> number
>>>>>>> of NETCONF/RESTCONF RPCs to improve operation efficiency?
>>>>>>> Especially for the case when the operator or client software
> needs to
>>>>>>> retrieve the object contents pointed by a leafref.
>>>>>>>
>>>>>>> For example, given the following simplified TE tunnel model,
>>>>>>>
>>>>>>> +--rw te
>>>>>>>
>>>>>>>         +--rw explicit-paths
>>>>>>>
>>>>>>>         |  +--rw explicit-path* [name]
>>>>>>>
>>>>>>>         |     +--rw name                      string
>>>>>>>
>>>>>>>         |        +--rw explicit-route-object* [index]
>>>>>>>
>>>>>>>         |           +--rw index                   uint32
>>>>>>>
>>>>>>>         |           +--rw explicit-route-usage?   identityref
>>>>>>>
>>>>>>>         +--rw tunnels
>>>>>>>
>>>>>>>         |  +--rw tunnel* [name]
>>>>>>>
>>>>>>>         |  |  +--rw name                   string
>>>>>>>
>>>>>>>         |  |  +--rw paths
>>>>>>>
>>>>>>>         |  |  |  +--rw path* [name]
>>>>>>>
>>>>>>> |  |  |     +--rw explicit-path?  ->
>>>>>>> |  |  |     ../../../../../explicit-paths/explicit-path/name
>>>>>>>
>>>>>>> when the client tries to retrieve a tunnels information based on
> the
>>>>>>> tunnel name, the get operation returns a list of leafrefs
> pointing
>>>>>>> to the paths of the tunnel.
>>>>>> Sorry, I'm afraid I don't follow. Can you explain exactly what
> your
>>>>>> "get" request is (protocol and payload), and where the "list of
>>>>>> leafref's" (whatever that may be) occurs in the reply?
>>>>>>
>>>>>> */[Xufeng] The get operation is the NETCONF/RESTCON <get>
> protocol
>>>>>> *operation, or the <get-data> operation described in
>>>>>> *https://tools.ietf.org/html/draft-dsdt-nmda-netconf-01 and the
> GET
>>>>>> *operations
>>>>>> on {+restconf}/ds/<datastore> described in
>>>>>> https://tools.ietf.org/html/draft-ietf-netconf-nmda-restconf-00./*
>>>>>>
>>>>>> */ /*
>>>>>>
>>>>>> */We have a list of leafref values because in this example model,
> each
>>>>>> *tunnel contains a list of paths, each of them contains a leafref.
> The
>>>>>> *get returns a value for each instance of such a leafref,
>>>>>> which (as a string value) will be used as a constraint (foreign
> key)
>>>>>> to retrieve the instance of an explicit-path in the model above./*
>>>>>>
>>>>>>
>>>>>>
>>>>>> JFYI, in case there is some fundamental misunderstanding here: a
> leaf
>>>>>> of
>>>>>> type leafref has a single value - *one* of those that satisfy the
>>>>>> leafref
>>>>>> constraint, in case there are multiple "candidates".
>>>>>>
>>>>>> --Per
>>>>>>
>>>>>>> The client needs to issue at
>>>>>>> least one more get operation to retrieve the path information
> about
>>>>>>> the given tunnel. The request is to combine these two operations
> into
>>>>>>> one.
>>>>>>>
>>>>>>> In the RDBMS SQL world, join can be used when SQL select is
>>>>>>> performed, but NETCONF/YANG currently does not have this
> capability.
>>>>>>> Wed like to ask whether such a request is considered reasonable.
>>>>>>>
>>>>>>> If the request is reasonable, the next question is how to
>>>>>>> proceed. This seems to be a protocol issue rather than YANG
> modeling
>>>>>>> issue. Is it acceptable to add a new operation to achieve such a
>>>>>>> <get-data> operation with expanded leafrefs?
>>>>>>>
>>>>>>> Comments and suggestions are appreciated.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> - Xufeng
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> netmod mailing list
>>>>>>> netmod@ietf.org <mailto:netmod@ietf.org>
>>>>>>> https://www.ietf.org/mailman/listinfo/netmod
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Netconf mailing list
>>>>>> Netconf@ietf.org <mailto:Netconf@ietf.org>
>>>>>> https://www.ietf.org/mailman/listinfo/netconf
>>>>>>
>>>>> _______________________________________________
>>>>> Netconf mailing list
>>>>> Netconf@ietf.org
>>>>> https://www.ietf.org/mailman/listinfo/netconf
>>>>>
>> _______________________________________________
>> netmod mailing list
>> netmod@ietf.org
>> https://www.ietf.org/mailman/listinfo/netmod
>> .
>>
>
>
>
> ------------------------------------------------------------------------
> --------
>
>
>> _______________________________________________
>> netmod mailing list
>> netmod@ietf.org
>> https://www.ietf.org/mailman/listinfo/netmod
>>
> .
>