Re: [netconf] restconf collections

Olof Hagsand <olof@hagsand.se> Wed, 30 September 2020 19:00 UTC

Return-Path: <olof@hagsand.se>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BB34C3A0AF6 for <netconf@ietfa.amsl.com>; Wed, 30 Sep 2020 12:00:05 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.111
X-Spam-Level:
X-Spam-Status: No, score=-2.111 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, NICE_REPLY_A=-0.213, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S3--Td-bTN_I for <netconf@ietfa.amsl.com>; Wed, 30 Sep 2020 12:00:01 -0700 (PDT)
Received: from smtp.outgoing.loopia.se (smtp.outgoing.loopia.se [93.188.3.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3E5C83A0AC5 for <netconf@ietf.org>; Wed, 30 Sep 2020 12:00:00 -0700 (PDT)
Received: from s807.loopia.se (localhost [127.0.0.1]) by s807.loopia.se (Postfix) with ESMTP id F34012F678EA for <netconf@ietf.org>; Wed, 30 Sep 2020 20:59:54 +0200 (CEST)
Received: from s500.loopia.se (unknown [172.22.191.6]) by s807.loopia.se (Postfix) with ESMTP id D4EC32E292AC; Wed, 30 Sep 2020 20:59:54 +0200 (CEST)
Received: from s474.loopia.se (unknown [172.22.191.6]) by s500.loopia.se (Postfix) with ESMTP id CF3A51E32B64; Wed, 30 Sep 2020 20:59:54 +0200 (CEST)
X-Virus-Scanned: amavisd-new at amavis.loopia.se
Received: from s500.loopia.se ([172.22.191.5]) by s474.loopia.se (s474.loopia.se [172.22.190.14]) (amavisd-new, port 10024) with LMTP id FZu_yGj35jb3; Wed, 30 Sep 2020 20:59:54 +0200 (CEST)
X-Loopia-Auth: user
X-Loopia-User: olof@hagsand.se
X-Loopia-Originating-IP: 78.82.254.64
Received: from [10.0.0.14] (c-40fe524e.712-1-64736c13.bbcust.telenor.se [78.82.254.64]) (Authenticated sender: olof@hagsand.se) by s500.loopia.se (Postfix) with ESMTPSA id 1DE3E1E32B2D; Wed, 30 Sep 2020 20:59:54 +0200 (CEST)
To: Kent Watsen <kent+ietf@watsen.net>
Cc: "netconf@ietf.org" <netconf@ietf.org>
References: <01000174daf6ded3-0ba5564d-f1c7-4c65-90dc-d8a22f2f9395-000000@email.amazonses.com> <20200929.205710.165167541047857442.id@4668.se> <01000174dbde18ac-c9293572-775f-4c8e-b8c1-a8522bd636b5-000000@email.amazonses.com> <20200930.084312.246821928590534256.id@4668.se> <01000174e0279006-63781c55-866c-4ea0-add6-a6fa8a11453c-000000@email.amazonses.com>
From: Olof Hagsand <olof@hagsand.se>
Message-ID: <54cd8891-6c04-3ded-22ca-43a68ac8397f@hagsand.se>
Date: Wed, 30 Sep 2020 20:59:53 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <01000174e0279006-63781c55-866c-4ea0-add6-a6fa8a11453c-000000@email.amazonses.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/bES3KpEu8fYU4WDg-vbf5iLqT8o>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 30 Sep 2020 19:00:07 -0000

On 2020-09-30 19:55, Kent Watsen wrote:
> 
>> If we agree to not do d) then we don't need any use cases for it ;-)
> 
> We don’t agree that sorting isn’t ever need, but I think that it’s fair to say that, amongst the various other parameters, it’s the least important, and so enabling it via a feature seems prudent.
> 
> BTW, sort (d) comes after filter (e), in our e-->a processing model.  Hopefully the (fast) filter whittles the result-set down to something manageable so the (slow) sort isn’t painfully too egregious.

I agree sorting is less important (than a,b,e).
Is it not also somewhat beyond the present yang concepts where there is
already ordered-by and key indices?
Supplying a differing sorting could in effect require a different set of
index(es) which could be quite costly in runtime for large data sets.
If you want to sort differently, should it not be present in the
underlying model, ie as declaring a secondary set of keys, for example,
as an extension of the original yang model (thus being out of scope here)?

> 
>> Anyway, if you want to sort on different fields you probably want to
>> do this regardless of how the list is orderd on the server.
> 
> Possibly, and certainly it would be easy to implement, I’m just trying to understand when that might be the case.  For instance, using a firewall rulebase (an OBU-list) as an example, when could getting the rules in any other order be meaningful to a client?
> 
> 
>>> Regarding “I don’t think they should be restricted to ‘indexable’
>>> columns”, <snip/>
>>
>> My comment was for both d) and e) (sort and filter).  As for filter,
>> an example is to get interfaces and with oper-status == 'down'.
> 
> “oper-status” appears to be an enum, an hence indexable…or perhaps this is a forward-reference to your next comment about some opstate not being in the DB...
> 
> 
>>> Yes, XPath is the “right answer”, but we need to ensure it’s
>>> constrained enough to be mappable to common DB-backends.

I would think a (strongly) constrained XPath would be a very nice solution.

>>
>> So the client talks to the server, and the server talks to the "db"
>> backend (note that in many implementations  operational state is
>> typically not stored in a real database).  
> 
> Some opstate must be persisted, e.g., long-lived counters, logs, etc., but it’s a good point about other opstate not being persisted.   Perhaps “node-tags” can be used here, to differentiate which is which…and servers can indicate if/how they support the ephemeral opstate leafs in queries?
> 
> 
>> So we have:
>>
>>  client ->  server ->  backend
>>
>> If we don't have any filter capabilities, the client has to get
>> everything, and then filter.  If we support XPath and the backend
>> doesn't support it, the server will have to filter.  This is still
>> more efficient than filtering in the client.
> 
> Probably, especially when assuming the server has better resources and/or the client <--> server bandwidth is constrained.
> 
> 
>> In ConfD/NSO we moved more and more filtering logic towards the
>> backend, to speed up performance.
> 
> Ack.
> 
> 
>> I think e) is way more important than c).  I suggest focus on a + b +
>> c.
> 
> Noted, but I think you meant a + b + e.
> 
> Note sure how others feel about “direction: (c), but my primary use-case revolves around time-series data (e.g., logs), where the interest is commonly on the most-recent entries, so "reverse-->offset—>limit” works nicely.  
> 
> Perhaps an alternative would be to lift a concept from Python with negative indexes so, for instance, offset=-N and limit=-N gives the last N entries?
> 

To me reverse direction seems a subset of ordering/sorting? If sorting
is less important, so would an alternate direction?
Unless there are some important use-cases, reverse order having the
advantage of the same sorting mechanism/indexes, just in reverse (not
introducing other keys, for example)

> 
>>> Sure, but I wonder if, e.g., a netmask filter, is supportable by
>>> common DB-backends.  I’m hoping we have some DB-experts on the list!
>>
>> See above.  It can be quite efficient even if the backend doesn't
>> support it.
> 
> I don’t see that above, but I don’t doubt that it can be so, it’s just a whole lot of implementation complexity.  It seems that we should/must support servers doing it, we just need to find a way (node-tags?) to enable them to express that ability.
> 
> 
> K.
> 
> 
> 
> _______________________________________________
> netconf mailing list
> netconf@ietf.org
> https://www.ietf.org/mailman/listinfo/netconf
>