Re: [netconf] restconf collections

Kent Watsen <kent+ietf@watsen.net> Wed, 30 September 2020 17:55 UTC

Return-Path: <01000174e0279006-63781c55-866c-4ea0-add6-a6fa8a11453c-000000@amazonses.watsen.net>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E19143A0A62 for <netconf@ietfa.amsl.com>; Wed, 30 Sep 2020 10:55:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.896
X-Spam-Level:
X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=amazonses.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z15pH5sjGmCw for <netconf@ietfa.amsl.com>; Wed, 30 Sep 2020 10:55:24 -0700 (PDT)
Received: from a8-96.smtp-out.amazonses.com (a8-96.smtp-out.amazonses.com [54.240.8.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9C4AF3A0A5E for <netconf@ietf.org>; Wed, 30 Sep 2020 10:55:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1601488523; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:Feedback-ID; bh=Jua7FRCviz+79ZCFL2K7egePMO8wrukDsI9uK5VI33g=; b=QA0GXm/NSMjtKBEg4Nm+fB4TzD9LKBiP96EVP00OZmJBFj6SqfWyW5KJabjumc+M L01I98Gr8cZIXM9JEswG21MtwFwYLxIprzjZqMSS7B6w/qMTnowmwHJznY7pXHdFmr8 Zc/QeB99mTNkWyAwBkf8885t3edIP+v8rWPYC8A0=
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\))
From: Kent Watsen <kent+ietf@watsen.net>
In-Reply-To: <20200930.084312.246821928590534256.id@4668.se>
Date: Wed, 30 Sep 2020 17:55:23 +0000
Cc: "netconf@ietf.org" <netconf@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-ID: <01000174e0279006-63781c55-866c-4ea0-add6-a6fa8a11453c-000000@email.amazonses.com>
References: <01000174daf6ded3-0ba5564d-f1c7-4c65-90dc-d8a22f2f9395-000000@email.amazonses.com> <20200929.205710.165167541047857442.id@4668.se> <01000174dbde18ac-c9293572-775f-4c8e-b8c1-a8522bd636b5-000000@email.amazonses.com> <20200930.084312.246821928590534256.id@4668.se>
To: =?utf-8?Q?Martin_Bj=C3=B6rklund?= <mbj+ietf@4668.se>
X-Mailer: Apple Mail (2.3608.80.23.2.2)
X-SES-Outgoing: 2020.09.30-54.240.8.96
Feedback-ID: 1.us-east-1.DKmIRZFhhsBhtmFMNikgwZUWVrODEw9qVcPhqJEI2DA=:AmazonSES
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/aRp-TiR0mw7AOC_ZdjRwmERxjrM>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 30 Sep 2020 17:55:26 -0000

> If we agree to not do d) then we don't need any use cases for it ;-)

We don’t agree that sorting isn’t ever need, but I think that it’s fair to say that, amongst the various other parameters, it’s the least important, and so enabling it via a feature seems prudent.

BTW, sort (d) comes after filter (e), in our e-->a processing model.  Hopefully the (fast) filter whittles the result-set down to something manageable so the (slow) sort isn’t painfully too egregious.


> Anyway, if you want to sort on different fields you probably want to
> do this regardless of how the list is orderd on the server.

Possibly, and certainly it would be easy to implement, I’m just trying to understand when that might be the case.  For instance, using a firewall rulebase (an OBU-list) as an example, when could getting the rules in any other order be meaningful to a client?


>> Regarding “I don’t think they should be restricted to ‘indexable’
>> columns”, <snip/>
> 
> My comment was for both d) and e) (sort and filter).  As for filter,
> an example is to get interfaces and with oper-status == 'down'.

“oper-status” appears to be an enum, an hence indexable…or perhaps this is a forward-reference to your next comment about some opstate not being in the DB...


>> Yes, XPath is the “right answer”, but we need to ensure it’s
>> constrained enough to be mappable to common DB-backends.
> 
> So the client talks to the server, and the server talks to the "db"
> backend (note that in many implementations  operational state is
> typically not stored in a real database).  

Some opstate must be persisted, e.g., long-lived counters, logs, etc., but it’s a good point about other opstate not being persisted.   Perhaps “node-tags” can be used here, to differentiate which is which…and servers can indicate if/how they support the ephemeral opstate leafs in queries?


> So we have:
> 
>  client ->  server ->  backend
> 
> If we don't have any filter capabilities, the client has to get
> everything, and then filter.  If we support XPath and the backend
> doesn't support it, the server will have to filter.  This is still
> more efficient than filtering in the client.

Probably, especially when assuming the server has better resources and/or the client <--> server bandwidth is constrained.


> In ConfD/NSO we moved more and more filtering logic towards the
> backend, to speed up performance.

Ack.


> I think e) is way more important than c).  I suggest focus on a + b +
> c.

Noted, but I think you meant a + b + e.

Note sure how others feel about “direction: (c), but my primary use-case revolves around time-series data (e.g., logs), where the interest is commonly on the most-recent entries, so "reverse-->offset—>limit” works nicely.  

Perhaps an alternative would be to lift a concept from Python with negative indexes so, for instance, offset=-N and limit=-N gives the last N entries?


>> Sure, but I wonder if, e.g., a netmask filter, is supportable by
>> common DB-backends.  I’m hoping we have some DB-experts on the list!
> 
> See above.  It can be quite efficient even if the backend doesn't
> support it.

I don’t see that above, but I don’t doubt that it can be so, it’s just a whole lot of implementation complexity.  It seems that we should/must support servers doing it, we just need to find a way (node-tags?) to enable them to express that ability.


K.