Re: [netconf] restconf collections

Kent Watsen <kent+ietf@watsen.net> Thu, 01 October 2020 12:36 UTC

Return-Path: <01000174e42a0ab6-d29a9843-5a7a-4804-acba-276a859e4612-000000@amazonses.watsen.net>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4CFEE3A0FFA for <netconf@ietfa.amsl.com>; Thu, 1 Oct 2020 05:36:37 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level:
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=amazonses.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 024NBU2E0pNj for <netconf@ietfa.amsl.com>; Thu, 1 Oct 2020 05:36:36 -0700 (PDT)
Received: from a48-90.smtp-out.amazonses.com (a48-90.smtp-out.amazonses.com [54.240.48.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1DB3C3A0FB7 for <netconf@ietf.org>; Thu, 1 Oct 2020 05:36:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1601555794; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:Feedback-ID; bh=usURwUXrBfmyQXlVtVOm7QImjwGUI0keGj1LeqcQn7o=; b=EgV64SkUrSFXgf8DGewwTYZH+QX6yb8dnWxGC0zTQT0+lZXLG357Lr3GWdlPITHI MOpkV2nuOMxcw37ffspuuKS7NzNzSjRac+heSu5rdRAic2ynPA3FB8Eq3mGroQbAfHY TWDqEZPfjYPQftciJD6Bm3Tgr573qcG2HiNho3AY=
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\))
From: Kent Watsen <kent+ietf@watsen.net>
In-Reply-To: <B8F9A780D330094D99AF023C5877DABAADA3660D@dggeml531-mbs.china.huawei.com>
Date: Thu, 1 Oct 2020 12:36:34 +0000
Cc: =?utf-8?Q?Martin_Bj=C3=B6rklund?= <mbj+ietf@4668.se>, "netconf@ietf.org" <netconf@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-ID: <01000174e42a0ab6-d29a9843-5a7a-4804-acba-276a859e4612-000000@email.amazonses.com>
References: <B8F9A780D330094D99AF023C5877DABAADA3660D@dggeml531-mbs.china.huawei.com>
To: Qin Wu <bill.wu@huawei.com>
X-Mailer: Apple Mail (2.3608.80.23.2.2)
X-SES-Outgoing: 2020.10.01-54.240.48.90
Feedback-ID: 1.us-east-1.DKmIRZFhhsBhtmFMNikgwZUWVrODEw9qVcPhqJEI2DA=:AmazonSES
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/rFFX0xrr_HK68ytE5oFnA1_tYvw>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 01 Oct 2020 12:36:37 -0000

Hi Qin,

> Some opstate must be persisted, e.g., long-lived counters, logs, etc., but it’s a good point about other opstate not being persisted.   Perhaps “node-tags” can be used here, to differentiate which is which…and servers can indicate if/how they support the ephemeral opstate leafs in queries?
> 
> [Qin]:That's a good case for node tag, in earlier discussion, we discussed operation type, which distinguishs cumulative statistics value from current value. The case discussed here is very close to operation type proposal discussed earlier.

Yes.  Thank you for pointing that out.  I meant to make the same observation before.  Indeed, such node-tags could have dual-purpose: to guide a streaming-strategy and a querying-strategy for certain nodes.


> Note sure how others feel about “direction: (c), but my primary use-case revolves around time-series data (e.g., logs), where the interest is commonly on the most-recent entries, so "reverse-->offset—>limit” works nicely.  
> 
> Perhaps an alternative would be to lift a concept from Python with negative indexes so, for instance, offset=-N and limit=-N gives the last N entries?
> [Qin]: Yes, that's what I thought as well, with negative indexes, (b) and (c) seems to me, can be combined.

Can others comment on this?

Presumably, we could eliminate “direction” (c) with this approach.  

Without “direction”, I think that UIs can still support the ability to do column-sorts, whereby the user clicks on a column’s header to toggle ascending vs. descending presentation, but they’ll have to do it client-side.

That is, if wanting to see the 2nd page of results sorted by a column, something like:

	sort(column-name) --> offset(-2*pagesize) --> limit(pagesize)

Followed by the client then flipping the results to present the results in the user-selected order, right?

That said, given that DB-backends that support sorts commonly also support direction, it's unclear what this buys us.


>>> Sure, but I wonder if, e.g., a netmask filter, is supportable by 
>>> common DB-backends.  I’m hoping we have some DB-experts on the list!
>> 
>> See above.  It can be quite efficient even if the backend doesn't 
>> support it.
> 
> I don’t see that above, but I don’t doubt that it can be so, it’s just a whole lot of implementation complexity.  It seems that we should/must support servers doing it, we just need to find a way (node-tags?) to enable them to express that ability.
> [Qin]: My feeling is this efficiency more depends on the amount of data we need to request. If amount of data we request is huge, maybe, client-> server-> backend may be the better choice.

Is it the amount of data requested or the number of entries in the list?  At least, in my worldview, clients are always requesting a “page” of data, so that part is rather consistently small.

If the intention is to get a complete dump, then maybe the comment from yesterday applies, whereby streaming to an external repository that can be queried offline makes more sense?    - especially considering that the number of on-box logs is likely to be only the most recent (e.g., days), whereas the complete-dump type queries likely wish to extend well-past that.

K.