Re: [netconf] restconf collections

Kent Watsen <kent+ietf@watsen.net> Fri, 25 September 2020 15:49 UTC

Return-Path: <01000174c5f430be-3953d7e9-a364-4c55-8abb-58dbc233214f-000000@amazonses.watsen.net>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 14B333A1174 for <netconf@ietfa.amsl.com>; Fri, 25 Sep 2020 08:49:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.917
X-Spam-Level:
X-Spam-Status: No, score=-1.917 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=amazonses.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VjDXCV0Amafs for <netconf@ietfa.amsl.com>; Fri, 25 Sep 2020 08:49:10 -0700 (PDT)
Received: from a48-92.smtp-out.amazonses.com (a48-92.smtp-out.amazonses.com [54.240.48.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F96D3A1158 for <netconf@ietf.org>; Fri, 25 Sep 2020 08:49:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1601048949; h=From:Message-Id:Content-Type:Mime-Version:Subject:Date:In-Reply-To:Cc:To:References:Feedback-ID; bh=X0sNhrWQR7Y5V9lnLkR0tufdAGo3o/eMAMtVEfH1ibY=; b=AzwfQ5aT1pICnrbu5dOJvHEWhbq7LIV5Q/o3IC4AbVyaApbyuEizEHQizpOtCF8k us1Tv+InxGx6CL98a9xmjhqJJGRIhB7XPFtyGYXK/DH4YAT3XwSh+MID4ctIc1ZRr2I 01aq0BasC66mxHOUGuUk0AGWvgXCS1jywVKCOcac=
From: Kent Watsen <kent+ietf@watsen.net>
Message-ID: <01000174c5f430be-3953d7e9-a364-4c55-8abb-58dbc233214f-000000@email.amazonses.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_7457EE80-8B90-4ED1-B2F1-E4B900ADA6A5"
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\))
Date: Fri, 25 Sep 2020 15:49:09 +0000
In-Reply-To: <MWHPR11MB203298217DBFF10B168BC693DB360@MWHPR11MB2032.namprd11.prod.outlook.com>
Cc: "netconf@ietf.org" <netconf@ietf.org>
To: "Per Andersson (perander)" <perander@cisco.com>
References: <01000174c4eeb24e-c672ebe0-23a1-4f0e-afba-5b44d932e0ed-000000@email.amazonses.com> <MWHPR11MB203298217DBFF10B168BC693DB360@MWHPR11MB2032.namprd11.prod.outlook.com>
X-Mailer: Apple Mail (2.3608.80.23.2.2)
X-SES-Outgoing: 2020.09.25-54.240.48.92
Feedback-ID: 1.us-east-1.DKmIRZFhhsBhtmFMNikgwZUWVrODEw9qVcPhqJEI2DA=:AmazonSES
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/NOC9mz0L8R4ORRT8Rr3w0A_jWK0>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 25 Sep 2020 15:49:12 -0000

Hi Per,

> I agree it should be brushed off.

Interested in being a co-author?


> I have been looking at restconf-next on github where this has been mentioned.

Ref: https://github.com/netconf-wg/restconf-next/issues/1 <https://github.com/netconf-wg/restconf-next/issues/1>


> Will it be part of restconf-next or a separate effort?

If an NBC change is needed, it would have to be a -bis, otherwise, if all backwards-compatible, it could be a -bis or a new RFC that "updates" RFC 8040.


> Today we implement limit and offset as proprietary extensions to RESTCONF.
> It would be good to have them (or some other pagination mechanism)
> standardized.

By "we", do you mean the Tail-F group within Cisco?  What I'm really asking here is if the proprietary extensions are essentially captured by Martin in the restconf-collections draft.


> If I recall correctly, we have use cases for config true list sizes in the thousands
> or even tens of thousands.

List of "interfaces" is the quintessential example I've heard in the past.   Does your implementation support the extensions of CT lists?    If so, does it support them on "ordered-by user" lists too?    Is the use-case (to support on CT lists) a "nice-to-have" or a "must-have"?   If a "must-have", how is it justified?



>> Regarding solution, I suggest a two-fold approach:
>> 
>> 1) only support query parameters (e.g., offset, limit, sort, where) if a "config false" list is queried directly.  E.g., GET {+restconf}/ds/ietf-datastores:operational/example-module:widgets/widget?offset=20?limit=50.  Note that RESTCONF does not allow the list itself to be a resource target is exactly why this makes sense.
> 
> I would suggest supporting (at least) pagination for any list.

You mean to let the "limit" and "offset" params be available for both CT and CF lists, but limit "sort" and "where" to CF lists?  That might work, as it sidesteps the odd "sorting a user-sorted list" case...


> What is the point of selective support? As a user I would find it strange.

If by "selective support" you mean the "where" parameter, I think that it's only useful for CF lists (e.g., logs) whereby the client only wishes to fetch entries matching some criteria (e.g., where user==foo or source-addr=bar).  Makes sense?


> It feels like the implementation could be simpler with the implicit ?limit=1
> behaviour. Even with pagination the list could be large and perhaps
> streamed out which would make collection of metadata more challenging.

An implicit ?limit=1 would be simple to implement, but assuming we're trying to introduce the behavior in a backwards-compatible manner, how to differentiate a truncated list from a list that actually contains just one value?   Assuming a legacy client (that does not know to look for or grok a ietf-restconf-collections module in yang-library)...or are you saying that the current behavior is so broken that such an NBC change would actually be welcomed by legacy clients?


> Another possibility is to let the client handle such queries, instead of
> replacing list contents with a placeholder let the client set ?depth=1
> and effectively omit list contents..

Assuming the "depth" query parameter is supported, but the server could still *implicitly* support ?depth=1 w/o supporting the capability.  That said, the response would still be the size of the list, right?   I've also thought to let server's implicitly return as if "?fields=<key>*" (i.e., just return the keys) has been passed but, again, the response would be the size of the list.  Such options might be okay for CT-lists, but not so much for the potentially enormous CF-lists I have in mind...


K.