Re: [netconf] restconf collections

Martin Björklund <mbj+ietf@4668.se> Wed, 30 September 2020 06:43 UTC

Return-Path: <mbj+ietf@4668.se>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C8063A1272 for <netconf@ietfa.amsl.com>; Tue, 29 Sep 2020 23:43:28 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.1
X-Spam-Level:
X-Spam-Status: No, score=-0.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_NAKED_TO_NUMERO=1.997, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=4668.se header.b=pKn1hjJx; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=HNMXJzLE
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XUnLpVrJZKcA for <netconf@ietfa.amsl.com>; Tue, 29 Sep 2020 23:43:27 -0700 (PDT)
Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2B0273A0AA6 for <netconf@ietf.org>; Tue, 29 Sep 2020 23:43:16 -0700 (PDT)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 072D95C01F9; Wed, 30 Sep 2020 02:43:15 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 30 Sep 2020 02:43:15 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=4668.se; h=date :message-id:to:cc:subject:from:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm2; bh= Bz3r5b5A36pUrQQVrZ4LoS2H6oPA9fsVcqsSelZcZ+s=; b=pKn1hjJxXOfufk64 0+GXXn2syRcXYY3Pn0ZGGQwOOKILOAK31h33OxSI7lYac+ctnl5+dhxRbuk9EouU zzHZDGimAhI16S7dNyj4TFmgf5Q3fL3hZ2TPlIzBFvheJY4ghNMTJz1M3YoiiPab jy9tZziEepeZ41M5idiRWc66WGK7OlY2ELat6mFWpxB/Re4rrbc8QsufLCGPtE+c Q2NQYyIlKVwufldxSzKP6B/Mn9p3xYQwtJQ3mat25osEH4aYuI+O/blKNcPbO6iq RU6lA5c+9K4S+jGwtJfC6Mu20ICBVzCk3aiAVYRsFK7yD80Z+FkZjaK7zEP8mgfs CFhwWw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=Bz3r5b5A36pUrQQVrZ4LoS2H6oPA9fsVcqsSelZcZ +s=; b=HNMXJzLE/0cq8z+Xed8/zB3RC448W9463FS0G0d5NA7V5dXUFHmqTDOQ9 c80ZN+yaTk+PbhGIZQJFjeLs/cx9k+pfUajwK990wMN52EmFOuXbibfunXiBa6t/ qq2NnaC9ePKN2p3nfe2XVwDQdKUB8j0kHiwhqyC4C5BdQah1sO+0EBT5j+O2Usgu WuuM8TTzT0kBpVVq7XG+JgxGOPU5eR5zWJzKQRifq0E/eGgIGI2q8lhZ7VOaUxa4 /xmc1IVzfTPcf8ua9E0Ja6GWUF7WLBq9usgg4KS2cjKHgx9F6N1o/U60S9MiZkEl UdWHBBrc+h8/d7ZN6eoqFXhjcLBdg==
X-ME-Sender: <xms:Ail0X4RxYFNE791aXnMHzji1fOZCgkUWhZIZH66oSlTjyGUKpUlPpA> <xme:Ail0X1wJs4se36ZvVlmil-cGduZjXZLfGgX9b_fui22QfOJy9omomBpoxL6bIFWg_ gq5kBIYUENZcxyCfyA>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrfedtgdduuddtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffkffvuffhjghfofggtgfgsehtsg ertdertdejnecuhfhrohhmpeforghrthhinhcuuehjnphrkhhluhhnugcuoehmsghjodhi vghtfhesgeeiieekrdhsvgeqnecuggftrfgrthhtvghrnhepheetgedtfffggeffkedvje ekveelteeuuddttdffhfelgfetvdevhedvgeeutddunecukfhppeduheekrddujeegrdeg rddvudehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomh epmhgsjhdoihgvthhfseegieeikedrshgv
X-ME-Proxy: <xmx:Ail0X13807XKZchaUm8gDJQn_V_T9PeL2U_2rWwnHSJ0z04HOMdZcg> <xmx:Ail0X8BtnXDTGf32_z-0e26EzRC9KDXkaCaPXu91yfUM5bbWIgEjDg> <xmx:Ail0Xxj_5UzHQM52QWDwhrRG3pzSJTUr-VjpTC5ovDHSV35Fe8yyVw> <xmx:Ayl0X7eL9CK6Jvq27M8fnjcSAXUuX9H3PKMt-713zw6TH_kFJ7HU9Q>
Received: from localhost (unknown [158.174.4.215]) by mail.messagingengine.com (Postfix) with ESMTPA id 69C583064674; Wed, 30 Sep 2020 02:43:14 -0400 (EDT)
Date: Wed, 30 Sep 2020 08:43:12 +0200 (CEST)
Message-Id: <20200930.084312.246821928590534256.id@4668.se>
To: kent+ietf@watsen.net
Cc: netconf@ietf.org
From: Martin =?iso-8859-1?Q?Bj=F6rklund?= <mbj+ietf@4668.se>
In-Reply-To: <01000174dbde18ac-c9293572-775f-4c8e-b8c1-a8522bd636b5-000000@email.amazonses.com>
References: <01000174daf6ded3-0ba5564d-f1c7-4c65-90dc-d8a22f2f9395-000000@email.amazonses.com> <20200929.205710.165167541047857442.id@4668.se> <01000174dbde18ac-c9293572-775f-4c8e-b8c1-a8522bd636b5-000000@email.amazonses.com>
X-Mailer: Mew version 6.8 on Emacs 26.3
Mime-Version: 1.0
Content-Type: Text/Plain; charset=utf-8
Content-Transfer-Encoding: base64
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/kmTdcpiHh-o-6QF3_48QbqwdRQA>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 30 Sep 2020 06:43:28 -0000

Hi,

Kent Watsen <kent+ietf@watsen.net> wrote:
> Hi Martin,
> 
> > I don't think d) and e) should be restricted to obu lists, and I don't
> > think they should be restricted to "indexable" columns.
> > 
> > And I think "expression of some sort" should be XPath.
> > 
> > Also, since the main objective is efficient retrieval, "sort-by"
> > can perhaps be removed.  Consider large lists in operational state
> > that also change often.
> 
> Your last paragraph, if I read it correctly, says “maybe don’t do
> ‘d’”, which if applied to the first part of your response becomes "I
> don't think 'e' should be restricted to OBU-lists”, which is something
> I can agree with.  As for ‘d’ being a reasonable thing to support for
> OBU-lists, please explain the use-case.

If we agree to not do d) then we don't need any use cases for it ;-)

Anyway, if you want to sort on different fields you probably want to
do this regardless of how the list is orderd on the server.

> Regarding “I don’t think they should be restricted to ‘indexable’
> columns”, we need to keep in mind that, if the DB-backend can’t do it,
> then the server logic will need to it, likely by retrieving all
> records and brute-forcing its way through them, which would not only
> be painfully slow, but a lot of implementation complexity.  That said,
> I’m unsure what use-case you have in mind whereby a non-indexable
> column is being filtered/sorted.  Only a couple of the built-in types
> aren’t indexable, right?

My comment was for both d) and e) (sort and filter).  As for filter,
an example is to get interfaces and with oper-status == 'down'.

> Yes, XPath is the “right answer”, but we need to ensure it’s
> constrained enough to be mappable to common DB-backends.

So the client talks to the server, and the server talks to the "db"
backend (note that in many implementations  operational state is
typically not stored in a real database).  So we have:

  client ->  server ->  backend

If we don't have any filter capabilities, the client has to get
everything, and then filter.  If we support XPath and the backend
doesn't support it, the server will have to filter.  This is still
more efficient than filtering in the client.

In ConfD/NSO we moved more and more filtering logic towards the
backend, to speed up performance.


> Updating my previous response (per above, plus adding leaf-list, plus
> locking-in names, which I’m choosing only to NOT sound like SQL):
> 
> 
> 	For all lists and leaf-lists:
> 
> 		a) count		(uint, default: 0)
> 		b) skip		(uint, default: 0)
> 		c) direction	(enum: forwards/backwards, default: forwards)
> 
> 
> 	For non “ordered-by user” lists only (N/A to leaf-lists), but
> 	only for indexable columns:
> 
> 		d) sort-by		(string: a single column/leaf name)
> 
> 
> 	For all lists and leaf-lists, but only for indexable columns (note:
> 	for
> 	a leaf-list, it is its own indexable column):
> 		e) filter		(a constrained Xpath expression)
> 
> 
> 	Again: sequence of operation is e —> a.
> 
> 
> We haven’t discussed “feature” statements yet, but it seems reasonable
> to me for both ‘d’ and ‘e’ to be optionally implemented.
> Additionally, it may be that additional Xpath expressions can be
> unlocked by feature statements).

I think e) is way more important than c).  I suggest focus on a + b +
c.

> > We could define a few more XPath functions for such comparisons.
> 
> Sure, but I wonder if, e.g., a netmask filter, is supportable by
> common DB-backends.  I’m hoping we have some DB-experts on the list!

See above.  It can be quite efficient even if the backend doesn't
support it.


/martin