Re: [netconf] restconf collections

Martin Björklund <mbj+ietf@4668.se> Thu, 01 October 2020 06:24 UTC

Return-Path: <mbj+ietf@4668.se>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4356E3A0AE6 for <netconf@ietfa.amsl.com>; Wed, 30 Sep 2020 23:24:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.101
X-Spam-Level:
X-Spam-Status: No, score=-0.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_NAKED_TO_NUMERO=1.997, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=4668.se header.b=jukKqKQF; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=UIFPqZHc
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FZbB0MB6u3bn for <netconf@ietfa.amsl.com>; Wed, 30 Sep 2020 23:24:23 -0700 (PDT)
Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 697223A0AE9 for <netconf@ietf.org>; Wed, 30 Sep 2020 23:24:23 -0700 (PDT)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id A9FF25C07A5; Thu, 1 Oct 2020 02:24:22 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Thu, 01 Oct 2020 02:24:22 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=4668.se; h=date :message-id:to:cc:subject:from:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm2; bh= EYY5zIepcpy0kQPIniWK6t3ygOniHeYKEz8Xq69Ng6M=; b=jukKqKQFiql9VIX8 P0ZGQ3Szy+r/sEc575NrpcsgeEY1or4lHAPsiLDX30C3ACK+Z2ZrkzGeLSQWs3Pv twalumophtgx/Ttt1wSvABFVSNWjMenpUHY5XmeNDk4FjAPcKQHEv4xkNAbvxRR8 7leBg48pyhuhnq82eMhsP/azzMXl9G5ogdTBcPERfmdrCusABqUda+w7+NXxwSKI Arj/tTK37pqpQG1EJaufzgVUKKrVyFvk/YWsxKWGrBC0ylD8hgr4+LjWpFOTJNzg PoratGmQzQmYkz0Hk0+5M+i4s52zvb+vSk1alzpsSX7iftLyO1X87QDHRh/F0KnH lNkvGQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=EYY5zIepcpy0kQPIniWK6t3ygOniHeYKEz8Xq69Ng 6M=; b=UIFPqZHcfBUm9mApyy49Rfc2TbogT2h3IGH82X4AdWuonYwxqDbxA0D+F V2+RBcJ78VPKWR+7y1zxLjBtuptHb9PvKWm+DXj+ajVgnQ01UIJJR/3EQB6Tgq4E hiMkAz7zb7i29zq1XbE5/RIyKImC27X6seE85CYPSkFdUwIoFbonlmq3gCbkUsty CNjRMzJ+y8Pnqg9FIudtlMYpcGD/zrF1ZmoS+Z8x4PsQwQmx0g1GQOhMKTdQ6lMI +t9eamiJ8oqf1cnSEGFDx4gzvLX/+lVS1H/lIHXNI6ifBFwT85fp0rzdRLqyNNHb 5s89Bf2ONP45DBfL/n22pFD09X6Ig==
X-ME-Sender: <xms:FnZ1X6EmH-OulSx0c5MSQLPXD_-ErBIQqSapIX060GrEVrcZgOlP3Q> <xme:FnZ1X7U0WjNq2knaIcFtu-yibMyKhOqdelCGeb7EA3DUVQDpZSj7uD0pRefzBl1qb KjZyI_KL2KcXVkfrts>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrfeefgddutdelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffkffvuffhjghfofggtgfgsehtsg ertdertdejnecuhfhrohhmpeforghrthhinhcuuehjnphrkhhluhhnugcuoehmsghjodhi vghtfhesgeeiieekrdhsvgeqnecuggftrfgrthhtvghrnhepfeehkefgudekfeehieehhf ekgedvffehteevgfeffeeghfdvfeeuleduvddvgfehnecuffhomhgrihhnpehivghtfhdr ohhrghenucfkphepudehkedrudejgedrgedrvdduheenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmsghjodhivghtfhesgeeiieekrdhsvg
X-ME-Proxy: <xmx:FnZ1X0Jn_2wC-kX_mFmJtPldjj3iTKM4ghf5I77VcHaCWikHhLq7zw> <xmx:FnZ1X0GvP5Eys2FuCWq88VrXDEmF41E1CFMY1BvHEXKDyvooT31tmQ> <xmx:FnZ1XwXJwTnb5DIYTM2RJxfc5KoDwbGdytQBS5BYpPsh1PsETl8CmA> <xmx:FnZ1X5dq5cB8BVZklGtUIk1zvJJZBFA4-QXaXKoGHF94JfnBhRZyTg>
Received: from localhost (unknown [158.174.4.215]) by mail.messagingengine.com (Postfix) with ESMTPA id 910B43064686; Thu, 1 Oct 2020 02:24:21 -0400 (EDT)
Date: Thu, 01 Oct 2020 08:24:18 +0200 (CEST)
Message-Id: <20201001.082418.2014472804547759381.id@4668.se>
To: olof@hagsand.se
Cc: kent+ietf@watsen.net, netconf@ietf.org
From: Martin =?iso-8859-1?Q?Bj=F6rklund?= <mbj+ietf@4668.se>
In-Reply-To: <54cd8891-6c04-3ded-22ca-43a68ac8397f@hagsand.se>
References: <20200930.084312.246821928590534256.id@4668.se> <01000174e0279006-63781c55-866c-4ea0-add6-a6fa8a11453c-000000@email.amazonses.com> <54cd8891-6c04-3ded-22ca-43a68ac8397f@hagsand.se>
X-Mailer: Mew version 6.8 on Emacs 26.3
Mime-Version: 1.0
Content-Type: Text/Plain; charset=utf-8
Content-Transfer-Encoding: base64
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/7tyN7bYBMj-8vSrrk-TwL8YVRic>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 01 Oct 2020 06:24:25 -0000

Olof Hagsand <olof@hagsand.se> wrote:
> On 2020-09-30 19:55, Kent Watsen wrote:
> > 
> >> If we agree to not do d) then we don't need any use cases for it ;-)
> > 
> > We don’t agree that sorting isn’t ever need, but I think that it’s fair to say that, amongst the various other parameters, it’s the least important, and so enabling it via a feature seems prudent.
> > 
> > BTW, sort (d) comes after filter (e), in our e-->a processing model.  Hopefully the (fast) filter whittles the result-set down to something manageable so the (slow) sort isn’t painfully too egregious.
> 
> I agree sorting is less important (than a,b,e).
> Is it not also somewhat beyond the present yang concepts where there is
> already ordered-by and key indices?
> Supplying a differing sorting could in effect require a different set of
> index(es) which could be quite costly in runtime for large data sets.

Agreed.

> If you want to sort differently, should it not be present in the
> underlying model, ie as declaring a secondary set of keys, for example,
> as an extension of the original yang model (thus being out of scope here)?

The problem is that the designer of the data model is in most cases
not the client who asks for a particular sort order, and it is not
easy to anticipate all future use cases when a model is designed.


> 
> > 
> >> Anyway, if you want to sort on different fields you probably want to
> >> do this regardless of how the list is orderd on the server.
> > 
> > Possibly, and certainly it would be easy to implement, I’m just trying to understand when that might be the case.  For instance, using a firewall rulebase (an OBU-list) as an example, when could getting the rules in any other order be meaningful to a client?
> > 
> > 
> >>> Regarding “I don’t think they should be restricted to ‘indexable’
> >>> columns”, <snip/>
> >>
> >> My comment was for both d) and e) (sort and filter).  As for filter,
> >> an example is to get interfaces and with oper-status == 'down'.
> > 
> > “oper-status” appears to be an enum, an hence indexable…or perhaps this is a forward-reference to your next comment about some opstate not being in the DB...
> > 
> > 
> >>> Yes, XPath is the “right answer”, but we need to ensure it’s
> >>> constrained enough to be mappable to common DB-backends.
> 
> I would think a (strongly) constrained XPath would be a very nice solution.

I think constraing would be a mistake.  Note that we're talking about
XPath 1.0 here.  (I hope).



/martin


> >> So the client talks to the server, and the server talks to the "db"
> >> backend (note that in many implementations  operational state is
> >> typically not stored in a real database).  
> > 
> > Some opstate must be persisted, e.g., long-lived counters, logs, etc., but it’s a good point about other opstate not being persisted.   Perhaps “node-tags” can be used here, to differentiate which is which…and servers can indicate if/how they support the ephemeral opstate leafs in queries?
> > 
> > 
> >> So we have:
> >>
> >>  client ->  server ->  backend
> >>
> >> If we don't have any filter capabilities, the client has to get
> >> everything, and then filter.  If we support XPath and the backend
> >> doesn't support it, the server will have to filter.  This is still
> >> more efficient than filtering in the client.
> > 
> > Probably, especially when assuming the server has better resources and/or the client <--> server bandwidth is constrained.
> > 
> > 
> >> In ConfD/NSO we moved more and more filtering logic towards the
> >> backend, to speed up performance.
> > 
> > Ack.
> > 
> > 
> >> I think e) is way more important than c).  I suggest focus on a + b +
> >> c.
> > 
> > Noted, but I think you meant a + b + e.
> > 
> > Note sure how others feel about “direction: (c), but my primary use-case revolves around time-series data (e.g., logs), where the interest is commonly on the most-recent entries, so "reverse-->offset—>limit” works nicely.  
> > 
> > Perhaps an alternative would be to lift a concept from Python with negative indexes so, for instance, offset=-N and limit=-N gives the last N entries?
> > 
> 
> To me reverse direction seems a subset of ordering/sorting? If sorting
> is less important, so would an alternate direction?
> Unless there are some important use-cases, reverse order having the
> advantage of the same sorting mechanism/indexes, just in reverse (not
> introducing other keys, for example)
> 
> > 
> >>> Sure, but I wonder if, e.g., a netmask filter, is supportable by
> >>> common DB-backends.  I’m hoping we have some DB-experts on the list!
> >>
> >> See above.  It can be quite efficient even if the backend doesn't
> >> support it.
> > 
> > I don’t see that above, but I don’t doubt that it can be so, it’s just a whole lot of implementation complexity.  It seems that we should/must support servers doing it, we just need to find a way (node-tags?) to enable them to express that ability.
> > 
> > 
> > K.
> > 
> > 
> > 
> > _______________________________________________
> > netconf mailing list
> > netconf@ietf.org
> > https://www.ietf.org/mailman/listinfo/netconf
> > 
> 
> _______________________________________________
> netconf mailing list
> netconf@ietf.org
> https://www.ietf.org/mailman/listinfo/netconf