Re: [netconf] restconf collections

Martin Björklund <> Thu, 01 October 2020 14:55 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 8F4223A10B9 for <>; Thu, 1 Oct 2020 07:55:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -0.801
X-Spam-Status: No, score=-0.801 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_NAKED_TO_NUMERO=1.997, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key) header.b=HRUb5b/V; dkim=pass (2048-bit key) header.b=FOXqld+R
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 1Zf6BTo7_BWP for <>; Thu, 1 Oct 2020 07:55:54 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id DB9923A10BC for <>; Thu, 1 Oct 2020 07:55:54 -0700 (PDT)
Received: from compute4.internal (compute4.nyi.internal []) by mailout.west.internal (Postfix) with ESMTP id 13AC729E; Thu, 1 Oct 2020 10:55:53 -0400 (EDT)
Received: from mailfrontend2 ([]) by compute4.internal (MEProxy); Thu, 01 Oct 2020 10:55:53 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; h=date :message-id:to:cc:subject:from:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm2; bh= NSgCjzXMLC1RCgv434QIzqz06KnMY9+xX/4LED9Vi34=; b=HRUb5b/VLq+0ROBo GbOT03gqnVq3gNQUp/MUUX89kcnb1YFh4totY7/k8AjjaWb1Xrp1gAoaE/LIs89j cWhcqKUTWEc0OVJUDQdDyqMIAETFjsdEws/C78h8SxM5qutOKleoWOrrI/pvccn5 8UbdK9fl42aEDkWI0V5o+eUS4/njRUTH7UV75AfnGehesmzB65+tCcL2Eve6KNqf BIUoFYUGhVF2WZiRGdH1pZ+KoxH35MaLLe/p8FywTLsOAupSTb5ZAilJ1hU07ax7 sKlFnakD58IP01TJO5fpUQSs/ad35HS/OUgcGBGv2ogcPXPDHX0uVIzvn/WEFkOg 58dazw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=NSgCjzXMLC1RCgv434QIzqz06KnMY9+xX/4LED9Vi 34=; b=FOXqld+Rj6940nON9KxkcSAcJOG8ub4NLLeU49A9+xmHjXV4Q5shF77bp 7qCI4N1vtlRfBuuPFQmGGJ2K1/0eZZuFTimxRU7ye2AoPCJhLR/DrwUFZ8thV3d0 HASk04SAQa/3f4OSOIY+/Qu3Um7Tjm5ekZfH+otDSSVqT9ntSA9m/Hq/MHiCnqID 9sFlBFbpqLv69CHGAdCb6CwJDzw1T4zj0M363rSg/G8EGTbWr9WRNcHyr6M3Rj0k e9MMD8PW+eSJofkTbqb6yTQSUhPjygssc2bYDDMawk2OEX7Eh8OS3PoYvmSeP8Rq Md/kKFiz9Wii9CdmwAQuV7LGolChg==
X-ME-Sender: <xms:9-11Xziy640cBFGMIk20qnvhk_MIPI4t3bzk9y2fhxXOw9nx3Uvcdg> <xme:9-11XwBOD1NgR6nHb3kUTkKrUtFlpMYnn_Y4DESv2AKxlDSgPAcN-7flg7sPPbqE0 o3bN_W81wKm_6sfrJQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrfeeggdekfecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffkvffuhfgjfhfogggtgfesthgsre dtredtjeenucfhrhhomhepofgrrhhtihhnuceujhpnrhhklhhunhguuceomhgsjhdoihgv thhfseegieeikedrshgvqeenucggtffrrghtthgvrhhnpeefheekgfdukeefheeihefhke egvdffheetvefgfeefgefhvdefueeluddvvdfgheenucffohhmrghinhepihgvthhfrdho rhhgnecukfhppeduheekrddujeegrdegrddvudehnecuvehluhhsthgvrhfuihiivgeptd enucfrrghrrghmpehmrghilhhfrhhomhepmhgsjhdoihgvthhfseegieeikedrshgv
X-ME-Proxy: <xmx:9-11XzGO-PzyJx4iFyz1Z3dAOHPKZxAdpO5B7T-ziaNBNvkbnJf14g> <xmx:9-11XwT9o92G0g9h7G0i0g9iG6Ka7--dge6YWLR5VWnuMw0L27HUwg> <xmx:9-11XwwXGKgV8wIuCA-4aAgfxfdMrMw4BkxhCaezl21BMtL3EMIrCQ> <xmx:-O11X2bgvCAsWD7ypPZ6KCQNZ6EhnsurDtRjeFSWVwrxTn819hIJOQ>
Received: from localhost (unknown []) by (Postfix) with ESMTPA id 84A033064682; Thu, 1 Oct 2020 10:55:50 -0400 (EDT)
Date: Thu, 01 Oct 2020 16:55:48 +0200
Message-Id: <>
From: Martin Björklund <>
In-Reply-To: <>
References: <> <> <>
X-Mailer: Mew version 6.8 on Emacs 26.3
Mime-Version: 1.0
Content-Type: Text/Plain; charset="utf-8"
Content-Transfer-Encoding: base64
Archived-At: <>
Subject: Re: [netconf] restconf collections
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 01 Oct 2020 14:55:57 -0000


Kent Watsen <> wrote:
> Hi Martin,
> >> A couple messages I sent yesterday alluded to defining an extension
> >> statement that could be added to nodes in the YANG modules.
> >> 
> >> Now I’m thinking that one-size doesn’t necessarily fit all, and
> >> something like a server-specific “node-tag” would be more flexible.
> >> (
> >> <>)
> >> 
> >> Maybe the answer is somewhere in-between, i.e., an extension that acts
> >> like a “hint” and a node-tag indicating if the server delivered it.
> >> Bad news w/ an extension is that legacy modules would need to be
> >> revved or augmented.
> >> 
> >> One thing, though, filtering on non-indexed nodes is guaranteed to
> >> have slow performance, thus it seems that we need to enable indexing
> >> for more than just a list’s keys.
> > 
> > I disagree with both these statements.  This is an implementation
> > issue.
> Which two statements?

1. filtering on non-indexed nodes is guaranteed to have slow

2. it seems that we need to enable indexing  for more than just a
   list’s keys

> And how do you come to that conclusion, unless
> you’re suggested that all queries from particular server are equally
> slow?  ;)

I'm suggesting that this is highly implementation dependent.

> >> My conjecture is that, once we have the indexes for filtering (e),
> >> they can be used for sorting (d).  At least, that’s the way it works
> >> with some DB-backends I’m familiar with...
> > 
> > This is one implementation strategy.  We should not design the
> > standard around one implementation.
> Please elucidate how this isn’t the case.
> While a true statement, it’s an unfair implication, as pretty much
> *everything* I’ve written on this thread has been about trying to
> enable servers with various DB-backends to express which parts of the
> solution they can support.  Apparently some backends don’t support
> “sorts” while others do; some backends can support full Xpath while
> other can’t, etc.  All this means to me is that we need to define the
> necessary features and/or node-tags or whatever to enable each to
> advertise what it can do best, or at all.
> If the WG wants the lowest-common denominator, we will likely end up
> with just ‘a', ‘b', and a heavily redacted Xpath for ‘e’.  I’m hoping
> that we can do better than that, but I’m also okay with leaving it to
> each solution can then create proprietary extensions/node-tags, if
> that is the WG consensus.

I would prefer an unconstrained filter, with the XPath 1.0 that is
already used in YANG.  It would be ok if support for such a filter was
optional (a capability).  My point is that even if the backend doesn't
support filtering on a particular node, filtering in the server will
be much more efficient than filtering in the client.  Hence filtering
shouldn't be constrained to certain nodes.  It also makes the solution
much easier to use for clients.

> > I agree that this can be useful.  But as has been pointed out earlier
> > in this thread, sorting implies a stateful solution which will consume
> > more resources and probably comes with some additional complexity.
> True, except I think the augment was in *favor* of doing that, at
> least in some cases.
> Also note that the “more resources” augment needs to be amortized over
> the number of queries.  There must be a break-even point whereby
> preparing a (resource-intensive) snapshot actually uses less resources
> over time, after it's been queried a certain number of times, than a
> server doing a fresh (lower-resource) query for each request.

To be clear, I think such a solution would be useful.

> > At Cisco/tail-f we implemented such a solution; a more generic "query"
> > mechanism, for both NETCONF and RESTCONF.  Perhaps Per A can share
> > some details.
> That would be helpful.  The analysis can only take into account that
> which is known.  I know the Tail-f products used a custom database,
> but few know anything about it.
> Creating a custom database is not easy.

Right, but note that this filtering can and should be applied to all
nodes, also operational state nodes that isn't stored in any database
at all, imo.


> I know, as I personally wrote
> a very-fast logging system, complete with indices, for a logging
> database taking in logs at the rate of 50,000 logs per second, with
> the total number of logs retained measured in the billions.
> I personally do not wish for a *basic* server to have to go through
> that pain, hence the interest in supporting common DB-backends (NoSQL,
> SQL, etc.), while also enabling systems with special abilities to
> present that which can.
> K.