Re: [netconf] restconf collections

Kent Watsen <kent+ietf@watsen.net> Thu, 01 October 2020 13:36 UTC

Return-Path: <01000174e4610ae6-d1488376-ae7e-4d95-ba7e-005fc44b9592-000000@amazonses.watsen.net>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2C8F13A1030 for <netconf@ietfa.amsl.com>; Thu, 1 Oct 2020 06:36:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.895
X-Spam-Level:
X-Spam-Status: No, score=-1.895 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=amazonses.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JJcDSfa7mOAa for <netconf@ietfa.amsl.com>; Thu, 1 Oct 2020 06:36:40 -0700 (PDT)
Received: from a48-95.smtp-out.amazonses.com (a48-95.smtp-out.amazonses.com [54.240.48.95]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5F25D3A1056 for <netconf@ietf.org>; Thu, 1 Oct 2020 06:36:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1601559399; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:Feedback-ID; bh=fnQNj64pj0GqOdOM4hUlAJP29I0fiiktCwWytHqIiGM=; b=Of0iQjYYViAXxzKepEPMT9kqKXjrgm0fx/IVMkL9Yc4wCqIrdmwWoRUrlFneipzv 5yucXVAwNMHP3gNUR1eiglHaLQgyJ4NrfhWY4mGujanoycV6ZU51gKrP2Uma2SzTAn5 67nLbuLb0clvjh0opeUb+Vkn1mdZfvrlwwgsGhh8=
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\))
From: Kent Watsen <kent+ietf@watsen.net>
In-Reply-To: <20201001.083349.1861081830665539794.id@4668.se>
Date: Thu, 1 Oct 2020 13:36:39 +0000
Cc: Olof Hagsand <olof@hagsand.se>, "netconf@ietf.org" <netconf@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-ID: <01000174e4610ae6-d1488376-ae7e-4d95-ba7e-005fc44b9592-000000@email.amazonses.com>
References: <01000174e0279006-63781c55-866c-4ea0-add6-a6fa8a11453c-000000@email.amazonses.com> <54cd8891-6c04-3ded-22ca-43a68ac8397f@hagsand.se> <01000174e09a04a9-f53134f8-009f-4d0a-84a1-3165a494054b-000000@email.amazonses.com> <20201001.083349.1861081830665539794.id@4668.se>
To: =?utf-8?Q?Martin_Bj=C3=B6rklund?= <mbj+ietf@4668.se>
X-Mailer: Apple Mail (2.3608.80.23.2.2)
X-SES-Outgoing: 2020.10.01-54.240.48.95
Feedback-ID: 1.us-east-1.DKmIRZFhhsBhtmFMNikgwZUWVrODEw9qVcPhqJEI2DA=:AmazonSES
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/Jkp4EgnQC_dhKKen-VSQldXJ6O0>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 01 Oct 2020 13:36:42 -0000

Hi Martin,

>> A couple messages I sent yesterday alluded to defining an extension
>> statement that could be added to nodes in the YANG modules.
>> 
>> Now I’m thinking that one-size doesn’t necessarily fit all, and
>> something like a server-specific “node-tag” would be more flexible.
>> (https://tools.ietf.org/html/draft-tao-netmod-yang-node-tags
>> <https://tools.ietf.org/html/draft-tao-netmod-yang-node-tags>)
>> 
>> Maybe the answer is somewhere in-between, i.e., an extension that acts
>> like a “hint” and a node-tag indicating if the server delivered it.
>> Bad news w/ an extension is that legacy modules would need to be
>> revved or augmented.
>> 
>> One thing, though, filtering on non-indexed nodes is guaranteed to
>> have slow performance, thus it seems that we need to enable indexing
>> for more than just a list’s keys.
> 
> I disagree with both these statements.  This is an implementation
> issue.

Which two statements?   And how do you come to that conclusion, unless you’re suggested that all queries from particular server are equally slow?  ;)


>> My conjecture is that, once we have the indexes for filtering (e),
>> they can be used for sorting (d).  At least, that’s the way it works
>> with some DB-backends I’m familiar with...
> 
> This is one implementation strategy.  We should not design the
> standard around one implementation.

Please elucidate how this isn’t the case.

While a true statement, it’s an unfair implication, as pretty much *everything* I’ve written on this thread has been about trying to enable servers with various DB-backends to express which parts of the solution they can support.   Apparently some backends don’t support “sorts” while others do; some backends can support full Xpath while other can’t, etc.   All this means to me is that we need to define the necessary features and/or node-tags or whatever to enable each to advertise what it can do best, or at all.

If the WG wants the lowest-common denominator, we will likely end up with just ‘a', ‘b', and a heavily redacted Xpath for ‘e’.  I’m hoping that we can do better than that, but I’m also okay with leaving it to each solution can then create proprietary extensions/node-tags, if that is the WG consensus.


> I agree that this can be useful.  But as has been pointed out earlier
> in this thread, sorting implies a stateful solution which will consume
> more resources and probably comes with some additional complexity.

True, except I think the augment was in *favor* of doing that, at least in some cases.

Also note that the “more resources” augment needs to be amortized over the number of queries.  There must be a break-even point whereby preparing a (resource-intensive) snapshot actually uses less resources over time, after it's been queried a certain number of times, than a server doing a fresh (lower-resource) query for each request.


> At Cisco/tail-f we implemented such a solution; a more generic "query"
> mechanism, for both NETCONF and RESTCONF.  Perhaps Per A can share
> some details.

That would be helpful.  The analysis can only take into account that which is known.  I know the Tail-f products used a custom database, but few know anything about it.  

Creating a custom database is not easy.  I know, as I personally wrote a very-fast logging system, complete with indices, for a logging database taking in logs at the rate of 50,000 logs per second, with the total number of logs retained measured in the billions.  

I personally do not wish for a *basic* server to have to go through that pain, hence the interest in supporting common DB-backends (NoSQL, SQL, etc.), while also enabling systems with special abilities to present that which can.

K.