Re: [netconf] restconf collections

Kent Watsen <kent+ietf@watsen.net> Thu, 01 October 2020 19:54 UTC

Return-Path: <01000174e5ba8cc5-39edb446-48d9-4896-a6b5-dcf80f962762-000000@amazonses.watsen.net>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5158D3A0410 for <netconf@ietfa.amsl.com>; Thu, 1 Oct 2020 12:54:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.896
X-Spam-Level:
X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=amazonses.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FQEsut3Q5bOh for <netconf@ietfa.amsl.com>; Thu, 1 Oct 2020 12:54:05 -0700 (PDT)
Received: from a48-110.smtp-out.amazonses.com (a48-110.smtp-out.amazonses.com [54.240.48.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 960003A0400 for <netconf@ietf.org>; Thu, 1 Oct 2020 12:54:03 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1601582042; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:Feedback-ID; bh=YvU8fj+dzr7kx4K99DgMfYvv5EGTOe7IeIdspn+2WeM=; b=SRrZ84kVTOBbag5b3jqAvMBM6WOt4ZgQ6FQ5RZ/3L6gJE1L5Wvf5KkkQWMwLowih HvkL/bMIWr6iVmzaWyYiMN1vF81PcYwzyeGaylM+J5pes+jrE/zS89X8DrjoqqP+dNf ACz2oGWMKp1Gd/kARxCpY3B0Q2Ryov5l9PlfLYvo=
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\))
From: Kent Watsen <kent+ietf@watsen.net>
In-Reply-To: <20201001.165548.1275939966226069939.id@4668.se>
Date: Thu, 1 Oct 2020 19:54:02 +0000
Cc: Olof Hagsand <olof@hagsand.se>, "netconf@ietf.org" <netconf@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-ID: <01000174e5ba8cc5-39edb446-48d9-4896-a6b5-dcf80f962762-000000@email.amazonses.com>
References: <01000174e09a04a9-f53134f8-009f-4d0a-84a1-3165a494054b-000000@email.amazonses.com> <20201001.083349.1861081830665539794.id@4668.se> <01000174e4610ae6-d1488376-ae7e-4d95-ba7e-005fc44b9592-000000@email.amazonses.com> <20201001.165548.1275939966226069939.id@4668.se>
To: =?utf-8?Q?Martin_Bj=C3=B6rklund?= <mbj+ietf@4668.se>
X-Mailer: Apple Mail (2.3608.80.23.2.2)
X-SES-Outgoing: 2020.10.01-54.240.48.110
Feedback-ID: 1.us-east-1.DKmIRZFhhsBhtmFMNikgwZUWVrODEw9qVcPhqJEI2DA=:AmazonSES
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/U3EaYMdlpDRIW28Ul1xcjV7EyKI>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 01 Oct 2020 19:54:06 -0000

Hi Martin,

> I would prefer an unconstrained filter, with the XPath 1.0 that is
> already used in YANG.  It would be ok if support for such a filter was
> optional (a capability).  My point is that even if the backend doesn't
> support filtering on a particular node, filtering in the server will
> be much more efficient than filtering in the client.  Hence filtering
> shouldn't be constrained to certain nodes.  It also makes the solution
> much easier to use for clients.

1) Current NC/RC Xpath-filter found in <get> was not intended to filter large sets of logs.

2) The Xpath-filters used for streaming (s4.4.3 in rfc8040 and "stream-xpath-filter” in rfc8639) aren’t a fair comparison, as said stream-filters are already dispatching one event at a time.  But do note that rc8639 says the following, indciatiing that full Xpath cannot apply (e.g., preceding-sibling):

	The XPath expression is evaluated on the representation of
	individual, delineated event records as contained in
	the event stream.

3) Here we’re talking about querying a potentially large repository of logs that are at rest, and full table scans should avoided.  I suggest that we’re dealing with something new here.



IMO, for pretty much any CT data, full table scans isn’t a deal-breaker, as I don’t feel that configured data is ever that big (even for interfaces or a firewall rulebases).  It would be unfortunate for servers to have a reimplement all the filtering logic that is normally provided by backends (just because Xpath supports something that isn’t mappable), and it wouldn’t be as fast, but it is possible to implement and, to your point, it’s likely faster for the server to do it than the client.

But when it comes to CF data, doing a full table scan is impractical and index-backed queries can make the difference between the client waiting seconds vs minutes.  This is why I was initially trying to focus on just solving the CF problem, where the pain is acute.

That said, I propose we do “both”.  That is, the default is a customer-satisfying Xpath-subset known to be mappable to common DB-backends, and a feature statement is used to enable full Xpath support.  In both cases, the Xpath context node is the individual list-entry.

For some servers claiming full Xpath support, it could be that if a client’s query uses something beyond the mappable-subset, then the server silently switches to its “slow-path” logic (e.g., table scan until “count” is reached, if specified).  The server’s documentation can describe the behavior,  and even go so far as to suggests that it’s not that big of a deal for CT lists, but that the client may be waiting for a long time and/or the server may run out of resources, if applied to a big CF list.



> Right, but note that this filtering can and should be applied to all
> nodes, also operational state nodes that isn't stored in any database
> at all, imo.

This could be when a server silently switches to its slow-path logic.

K.