Re: [netconf] restconf collections

Martin Björklund <mbj+ietf@4668.se> Fri, 02 October 2020 06:23 UTC

Return-Path: <mbj+ietf@4668.se>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 80E573A0E7C for <netconf@ietfa.amsl.com>; Thu, 1 Oct 2020 23:23:37 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.1
X-Spam-Level:
X-Spam-Status: No, score=-0.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_NAKED_TO_NUMERO=1.997, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=4668.se header.b=evBbUCfV; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=jZzoU6Xt
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K9vjRvgZ-DRD for <netconf@ietfa.amsl.com>; Thu, 1 Oct 2020 23:23:36 -0700 (PDT)
Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0129E3A0E7A for <netconf@ietf.org>; Thu, 1 Oct 2020 23:23:35 -0700 (PDT)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 39D1E5C00D1; Fri, 2 Oct 2020 02:23:35 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 02 Oct 2020 02:23:35 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=4668.se; h=date :message-id:to:cc:subject:from:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm2; bh= u291mqT7TWW8Z+MNIgDDJ+8K7UPGF1t4Yzqmi5fTZ2Y=; b=evBbUCfVejwkgXzJ Z3o87JPlRAYLaJ5YeF1CpSkBu5p8D1tr2uT+umzCCJo1S6fGHxQ8LJe07cn43lIU mjLvUTXJJdtFVcegyqYVK1z/aWY5OU6WSYtyHmpCiKOCYdecMC/th0zIDrtkpvwM A8z/nb13t1Uca6CCaBt+UqKcHROZTdVvhRHXA/i5Mndp+nNrMfPr4QOhMb0sCE+m Hka2b4phsAW5Thw+zWUQje4IIrLm60j2MuTOqq+oLaukhw4hbA+Q7uO7pZkyym8R SOhRQfWKfRyi02Q5B6nppTKw3hyZDP7guGWrmZ3DrLJbX7zSsXzP1wJPA3X4RD3T 3U4Z2Q==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=u291mqT7TWW8Z+MNIgDDJ+8K7UPGF1t4Yzqmi5fTZ 2Y=; b=jZzoU6XtmZDg9qGrPVHlHT4NMhTzKZXkpOFk/zEggvKq5mt0Mo2Wf3NHt 9ovJ7gVunS8HKq4iIKTB3kJ0u0IkT6cxQs4iD+a8YlvMvsmMqPdb+vMyEZAFRMrg 3wXOrZcCyRKBpRxtxnCs83n/3o4V6kGnd8fc/Czh7XNHLdUqMyMR+pbZdBSe9emJ xTocx4OFY7CLu03Iqt9EZf0yoAbw/hXer7vJjgOPKSr0FTUNz+AkwH7kyKhdqy+U ZG2ZoWmSAcuWc4IdjMZ6+7g86DO8m2DedHAIYuFzyRefCN07T3FmgN0bHE8oqr7v WY9HqM4XJpRkO/jnDmTKbJFpoueZA==
X-ME-Sender: <xms:Zcd2X2q63cE2ttmeiKa7tXT50Y2rpTIjy7h_Ch1cFQ2sjE85L-mEAA> <xme:Zcd2X0q3kc5LqjTX_T2I_MtWPOj7Gq1VgcTehMUimoUvTU4lx_6HkSxTxNcfONTqD piecvmv-8XBnJUBsFc>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrfeehgddutdefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffkffvuffhjghfofggtgfgsehtsg ertdertdejnecuhfhrohhmpeforghrthhinhcuuehjnphrkhhluhhnugcuoehmsghjodhi vghtfhesgeeiieekrdhsvgeqnecuggftrfgrthhtvghrnhepheetgedtfffggeffkedvje ekveelteeuuddttdffhfelgfetvdevhedvgeeutddunecukfhppeduheekrddujeegrdeg rddvudehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomh epmhgsjhdoihgvthhfseegieeikedrshgv
X-ME-Proxy: <xmx:Zcd2X7PeT_qJNa7PjM0tjR1x3PuI16xVL3PGvaMtWEnriZlB_0fjWw> <xmx:Zcd2X14Q_iEX4fMexRzY5tUspex_tal1J1w_jpWRPwN7GrDnwdwKUA> <xmx:Zcd2X14wzZLt_Osw4MwpuEY6XQgW5wAS4x9rdvwCme3v29nulj27ng> <xmx:Z8d2Xyj_yGSIGFZDPcu7clll4D9vnCGTIJzKTGFTnt6-04V0xxSxUw>
Received: from localhost (unknown [158.174.4.215]) by mail.messagingengine.com (Postfix) with ESMTPA id 2D8793064680; Fri, 2 Oct 2020 02:23:33 -0400 (EDT)
Date: Fri, 02 Oct 2020 08:23:31 +0200 (CEST)
Message-Id: <20201002.082331.2242297379846191383.id@4668.se>
To: kent+ietf@watsen.net
Cc: olof@hagsand.se, netconf@ietf.org
From: Martin =?iso-8859-1?Q?Bj=F6rklund?= <mbj+ietf@4668.se>
In-Reply-To: <01000174e5ba8cc5-39edb446-48d9-4896-a6b5-dcf80f962762-000000@email.amazonses.com>
References: <01000174e4610ae6-d1488376-ae7e-4d95-ba7e-005fc44b9592-000000@email.amazonses.com> <20201001.165548.1275939966226069939.id@4668.se> <01000174e5ba8cc5-39edb446-48d9-4896-a6b5-dcf80f962762-000000@email.amazonses.com>
X-Mailer: Mew version 6.8 on Emacs 26.3
Mime-Version: 1.0
Content-Type: Text/Plain; charset=utf-8
Content-Transfer-Encoding: base64
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/eh6eSjD9x9g33DqT8WJgZmiVmSw>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 02 Oct 2020 06:23:38 -0000

Hi,

Kent Watsen <kent+ietf@watsen.net> wrote:
> Hi Martin,
> 
> > I would prefer an unconstrained filter, with the XPath 1.0 that is
> > already used in YANG.  It would be ok if support for such a filter was
> > optional (a capability).  My point is that even if the backend doesn't
> > support filtering on a particular node, filtering in the server will
> > be much more efficient than filtering in the client.  Hence filtering
> > shouldn't be constrained to certain nodes.  It also makes the solution
> > much easier to use for clients.
> 
> 1) Current NC/RC Xpath-filter found in <get> was not intended to
> filter large sets of logs.
> 
> 2) The Xpath-filters used for streaming (s4.4.3 in rfc8040 and
> "stream-xpath-filter” in rfc8639) aren’t a fair comparison, as said
> stream-filters are already dispatching one event at a time.  But do
> note that rc8639 says the following, indciatiing that full Xpath
> cannot apply (e.g., preceding-sibling):
> 
> 	The XPath expression is evaluated on the representation of
> 	individual, delineated event records as contained in
> 	the event stream.
> 
> 3) Here we’re talking about querying a potentially large repository of
> logs that are at rest, and full table scans should avoided.  I suggest
> that we’re dealing with something new here.

Perhaps we don't understand each other.  Here's an pseudo-code example
of what I envision:

  GET /interfaces/interface
  filter  'oper-status = "up" and statistics/in-errors > 0'

This is XPath 1.0, applied to one instance at the time.



> IMO, for pretty much any CT data, full table scans isn’t a
> deal-breaker, as I don’t feel that configured data is ever that big
> (even for interfaces or a firewall rulebases).  It would be
> unfortunate for servers to have a reimplement all the filtering logic
> that is normally provided by backends (just because Xpath supports
> something that isn’t mappable), and it wouldn’t be as fast, but it is
> possible to implement and, to your point, it’s likely faster for the
> server to do it than the client.
> 
> But when it comes to CF data, doing a full table scan is impractical
> and index-backed queries can make the difference between the client
> waiting seconds vs minutes.  This is why I was initially trying to
> focus on just solving the CF problem, where the pain is acute.

Well, it is less likely that an implementation uses a feature-rich (in
terms of query language) DB to store all its state, than its config.
So it would probably be easier to implement this using "an index" for
config than for config false...

> That said, I propose we do “both”.  That is, the default is a
> customer-satisfying Xpath-subset known to be mappable to common
> DB-backends, and a feature statement is used to enable full Xpath
> support.  In both cases, the Xpath context node is the individual
> list-entry.
> 
> For some servers claiming full Xpath support, it could be that if a
> client’s query uses something beyond the mappable-subset, then the
> server silently switches to its “slow-path” logic (e.g., table scan
> until “count” is reached, if specified).  The server’s documentation
> can describe the behavior, and even go so far as to suggests that it’s
> not that big of a deal for CT lists, but that the client may be
> waiting for a long time and/or the server may run out of resources, if
> applied to a big CF list.

I agree.

> > Right, but note that this filtering can and should be applied to all
> > nodes, also operational state nodes that isn't stored in any database
> > at all, imo.
> 
> This could be when a server silently switches to its slow-path logic.
> 
> K.


/martin