Re: [netconf] restconf collections

[Kent Watsen <[e-mail]>]

Hi Martin,

> I would prefer an unconstrained filter, with the XPath 1.0 that is
> already used in YANG.  It would be ok if support for such a filter was
> optional (a capability).  My point is that even if the backend doesn't
> support filtering on a particular node, filtering in the server will
> be much more efficient than filtering in the client.  Hence filtering
> shouldn't be constrained to certain nodes.  It also makes the solution
> much easier to use for clients.

1) Current NC/RC Xpath-filter found in <get> was not intended to filter large sets of logs.

2) The Xpath-filters used for streaming (s4.4.3 in rfc8040 and "stream-xpath-filter” in rfc8639) aren’t a fair comparison, as said stream-filters are already dispatching one event at a time.  But do note that rc8639 says the following, indciatiing that full Xpath cannot apply (e.g., preceding-sibling):

	The XPath expression is evaluated on the representation of
	individual, delineated event records as contained in
	the event stream.

3) Here we’re talking about querying a potentially large repository of logs that are at rest, and full table scans should avoided.  I suggest that we’re dealing with something new here.



IMO, for pretty much any CT data, full table scans isn’t a deal-breaker, as I don’t feel that configured data is ever that big (even for interfaces or a firewall rulebases).  It would be unfortunate for servers to have a reimplement all the filtering logic that is normally provided by backends (just because Xpath supports something that isn’t mappable), and it wouldn’t be as fast, but it is possible to implement and, to your point, it’s likely faster for the server to do it than the client.

But when it comes to CF data, doing a full table scan is impractical and index-backed queries can make the difference between the client waiting seconds vs minutes.  This is why I was initially trying to focus on just solving the CF problem, where the pain is acute.

That said, I propose we do “both”.  That is, the default is a customer-satisfying Xpath-subset known to be mappable to common DB-backends, and a feature statement is used to enable full Xpath support.  In both cases, the Xpath context node is the individual list-entry.

For some servers claiming full Xpath support, it could be that if a client’s query uses something beyond the mappable-subset, then the server silently switches to its “slow-path” logic (e.g., table scan until “count” is reached, if specified).  The server’s documentation can describe the behavior,  and even go so far as to suggests that it’s not that big of a deal for CT lists, but that the client may be waiting for a long time and/or the server may run out of resources, if applied to a big CF list.



> Right, but note that this filtering can and should be applied to all
> nodes, also operational state nodes that isn't stored in any database
> at all, imo.

This could be when a server silently switches to its slow-path logic.

K.