Re: [netconf] restconf collections

[Martin Björklund <[e-mail]>]

Hi,

Kent Watsen <[e-mail]> wrote:
> Hi Martin,
> 
> > I would prefer an unconstrained filter, with the XPath 1.0 that is
> > already used in YANG.  It would be ok if support for such a filter was
> > optional (a capability).  My point is that even if the backend doesn't
> > support filtering on a particular node, filtering in the server will
> > be much more efficient than filtering in the client.  Hence filtering
> > shouldn't be constrained to certain nodes.  It also makes the solution
> > much easier to use for clients.
> 
> 1) Current NC/RC Xpath-filter found in <get> was not intended to
> filter large sets of logs.
> 
> 2) The Xpath-filters used for streaming (s4.4.3 in rfc8040 and
> "stream-xpath-filter” in rfc8639) aren’t a fair comparison, as said
> stream-filters are already dispatching one event at a time.  But do
> note that rc8639 says the following, indciatiing that full Xpath
> cannot apply (e.g., preceding-sibling):
> 
> 	The XPath expression is evaluated on the representation of
> 	individual, delineated event records as contained in
> 	the event stream.
> 
> 3) Here we’re talking about querying a potentially large repository of
> logs that are at rest, and full table scans should avoided.  I suggest
> that we’re dealing with something new here.

Perhaps we don't understand each other.  Here's an pseudo-code example
of what I envision:

  GET /interfaces/interface
  filter  'oper-status = "up" and statistics/in-errors > 0'

This is XPath 1.0, applied to one instance at the time.



> IMO, for pretty much any CT data, full table scans isn’t a
> deal-breaker, as I don’t feel that configured data is ever that big
> (even for interfaces or a firewall rulebases).  It would be
> unfortunate for servers to have a reimplement all the filtering logic
> that is normally provided by backends (just because Xpath supports
> something that isn’t mappable), and it wouldn’t be as fast, but it is
> possible to implement and, to your point, it’s likely faster for the
> server to do it than the client.
> 
> But when it comes to CF data, doing a full table scan is impractical
> and index-backed queries can make the difference between the client
> waiting seconds vs minutes.  This is why I was initially trying to
> focus on just solving the CF problem, where the pain is acute.

Well, it is less likely that an implementation uses a feature-rich (in
terms of query language) DB to store all its state, than its config.
So it would probably be easier to implement this using "an index" for
config than for config false...

> That said, I propose we do “both”.  That is, the default is a
> customer-satisfying Xpath-subset known to be mappable to common
> DB-backends, and a feature statement is used to enable full Xpath
> support.  In both cases, the Xpath context node is the individual
> list-entry.
> 
> For some servers claiming full Xpath support, it could be that if a
> client’s query uses something beyond the mappable-subset, then the
> server silently switches to its “slow-path” logic (e.g., table scan
> until “count” is reached, if specified).  The server’s documentation
> can describe the behavior, and even go so far as to suggests that it’s
> not that big of a deal for CT lists, but that the client may be
> waiting for a long time and/or the server may run out of resources, if
> applied to a big CF list.

I agree.

> > Right, but note that this filtering can and should be applied to all
> > nodes, also operational state nodes that isn't stored in any database
> > at all, imo.
> 
> This could be when a server silently switches to its slow-path logic.
> 
> K.


/martin