Re: [netconf] restconf collections

[Hongwei Li <[e-mail]>]

Hi Kent,


On Fri, Oct 2, 2020 at 9:07 AM Kent Watsen <[e-mail]> wrote:

>
> 3) Here we’re talking about querying a potentially large repository of
> logs that are at rest, and full table scans should avoided.  I suggest
> that we’re dealing with something new here.
>
>
> Perhaps we don't understand each other.  Here's an pseudo-code example
> of what I envision:
>
>  GET /interfaces/interface
>  filter  'oper-status = "up" and statistics/in-errors > 0'
>
> This is XPath 1.0, applied to one instance at the time.
>
>
> Of course I understand Xpath filters, other relevant examples here:
>
> - https://tools.ietf.org/html/rfc5277#section-5.2
> - https://tools.ietf.org/html/rfc8040#appendix-B.3.6
>
> The point was/is that supporting full-Xpath for stream-filters is easy, or
> rather doesn’t entail special consideration, as already each event is in
> memory for an Xpath analysis.  Thus, pointing to existing usage as evidence
> that unconstrained Xpath can be used everywhere, including for potentially
> very-large CF-lists, is not a given, as it would be performance-limiting to
> bring every list-entry into application-level memory in order to use an
> Xpath library.  In this case, it is very much desired to push all the
> querying logic to the DB-backend.
>
>
> But when it comes to CF data, doing a full table scan is impractical
> and index-backed queries can make the difference between the client
> waiting seconds vs minutes.  This is why I was initially trying to
> focus on just solving the CF problem, where the pain is acute.
>
>
> Well, it is less likely that an implementation uses a feature-rich (in
> terms of query language) DB to store all its state, than its config.
> So it would probably be easier to implement this using "an index" for
> config than for config false...
>
>
> Really?  ...my experience is seems opposite.
>
> A number of server implementations I’m aware of persist the entire
> configuration as a flat-file (e.g., pretty-print XML or JSON).  They bring
> it into memory and act on it using a feature-rich library (e.g.,
> Yangson).  In my view, this approach hits the low-bar in terms of what
> constitutes a “database”.
>
> On the flip side, handling large CF-lists (e.g., logs) is where servers
> pretty much have to use a real database.  An RDB (relational DB) is common,
> but a TSDB (time-series DB)
> <https://en.wikipedia.org/wiki/Time_series_database> is better suited.
> [HL] A user case for time series data (e.g., logs), customers want to get
> the data in between Time A and Time B. Do we use timestamp filter here?
>


> Perhaps we’re saying the same thing (yes, config APIs tends to have better
> query expressions)…it might just be your last line regarding “index” that’s
> throwing me (as clearly the DBs storing large CF-lists are having indexes).
>
>
> That said, I propose we do “both”.  <snip/>
>
>
> I agree.
>
>
> Okay, we’ll shoot for a two-pronged approach.
>
>
>
> Separately, for the RC-list draft only, the plan is to declare “list” and
> “leaf-list” nodes (not their entries) as data resource targets.  We only
> need to do this for GET, but I assume that it's better to cover all the
> HTTP operations - thoughts?
> [HL] Data resources should not be bound to some specific operation.
> Correct me, if my understanding is wrong.
>
> K.
> _______________________________________________
> netconf mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/netconf
>