Re: [netconf] restconf collections

[Andy Bierman <[e-mail]>]

On Sat, Sep 26, 2020 at 11:57 AM Kent Watsen <[e-mail]> wrote:

>
> The <get-bulk> operation is not stateful.
>  http://www.netconfcentral.org/modulereport/yumaworks-getbulk
>
>
> This RPC is effectively 1-1 with RC query params approach being
> discussed.  I like it.
>
> The “count” param is like “limit”, and “list-test” is like a “where”.
>
> One difference is that <get-bulk> uses "last-keys” instead of an integral
> “offset”.   Any thoughts about the pros and cons?  I imagine performance
> depends on the backend database used.  For instance, SQL supports “offset”
> so, to support "last-keys” instead would entail a first query to determine
> the offset for a key.
>
>

It would be unlikely that a distributed server (e.g. data can come from
linecards)
that it is efficient to find the Nth entry.

Using a count approach is obviously vulnerable to additions and deletions
to the dataset.
If the count keeps changing then the relative Nth entry also keeps changing.
This causes duplicates or missed entries in the paginated data.  Using key
values
instead of a count this source of inaccuracy is eliminated.  Both approaches
are vulnerable to "moved" entries (e.g. user-ordered list entry changes
position).
This can happen in theory but (1) user-ordered config=false data is very
rare,
(2) the "move" operation for user-ordered data is very rarely used and (3)
an entry would have to be moved just at the time the walk was in progress.



> Missing is a “sort”.  That said, of the four params mentioned (limit,
> offset, where, sort), it might be least important.  At least, a general
> sort is…and then there is the nasty issue of sorting a user-sorted list.  A
> likely must-have fallback would be “direction” so, e.g., queries return the
> most-recent (not the oldest) entires in a time-series list (e.g., a log).
> A “direction” parameter would also work well on a user-sorted list.
>


sort is much more expensive to implement correctly and makes more
sense when a stateful snapshot approach is used. Sorting each chunk
vs. the entire data-set is not that useful.



>
>
> There are 3 completely different approaches:
>
>   1) stateless
>   2) stateful
>   3) stateful snapshot
>
> How to efficiently iterate through data that is changing constantly?
> This problem is harder than it looks.
>
>
> Agreed.  I recommend we NOT try to solve that problem.
>
>

But we are solving that problem to some degree.
Operational state is changing constantly and iterating through a list
vs. a single GET usually makes the problem worse.



> K.
>
>
Andy