Re: [netconf] restconf collections

[Kent Watsen <[e-mail]>]

Hi Martin,

>> A couple messages I sent yesterday alluded to defining an extension
>> statement that could be added to nodes in the YANG modules.
>> 
>> Now I’m thinking that one-size doesn’t necessarily fit all, and
>> something like a server-specific “node-tag” would be more flexible.
>> (https://tools.ietf.org/html/draft-tao-netmod-yang-node-tags
>> <https://tools.ietf.org/html/draft-tao-netmod-yang-node-tags>)
>> 
>> Maybe the answer is somewhere in-between, i.e., an extension that acts
>> like a “hint” and a node-tag indicating if the server delivered it.
>> Bad news w/ an extension is that legacy modules would need to be
>> revved or augmented.
>> 
>> One thing, though, filtering on non-indexed nodes is guaranteed to
>> have slow performance, thus it seems that we need to enable indexing
>> for more than just a list’s keys.
> 
> I disagree with both these statements.  This is an implementation
> issue.

Which two statements?   And how do you come to that conclusion, unless you’re suggested that all queries from particular server are equally slow?  ;)


>> My conjecture is that, once we have the indexes for filtering (e),
>> they can be used for sorting (d).  At least, that’s the way it works
>> with some DB-backends I’m familiar with...
> 
> This is one implementation strategy.  We should not design the
> standard around one implementation.

Please elucidate how this isn’t the case.

While a true statement, it’s an unfair implication, as pretty much *everything* I’ve written on this thread has been about trying to enable servers with various DB-backends to express which parts of the solution they can support.   Apparently some backends don’t support “sorts” while others do; some backends can support full Xpath while other can’t, etc.   All this means to me is that we need to define the necessary features and/or node-tags or whatever to enable each to advertise what it can do best, or at all.

If the WG wants the lowest-common denominator, we will likely end up with just ‘a', ‘b', and a heavily redacted Xpath for ‘e’.  I’m hoping that we can do better than that, but I’m also okay with leaving it to each solution can then create proprietary extensions/node-tags, if that is the WG consensus.


> I agree that this can be useful.  But as has been pointed out earlier
> in this thread, sorting implies a stateful solution which will consume
> more resources and probably comes with some additional complexity.

True, except I think the augment was in *favor* of doing that, at least in some cases.

Also note that the “more resources” augment needs to be amortized over the number of queries.  There must be a break-even point whereby preparing a (resource-intensive) snapshot actually uses less resources over time, after it's been queried a certain number of times, than a server doing a fresh (lower-resource) query for each request.


> At Cisco/tail-f we implemented such a solution; a more generic "query"
> mechanism, for both NETCONF and RESTCONF.  Perhaps Per A can share
> some details.

That would be helpful.  The analysis can only take into account that which is known.  I know the Tail-f products used a custom database, but few know anything about it.  

Creating a custom database is not easy.  I know, as I personally wrote a very-fast logging system, complete with indices, for a logging database taking in logs at the rate of 50,000 logs per second, with the total number of logs retained measured in the billions.  

I personally do not wish for a *basic* server to have to go through that pain, hence the interest in supporting common DB-backends (NoSQL, SQL, etc.), while also enabling systems with special abilities to present that which can.

K.