Re: [netconf] restconf collections

[Martin Björklund <[e-mail]>]

Hi,

Kent Watsen <[e-mail]> wrote:
> Hi Martin,
> 
> >> A couple messages I sent yesterday alluded to defining an extension
> >> statement that could be added to nodes in the YANG modules.
> >> 
> >> Now I’m thinking that one-size doesn’t necessarily fit all, and
> >> something like a server-specific “node-tag” would be more flexible.
> >> (https://tools.ietf.org/html/draft-tao-netmod-yang-node-tags
> >> <https://tools.ietf.org/html/draft-tao-netmod-yang-node-tags>)
> >> 
> >> Maybe the answer is somewhere in-between, i.e., an extension that acts
> >> like a “hint” and a node-tag indicating if the server delivered it.
> >> Bad news w/ an extension is that legacy modules would need to be
> >> revved or augmented.
> >> 
> >> One thing, though, filtering on non-indexed nodes is guaranteed to
> >> have slow performance, thus it seems that we need to enable indexing
> >> for more than just a list’s keys.
> > 
> > I disagree with both these statements.  This is an implementation
> > issue.
> 
> Which two statements?

1. filtering on non-indexed nodes is guaranteed to have slow
   performance

2. it seems that we need to enable indexing  for more than just a
   list’s keys

> And how do you come to that conclusion, unless
> you’re suggested that all queries from particular server are equally
> slow?  ;)

I'm suggesting that this is highly implementation dependent.

> >> My conjecture is that, once we have the indexes for filtering (e),
> >> they can be used for sorting (d).  At least, that’s the way it works
> >> with some DB-backends I’m familiar with...
> > 
> > This is one implementation strategy.  We should not design the
> > standard around one implementation.
> 
> Please elucidate how this isn’t the case.
> 
> While a true statement, it’s an unfair implication, as pretty much
> *everything* I’ve written on this thread has been about trying to
> enable servers with various DB-backends to express which parts of the
> solution they can support.  Apparently some backends don’t support
> “sorts” while others do; some backends can support full Xpath while
> other can’t, etc.  All this means to me is that we need to define the
> necessary features and/or node-tags or whatever to enable each to
> advertise what it can do best, or at all.
> 
> If the WG wants the lowest-common denominator, we will likely end up
> with just ‘a', ‘b', and a heavily redacted Xpath for ‘e’.  I’m hoping
> that we can do better than that, but I’m also okay with leaving it to
> each solution can then create proprietary extensions/node-tags, if
> that is the WG consensus.

I would prefer an unconstrained filter, with the XPath 1.0 that is
already used in YANG.  It would be ok if support for such a filter was
optional (a capability).  My point is that even if the backend doesn't
support filtering on a particular node, filtering in the server will
be much more efficient than filtering in the client.  Hence filtering
shouldn't be constrained to certain nodes.  It also makes the solution
much easier to use for clients.


> > I agree that this can be useful.  But as has been pointed out earlier
> > in this thread, sorting implies a stateful solution which will consume
> > more resources and probably comes with some additional complexity.
> 
> True, except I think the augment was in *favor* of doing that, at
> least in some cases.
> 
> Also note that the “more resources” augment needs to be amortized over
> the number of queries.  There must be a break-even point whereby
> preparing a (resource-intensive) snapshot actually uses less resources
> over time, after it's been queried a certain number of times, than a
> server doing a fresh (lower-resource) query for each request.

To be clear, I think such a solution would be useful.

> > At Cisco/tail-f we implemented such a solution; a more generic "query"
> > mechanism, for both NETCONF and RESTCONF.  Perhaps Per A can share
> > some details.
> 
> That would be helpful.  The analysis can only take into account that
> which is known.  I know the Tail-f products used a custom database,
> but few know anything about it.
> 
> Creating a custom database is not easy.

Right, but note that this filtering can and should be applied to all
nodes, also operational state nodes that isn't stored in any database
at all, imo.


/martin


> I know, as I personally wrote
> a very-fast logging system, complete with indices, for a logging
> database taking in logs at the rate of 50,000 logs per second, with
> the total number of logs retained measured in the billions.
> 
> I personally do not wish for a *basic* server to have to go through
> that pain, hence the interest in supporting common DB-backends (NoSQL,
> SQL, etc.), while also enabling systems with special abilities to
> present that which can.
> 
> K.
> 
> 
> 
>