Re: [icnrg] Some comments on the CCNx Selectors draft (https://datatracker.ietf.org/doc/draft-mosko-icnrg-selectors/)

[[e-mail]]

Thanks David for creating a discussion around this!

Looking at discovery from a function invocation perspective certainly 
resonates well, of course. However, NFN focuses on producing immutable 
results out of immutable arguments, and still makes the most sense in 
such an environment. You target the discovery of potentially new 
content, such that the same query will return different results if asked 
again later, right? That leads me to this comment:

Your writeup did not mention time nor provenance, which I think are 
important in this discussion, another concept to mention would be "query 
expression".

With provenance I mean the state (context) where new content was 
published. This state acts as logical time and permits to "immutify" 
results. The exchange would be:

- fetch a repo's state, or time horizon, S
   (this is the only mutable result call needed and permits to bind to
   that state, instead of a server or service)

followed by one or more

- discover(repo=/ndn/ucla, horizon=S, selector=/unix/ls(/some/repo/path, "*.txt"))

This discovery call will return the same result even of the repo is 
replicated: any repo replica can execute the call in the specific 
context S, no need to pin your discovery to a specific server, and the 
result is fully cachable (if in clear).

A simple way to explain what a repo state S is, would be to point to the 
commit hash of a Git repo. A NDN repo would have to keep track of new 
content through commits, have a log of its actions. I think that the 
word "provenance" is appropriate here, namely how/where/when some 
content was added to the repo, or renamed, aliased etc.

Sure, manifests are good, but I think we can do better, and by better I 
mean repo-side filtering. Once results are immutable, we can have 
deterministic and stateless paging, cursors and other result mangling 
under the requestor's control:

The last 10 entries:
- /nfn/txt/tail(/ndn5/discover(horizon=S, selector=/unix/ls(/ccnx/some/prefix, "*.txt"), -10)
Just counting hits:
- /nfn/txt/wc(/ndn5/discover(horizon=S, selector=/unix/ls(/ccnx/some/prefix, "*.txt"))

I see your objection coming that general NFN is too heavy for minimal 
content discover, and I would agree. But what about a modest query 
_language_ for repo-side execution instead of exposing single RPCs that 
the caller has to chain in multiple RTs? We might pick a small set of 
result filters (selectors really), that are easy to implement in all 
repos, and permit to combine up to three filters in a query expression 
as an example. So no loops, no lambda... Filters could be:

   isDir(), newerThan(d), hasMagic(GIF), publishedBy(key), lexicoAfter(marker)

Then we add Python slicing as in [10:-5], or just classic head() and 
tail(), as well as AND, NOT, possibly OR.

Datalog could serve as a high-end starting point for such a discussion.

Best, c


On Sun, 2 Jun 2019, David R. Oran wrote:

> 
> All comments with <chair hat off>
> 
> Content discovery is important, so I’d like to see something adopted by the
> RG for formal progression to experimental.
> 
> I’d like to step back a bit though and ask if evolving the approach in CCNx
> 0.x and NDN is appropriate knowing what we’ve learned in the last few years.
> The current selectors draft
> (https://datatracker.ietf.org/doc/draft-mosko-icnrg-selectors/) basically
> fixes the original design mistake of forcing all forwarders to do discovery
> through prefix match on CS data. It does this by leveraging typed name
> components and using an encapsulation technique so that whatever object that
> matches the selectors can be returned as “native” - with its original name
> and signature.
> 
> On the other hand, it still has the following properties:
>
>  * 
>
>     returns only one object when multiple might match
>
>  * 
>
>     does not have a completely deterministic answer in the presence of
>     cache/repo/producer instance desynchronization (not clear this is
>     avoidable since we have a distributed system with at best eventual
>     consistency semantics)
>
>  * 
>
>     requires exclusions, which can make for big interests that in turn
>     causes fragmentation to be needed and complicates congestion control as
>     a consequence.
>
>  * 
>
>     seems to still permit cache exploration (since the selector interests
>     are not authenticated), which might be viewed as a privacy problem.
> 
> I don’t have a concrete proposal to make though. Here are a few thoughts
> about a possible alternative approach:
>
>  * 
>
>     Since we depend a lot more on Manifests now that the original NDN & CCNx
>     0.x designs, why not have the discovery return a Manifest instead of a
>     single encapsulated object?
>
>  * 
>
>     Since we have fast exact match and nameless objects along with now
>     permitting objects to have multiple names and hence multiple ways
>     objects can be discovered, perhaps we can exploit that to allow
>     different discovery patterns rather than smooshing everything into a
>     static/brittle set of selectors
>
>  * 
>
>     Some of the properties were driven by trying to make discovery cheap and
>     ubiquitous and hence forced into casting discovery as a direct single
>     Interest/Data exchange. Since we now know that pawing through a large CS
>     or Repo can be computational expensive (and even require a fair amount
>     of I/O), maybe this tradeoff isn’t attractive. We now have experience
>     with distributed method invocation (e.g. NFN and RICE) so maybe we do
>     discovery as an explicit remote method. This may solve a few problems:
>
>      +  We can do authentication and key exchange, so discovery operations
>         and results can be more private and cache exploration is no longer a
>         hazard
>      +  It might make it easier to build extensibility into the supported
>         discovery patterns as all of them might not need to be supported by
>         all discovery services. (Aside - it seems that a similar direction
>         is being taken in the fast-repo work for NDN, where a given
>         application can design patterns for the Repo to fetch the right
>         stuff for storage of that application’s data).
>      +  We can more easily manage the resources for expensive discovery
>         operations, and following the idea of using Manifests, make those
>         the result of the discovery computation.
>      +  By essentially “binding” to one of the discovery services/servers,
>         it may be possible to get a more internally-consistent set of
>         results.
> 
> Yes, this might make discovery a lot more “expensive” in terms of RTTs but
> in the end it might also make applications easier to write with patterns
> that look more like what the application considers “native” and the results
> as a Manifest easier to paw through and possibly iterate on.
> 
> Thoughts?
> 
> DaveO
> 
> 
>