Re: [icnrg] Some comments on the CCNx Selectors draft (https://datatracker.ietf.org/doc/draft-mosko-icnrg-selectors/)

["Mosko, Marc <[e-mail]>" <[e-mail]>]

Dave & Christian,

Thanks for starting to tackle this problem.

In regard to the Selectors draft, I do not want to spend too much time defending it.  As said before, it was a proof-of-concept that one can do opt-in discovery using similar mechanisms to CCNx 0.x and older NDN.  Doing 1 by 1 query response and returning the whole content object each time is not, I think, the right approach.

I'm not going to dive in to a language or NFN vs keywords, etc., at this time.  I think we should first describe what it's supposed to do.

I think it is important that each node be able to control the extent to which it participates in discovery, including the encryption of queries and responses.  Possibly also authenticated discovery with various access control.

Here are some examples of the types of questions I think could be issued:
1) What is the latest version of a prefix?
2) What publishers have published under a name? (i.e., are there multiple keyids for the same name?)
3) regex style matching under a prefix.
4) Where can I get everything in a manifest
5) Which pieces of a manifest do you have (branches, leafs, other ways of answering)?
6) POSIX-like 'ls' functions
7) Discovery sources vs discovery content (e.g. where can I get something versus what can I get).  I don't mean to make this host-based networking, but I might want to get something from my cell provider versus a cache elsewhere or from a privacy-based cache operator.

Like DNS, there we have unauthoritative content stores (CS) and repositories; and authoritative repositories and publishers.  There should, I think, be a way to distinguish between the two and maybe to select whom one queries.

I think it is important to be able to break a response into multiple transactions, such as for scale and performance.  And those should be able to be consistent, such as a cursor.  This could be achieved by talking to only one entity using local context or, as Christian pointed out, using some shared state and talking with an enclave.

One could return a Manifest-like thing in response, but I think one wants to include possibly more information:
- A link (name, keyid, hash) to objects (e.g., "files")
- A sub-namespace (e.g., "directories")
- Information on caching (e.g. the ExpiryTime or remaining RCT)
- Information about inter-linking (manifest relationships)

I think using CCNxKE-like setup for private sessions for a query (and possibly subsequent data transfer) is a good thing.  For a properly configured enclave, one CCNxKE setup could allow for enclave-wide queries and responses.

Marc

________________________________________
From: icnrg <[e-mail]> on behalf of [e-mail] <[e-mail]>
Sent: Monday, June 3, 2019 11:17 AM
To: David R. Oran
Cc: Mosko, Marc <[e-mail]>; ICNRG
Subject: Re: [icnrg] Some comments on the CCNx Selectors draft (https://datatracker.ietf.org/doc/draft-mosko-icnrg-selectors/)

Thanks David for creating a discussion around this!

Looking at discovery from a function invocation perspective certainly
resonates well, of course. However, NFN focuses on producing immutable
results out of immutable arguments, and still makes the most sense in
such an environment. You target the discovery of potentially new
content, such that the same query will return different results if asked
again later, right? That leads me to this comment:

Your writeup did not mention time nor provenance, which I think are
important in this discussion, another concept to mention would be "query
expression".

With provenance I mean the state (context) where new content was
published. This state acts as logical time and permits to "immutify"
results. The exchange would be:

- fetch a repo's state, or time horizon, S
   (this is the only mutable result call needed and permits to bind to
   that state, instead of a server or service)

followed by one or more

- discover(repo=/ndn/ucla, horizon=S, selector=/unix/ls(/some/repo/path, "*.txt"))

This discovery call will return the same result even of the repo is
replicated: any repo replica can execute the call in the specific
context S, no need to pin your discovery to a specific server, and the
result is fully cachable (if in clear).

A simple way to explain what a repo state S is, would be to point to the
commit hash of a Git repo. A NDN repo would have to keep track of new
content through commits, have a log of its actions. I think that the
word "provenance" is appropriate here, namely how/where/when some
content was added to the repo, or renamed, aliased etc.

Sure, manifests are good, but I think we can do better, and by better I
mean repo-side filtering. Once results are immutable, we can have
deterministic and stateless paging, cursors and other result mangling
under the requestor's control:

The last 10 entries:
- /nfn/txt/tail(/ndn5/discover(horizon=S, selector=/unix/ls(/ccnx/some/prefix, "*.txt"), -10)
Just counting hits:
- /nfn/txt/wc(/ndn5/discover(horizon=S, selector=/unix/ls(/ccnx/some/prefix, "*.txt"))

I see your objection coming that general NFN is too heavy for minimal
content discover, and I would agree. But what about a modest query
_language_ for repo-side execution instead of exposing single RPCs that
the caller has to chain in multiple RTs? We might pick a small set of
result filters (selectors really), that are easy to implement in all
repos, and permit to combine up to three filters in a query expression
as an example. So no loops, no lambda... Filters could be:

   isDir(), newerThan(d), hasMagic(GIF), publishedBy(key), lexicoAfter(marker)

Then we add Python slicing as in [10:-5], or just classic head() and
tail(), as well as AND, NOT, possibly OR.

Datalog could serve as a high-end starting point for such a discussion.

Best, c


On Sun, 2 Jun 2019, David R. Oran wrote:

>
> All comments with <chair hat off>
>
> Content discovery is important, so I’d like to see something adopted by the
> RG for formal progression to experimental.
>
> I’d like to step back a bit though and ask if evolving the approach in CCNx
> 0.x and NDN is appropriate knowing what we’ve learned in the last few years.
> The current selectors draft
> (https://datatracker.ietf.org/doc/draft-mosko-icnrg-selectors/) basically
> fixes the original design mistake of forcing all forwarders to do discovery
> through prefix match on CS data. It does this by leveraging typed name
> components and using an encapsulation technique so that whatever object that
> matches the selectors can be returned as “native” - with its original name
> and signature.
>
> On the other hand, it still has the following properties:
>
>  *
>
>     returns only one object when multiple might match
>
>  *
>
>     does not have a completely deterministic answer in the presence of
>     cache/repo/producer instance desynchronization (not clear this is
>     avoidable since we have a distributed system with at best eventual
>     consistency semantics)
>
>  *
>
>     requires exclusions, which can make for big interests that in turn
>     causes fragmentation to be needed and complicates congestion control as
>     a consequence.
>
>  *
>
>     seems to still permit cache exploration (since the selector interests
>     are not authenticated), which might be viewed as a privacy problem.
>
> I don’t have a concrete proposal to make though. Here are a few thoughts
> about a possible alternative approach:
>
>  *
>
>     Since we depend a lot more on Manifests now that the original NDN & CCNx
>     0.x designs, why not have the discovery return a Manifest instead of a
>     single encapsulated object?
>
>  *
>
>     Since we have fast exact match and nameless objects along with now
>     permitting objects to have multiple names and hence multiple ways
>     objects can be discovered, perhaps we can exploit that to allow
>     different discovery patterns rather than smooshing everything into a
>     static/brittle set of selectors
>
>  *
>
>     Some of the properties were driven by trying to make discovery cheap and
>     ubiquitous and hence forced into casting discovery as a direct single
>     Interest/Data exchange. Since we now know that pawing through a large CS
>     or Repo can be computational expensive (and even require a fair amount
>     of I/O), maybe this tradeoff isn’t attractive. We now have experience
>     with distributed method invocation (e.g. NFN and RICE) so maybe we do
>     discovery as an explicit remote method. This may solve a few problems:
>
>      +  We can do authentication and key exchange, so discovery operations
>         and results can be more private and cache exploration is no longer a
>         hazard
>      +  It might make it easier to build extensibility into the supported
>         discovery patterns as all of them might not need to be supported by
>         all discovery services. (Aside - it seems that a similar direction
>         is being taken in the fast-repo work for NDN, where a given
>         application can design patterns for the Repo to fetch the right
>         stuff for storage of that application’s data).
>      +  We can more easily manage the resources for expensive discovery
>         operations, and following the idea of using Manifests, make those
>         the result of the discovery computation.
>      +  By essentially “binding” to one of the discovery services/servers,
>         it may be possible to get a more internally-consistent set of
>         results.
>
> Yes, this might make discovery a lot more “expensive” in terms of RTTs but
> in the end it might also make applications easier to write with patterns
> that look more like what the application considers “native” and the results
> as a Manifest easier to paw through and possibly iterate on.
>
> Thoughts?
>
> DaveO
>
>
>