Re: [dsii] Potential IETF Work Items

[Ted Hardie <[e-mail]>]

On Wed, Aug 15, 2012 at 7:10 AM, Scott Brim <[e-mail]> wrote:
> The problems are large but I also wonder what part of this is in IETF scope.
> I asked about this in the halls after the BOF.  If this work takes place in
> the IETF I will be very involved, but IETF expertise is primarily in
> protocol engineering, and I don't see much need for that here.

Well, one way to look at the IETF is as a set of structures the enable
whoever is interested in a specific technical topic to come together
under known rules; so if the big data community does the work here,
that community is part of the IETF and thus does have the related
expertise.  One of advantages of participation as the key identifier,
rather than membership, is that we have that fluidity.

But beyond that, there is a lot of IETF-relevant protocol work to be
done in this space: in  network discovery, online access methods,and
in access control.  We didn't focus on that part of the work in the
BoF because it generally needs to come after the core identifier
interoperability work has gotten a mental framework built.  In many
ways, though, that's when real cross-fertilization with existing IETF
work will get going.   While you could argue that that is after that
core interoperability story is ready that it should come to the IETF,
I think we all benefit if it gets early exposure.

To take one example, a simple way to create interoperability in a
system like this is to have a layer of indirection which creates a new
identifier in a distributed global namespace by hashing the existing
identifier with other metadata so as to create a new globally unique
name within the new namespace.  You go to the new namespace manager to
retrieve the mapping and there you go, right? Of course the access
control methods on such a scheme are three kinds of nightmare and the
new namespace manager needs a global caching scheme to prevent the
index from being unwieldy, but that creates a discovery/redirection
scheme that looked so easy to avoid with a single, global, trusted
namespace manager.

If you know what a particular interoperability method will end
implying in terms of related network infrastructure, access methods,
and access control, you avoid that simplistic method of creating
interoperability via single indirection (or at least you know what
you're getting into).  Giving the Big Data folks a home here helps
makes sure that those issues get surfaced early, and that may be a
benefit to all of us.

Just my two cents,

Ted Hardie