Re: [icnrg] icnrg Digest, Vol 101, Issue 6

[Ken Calvert <[e-mail]>]

OK, I've read through all the inputs except DaveO's latest "new thread" message.  This is the first of a couple of messages responding to particular things.

A comment on one of Christian's points (and a related point by others) on inlining:

> packet := length-limited data object (NDN, CCN)
> 
>         I assume that a data object has type information such that
>         "real blob" can be distinguished from "manifest packet". Having
>         a hash name and retrieving the data object will tell us what
>         kind of packet it is. Anything else (e.g., tagging hash
>         pointers whether they are pointing to real blobs or a manifest
>         pkt is a claim and thus unreliable. It's fine to keep such
>         tags as a decoration but these are not needed to implement
>         the virtual blob abstraction.
> 
> r_pkt := packet with TLV for "real blob"
> m_pkt := packet with TLV for "manifest"
> 
> r_pkt.payload = any                              # the "real blob"
> m_pkt.payload = manifest_spec
> 
> manifest_spec := deco + cont_seq                 # two fields, must fit in one packet
> deco          := None | hashval | inlined
> cont_seq      := sequence of (hashval | inlined)

Yikes - you lost me here.  I am interpreting "inlined" to mean "application data contained directly *in* the manifest structure" (analogous to inode inlining). So here you mean the sequence is heterogeneous, and I have to figure out if each element in the cont_seq is a 32/64-byte hash or some amount of real data?  I don't like this.  I have the same reaction to inlining "r_blob" data in a manifest (with no pointers).  Again, I apologize if I've misunderstood.  (I think Marc made the same point in a later message.)

My personal view is that, unlike iNodes, which you MUST go through to get access to the r_blob of a file (and therefore inlining is a reasonable shortcut), manifests are an OPTIONAL overhead amortization mechanism.  If you don't need to break an object into chunks, you can retrieve it directly.  If editing or whatever results in less a chunk size smaller than a hash pointer, well, the additional overhead of retrieving that small chunk is just overhead.  And it is traded off against the cost of determining, for EVERY element of EVERY cont_seq, whether it is a hash or inlined data.

(Note: As Dave mentions in a later note, we would need a way to represent "holes" in a virtual blob, if pointers are un-annotated and homogeneous - something like an all-zero pointer value could work for that. If each pointer has its own TLV [oog], you could have a "hole type", I guess.) 

Ken