Re: [icnrg] icnrg Digest, Vol 101, Issue 6

On 8 Aug 2020, at 15:18, Ken Calvert wrote:

> OK, I've read through all the inputs except DaveO's latest "new 
> thread" message.  This is the first of a couple of messages responding 
> to particular things.
>
> A comment on one of Christian's points (and a related point by others) 
> on inlining:
>
>> packet := length-limited data object (NDN, CCN)
>>
>>         I assume that a data object has type information such that
>>         "real blob" can be distinguished from "manifest packet". 
>> Having
>>         a hash name and retrieving the data object will tell us what
>>         kind of packet it is. Anything else (e.g., tagging hash
>>         pointers whether they are pointing to real blobs or a 
>> manifest
>>         pkt is a claim and thus unreliable. It's fine to keep such
>>         tags as a decoration but these are not needed to implement
>>         the virtual blob abstraction.
>>
>> r_pkt := packet with TLV for "real blob"
>> m_pkt := packet with TLV for "manifest"
>>
>> r_pkt.payload = any                              # the "real blob"
>> m_pkt.payload = manifest_spec
>>
>> manifest_spec := deco + cont_seq                 # two fields, must 
>> fit in one packet
>> deco          := None | hashval | inlined
>> cont_seq      := sequence of (hashval | inlined)
>
> Yikes - you lost me here.  I am interpreting "inlined" to mean 
> "application data contained directly *in* the manifest structure" 
> (analogous to inode inlining). So here you mean the sequence is 
> heterogeneous, and I have to figure out if each element in the 
> cont_seq is a 32/64-byte hash or some amount of real data?  I don't 
> like this.  I have the same reaction to inlining "r_blob" data in a 
> manifest (with no pointers).  Again, I apologize if I've 
> misunderstood.  (I think Marc made the same point in a later message.)
>
I agree. For me the purpose of inlining is to cover the case where the 
combination of the manifest data structures and the data it would point 
to is smaller than a single ICN packet MTU. Anything else doesn’t save 
RTTs and just introduces complexity.

> My personal view is that, unlike iNodes, which you MUST go through to 
> get access to the r_blob of a file (and therefore inlining is a 
> reasonable shortcut), manifests are an OPTIONAL overhead amortization 
> mechanism.

So, this would be the case in NDN that starts from having every 
primitive object have a full name and a signature. CCNx however has a 
strong bias toward individual pieces of application data just be named 
with hashes (what are called “nameless objects” to the dismay of 
some of us). in CCNx Manifests are in fact necessary in almost all cases 
to be able to obtain the names used route interests.

> If you don't need to break an object into chunks, you can retrieve it
> directly.

Unless it is only hash-named as in CCNx and you don’t already know the 
Name prefix to attach to the Interest.

> If editing or whatever results in less a chunk size smaller than a 
> hash pointer, well, the
> additional overhead of retrieving that small chunk is just overhead.  
> And it is traded off against the cost of determining, for EVERY 
> element of EVERY cont_seq, whether it is a hash or inlined data.
>
Yes.

> (Note: As Dave mentions in a later note, we would need a way to 
> represent "holes" in a virtual blob, if pointers are un-annotated and 
> homogeneous - something like an all-zero pointer value could work for 
> that. If each pointer has its own TLV [oog], you could have a "hole 
> type", I guess.)

There’s a complexity/space tradeoff obviously, since if you put the 
offset into the array element for each hash pointer you don’t get the 
expansion that would occur if an object is “Holy” (i.e. has big 
holes). I suspect at the end of the day things will work out better if 
the Manifest actually attaches the offset to the pointer in the hash 
array, since that also provides the flexibility of not filling every 
hash pointer to the MTU.

>
> Ken
> _______________________________________________
> icnrg mailing list
> icnrg@irtf.org
> https://www.irtf.org/mailman/listinfo/icnrg

DaveO