RE: [Rmt] Should FLUTE File Aggregation I-D be a working group item?

[[e-mail]]

Hi All

If you haven't read the document yet there are two things as play which
may be confused:
Group of files into:
- logical groups OR physically one transmission object
Extened-FLUTE-FDT signalling of files in physically aggregated objects:
- using FLUTE normally (e.g. TOI 0) OR embedding the aggregated files'
FDT in the object

Magnus, correct me if I misunderstood - you're talking about the second
issue. (i.e. you are questioning the placement of the file transport
data - but are happy with the files going into multipart MIME?)

Yes, unfortunately it will not be backwards compatible 
>with the normal FDT, but it could have a number of benefit:

Well you could give the extended FDT a new MIME type and put it on
"other than TOI=0" and then it's backwards compatible. Also, you can
extend the FDT with "aggregated file" elements (instead of using the
"file" element and so you'd be able to change semantics for aggregated
files without gamaging "not-enhnaced" FLUTE receiveres. Is there
something I missed about backward compatability onthis?

>- The possibility to externally before receiving the object 
>determine if all files already has been received or if some 
>has. This is quite important to minimize the connection time 
>when doing repeated transmission to repair/complete unfinished 
>sessions.

This seems to confirm it's about where to signal the file details of
objects. Doing this as an "index FDT" in the multipart MIME assumes that
the kind of use case mentioned is not significant. i.e. that it's highly
unlikely that clients will have all the data of an object before it gets
sent, and it's highly unlikely that objects with different fileURIs but
the same files will get sent. If these are not good assumptions,
extending the FDT is indeed a better method: but not without cost as
bloated FDTs start to require their own FEC and reliability techniques
and (unlike embedding in the aggregated object) there is no garuntee
that all FDT instances will be received even though all files of an
aggregated object are.

So do we need to allow both methods, or is there a way to limit to just
one? (To me it seems both, but with advisory text as to when each is
better and some recommendations).

>- Possibility to extract data from an partially decoded block. 
>With small files and large FEC blocks this becomes highly 
>interesting. Thus the data offset and length needs to be 
>available from external source. 
>Special data integrity mechanisms to make this reliable may 
>also need to be considered.

There are two partically reveived deocding aspects:
 - partially received AOs, but some fully received blocks which enable
extraction of some complete files (relatively simple)
 - partially revieced blocks with some fancy maths to revover parts of
them (i.e. some files) (this is what Magnus said - but except for well
ordered FEC, or null FEC, codes seems improbable)

For well ordered FEC codes, indeed prior knowledge of byte ranges of
files can be mapped to source symbols and the FEC parameters can tell
the client whether it can restore the relevant source symbols for all or
part of certain files (generally only all is useful, but for some
applications even files with corruption may be partially renderable -
e.g. corrupt images on a web page).

For the extreme-FEC-dudes, putting into nomral FLUTE TOIs, instead of
embedding into the multipart MIME, is much more robust for partial
recovery from incomplete received blocks.


Cheers, Rod.



>-----Original Message-----
>From: [e-mail] [mailto:[e-mail]] On 
>Behalf Of ext Magnus Westerlund
>Sent: 21 November, 2005 12:24
>To: Christoph Neumann
>Cc: Vincent Roca; [e-mail]
>Subject: Re: [Rmt] Should FLUTE File Aggregation I-D be a 
>working group item?
>
>Hi Christoph,
>
>See inline.
>
>Christoph Neumann wrote:
>> Dear all,
>>  To start the discussion on the object aggregation draft, I 
>wanted to 
>> point out some issues that I think are in favour of having 
>this as an WG item.
>>  Physical aggregation scheme is of great advantage to avoid 
>the coupon 
>> collector problem. In the case of a large number of small files, 
>> physical aggregation combined with FEC is the only way to 
>effeciently 
>> transport the content.
>> Therefore I think it is necessary to have some standardized solution 
>> for that, which could possibly be reused in several use cases with 
>> different FEC schemes...
>> The draft exaclty addresses this problem.
>
>But in the process it looses a number of properties that 
>individual files. I think there would be a number of benefit 
>to NOT use multi-part mime and instead use an FDT based 
>scheme. Yes, unfortunately it will not be backwards compatible 
>with the normal FDT, but it could have a number of benefit:
>
>- The possibility to externally before receiving the object 
>determine if all files already has been received or if some 
>has. This is quite important to minimize the connection time 
>when doing repeated transmission to repair/complete unfinished 
>sessions.
>
>- Possibility to extract data from an partially decoded block. 
>With small files and large FEC blocks this becomes highly 
>interesting. Thus the data offset and length needs to be 
>available from external source. 
>Special data integrity mechanisms to make this reliable may 
>also need to be considered.
>
>I think these comments becomes extra important when 
>considering battery powered wireless devices.
>
>
>>  I heard that some physical aggragation scheme has also been 
>defined in 3GPP
>> MBMS, but I'm not really following 3GPP activities and are 
>therefore not
>> aware of that. Maybe someone has an input here?
>>  Beside the coupon collector issue, some FEC schemes perform 
>badly with
>> small files, and aggregating them, seems the only solution 
>in this context
>> (as it has been mentioned on the slides during the rmt session).
>
>The aggregation mechanism defined in 3GPP is to aggregate several 
>different UDP datagram streams used in streaming. Please see
>http://www.ietf.org/internet-drafts/draft-watson-tsvwg-fec-sf-00.txt
>for the outline of that solution and which is an initial proposal for 
>the most likely to be FECFrame WG. The actual 3GPP 
>specification of this 
>is TS 26.346:
>http://www.3gpp.org/ftp/Specs/html-info/26346.htm
>Select the latest version.
>
>Cheers
>
>Magnus Westerlund
>
>Multimedia Technologies, Ericsson Research EAB/TVA/A
>----------------------------------------------------------------------
>Ericsson AB                | Phone [phone]
>Torshamsgatan 23           | Fax   [phone]
>S-164 80 Stockholm, Sweden | mailto: [e-mail]
>
>_______________________________________________
>Rmt mailing list
>[e-mail]
>https://www1.ietf.org/mailman/listinfo/rmt
>

_______________________________________________
Rmt mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/rmt