Re: [AVT] AVC NALUs RTP packetisation

[Nikola Sprljan <[e-mail]>]

Ye-Kui,

Thank you for replying - I see now that a solution would be to buffer up
the RTP packets and keep them in a MANE until they are not needed any
more - the decision on forwarding/dropping is postponed until more
information on this packet becomes available. I was actually trying to
avoid solutions based on buffering at the RTP level as my intention was
to perform adaptation solely on a per-RTP packet basis, with all SVC
layers transmitted in a single RTP session.

One could argue that buffering is required at least for the case when
RTP packets are carrying fragmentation units, so they are not being
forwarded until it is made sure that all the fragments originating from
the same NALU have been received (I am not considering FGS here). But if
we are already at this level of complexity, and since a MANE can be made
aware of inter-layer dependencies, then why not observing decoding
dependencies between all received packets (i.e. their decoding
dependency to the lost packets) when deciding which ones can be safely
dropped? What I am trying say is - since MANEs will potentially have to
deal with lots of incoming streams, I would prefer having an option of
just forwarding or dropping the required packets, without employing any
"smart" repacketisation / thinning / retransmission techniques (I don't
consider parsing and reassembling the aggregation units to be "smart"),
and let the next node in the transmission chain deals with packet
losses. This could be seen as some kind of a "light" mode of MANE's
operation.

Basically, if one wants to avoid buffering packets in a MANE, the
following has to be ensured:
-fragmentation is not used at the RTP level;
-aggregation units are used for packetisation of AVC NALUs, along with
NALUs containing SVC-specific parameters in the same RTP packet.
However, since there always exists a possibility that I will encounter
some pre-encoded content that has slices larger than 64kB, as a
consequence I need to support FUs and implement buffering in the MANE.

Now, I know this has been discussed many times, but I am still of an
opinion that SSRC multiplexing could be helpful - especially in this
"no-buffer" scenario, as it would enable immediate forwarding/dropping
of RTP packets that would otherwise need to be buffered. Perhaps I am
just leaning towards having something close to a "Magic Device from
Heaven", but I also have a feeling that this could be useful for others
as well - e.g. it's still mentioned as a possibility in the current
version of draft-lennox-mmusic-sdp-source-attributes.

I guess what I'd like to ask this time is - is SSRC multiplexing gone
for good, or there's still a chance it could be reconsidered?

Best regards,

Nikola

[e-mail] wrote:
> Hi Nikola,
> 
> Thanks for your comments. I basically agree with those that I did not
> response in below. Please see inline. 
> 
>> -----Original Message-----
>> From: ext Nikola Sprljan [mailto:[e-mail]] 
>> Sent: Wednesday, March 28, 2007 1:38 PM
>> To: Wang Ye-Kui (Nokia-NRC/Tampere)
>> Cc: [e-mail]; [e-mail]
>> Subject: Re: [AVT] AVC NALUs RTP packetisation
>>
>> Ye-Kui,
>>
>> Thank you for your reply. Please find my comments inline.
>>
>> [e-mail] wrote:
>>> In SVC context, it is also possible to use FU's as Randell 
>> said below, 
>>> although it is not possible to include the PACSI or suffix or prefix 
>>> (newly introduced to the SVC draft in January) NALU in packets 
>>> containing FU's.
>> I see now that I omitted something that made my question 
>> imprecise/incorrect - when I said "AVC NALUs cannot be 
>> fragmented at the application layer", I actually wanted to say 
>> "AVC NALUs *aggregated* in a packet together with a NALU 
>> conveying the SVC-specific parameters cannot be fragmented 
>> above the UDP/IP level". It just seems to me that AVC NALUs 
>> that are not in aggregation units are not very useful in SVC 
>> context, and that this implies that fragmentation at the RTP 
>> level is never going to be used for AVC NALUs in SVC context. 
>> Is that correct?
>>
> 
> AVC NALUs are useful and actually important in SVC context. Note that
> "AVC NALUs in SVC context" does not mean that each RTP packet containing
> an AVC NALU or part thereof must contain the SVC-specific parameters.
> See further discussions of the three cases in below.
> 
>>> Currently there is no plan to support containing of 
>> additional NALU in 
>>> FU packets. The main reason is still that there is no compelling use 
>>> case requiring the support.
>> Since I think there is sometimes a confusion with what is done 
>> at which communication level, I will summarise briefly my 
>> understanding here:
>> *Application level* - deals with slices; if FMO is used, SVC 
>> encoder sets the maximum byte length limit for slices.
> 
> The encoder can always sets the maximum byte size limit for slices,
> regardless of whether FMO is in use. 
> 
>> *RTP level* - deals with NALUs; NALU packetisation modes are 
>> used, i.e., selection of fragmentation and aggregation modes. 
>> Fragmentation modes must be used for slices larger than 64kB 
>> to divide them into segments
>> (FUs) that will fit into packets sent over IPv4.
>> *UDP/IP (or any lower than RTP) level* - deals with RTP 
>> packets; fragmentation of RTP packets that exceed the MTU limit.
>>
>> Also, maybe it is worth noting that although I say I need 
>> "SVC-specific parameters for AVC NALUs", actually I only need 
>> the information on temporal level - to be able to decide 
>> whether to forward a packet containing an AVC NALU to a 
>> particular client. And using a PACSI or a suffix NALU seems to 
>> me is the only way to achieve that.
>>
>> Since I consider pre-encoded content here, the cases for 
>> packetisation of AVC NALUs in SVC context are then the following:
>>
>> 1. aggregated AVC NALU + SVC-specific parameters <=MTU, 
>> including overheads -> Can be put into an aggregation NALU 
>> without any fragmentation.
>> 2. aggregated AVC NALU + SVC-specific parameters >MTU and 
>> <=64kB, including overheads -> Can be put into an aggregation 
>> NALU, with fragmentation done on UDP/IP level.
>> 3. aggregated AVC NALU + SVC-specific parameters >64kB, 
>> including overheads -> Since this has to be fragmented at the 
>> RTP level, SVC parameters cannot be sent and therefore this 
>> pre-encoded bit-stream is not supported for sending over IPv4.
>>
> 
> Cases 1 and 2 are correct. For case 3, the SVC specific parameters (e.g.
> in the prefix NAL unit) can be sent in their own RTP packet, and the big
> AVC NALU, fragmented in RTP level, is sent in the subsequent RTP
> packets. Yes, there will be no information of temporal_level or
> priority_id in these packets. If a MANE would like to do something
> intelligent to these packets, it can buffer the previous packet and
> obtain the required information.
> 
>> I guess when you say "there is no compelling use case", that 
>> actually means that the case 3. is not likely to happen and is 
>> thus is not supported? If that is so, should it be explicitly 
>> stated in the packetisation rules?
>>
> 
> This is part of the reason. Another part is that a MANE can buffer up to
> a couple of RTP packets to get required information for some
> not-very-common cases. The reason why we did not put such things to the
> packetization rules was, we think handling of case 3 above, for example,
> seems to be obvious to people who would implement RTP packetizer. But I
> think it would not hurt if we do. So text proposals are welcome. 
> 
> BR, YK
> 
>>> In case one NALU is larger than MTU size but smaller than 64kB, the 
>>> packetizer can use FU's to avoid lower network layer fragmentation.
>> Only for SVC NALUs, since for AVC NALUs I must use aggregation units.
>>
>>> Alternatively, aggregation packet containing the SVC specific 
>>> parameters can be used. In this case, after fragmentation is done at 
>>> lower network layer, the SVC specific parameters would be present in 
>>> the first fragmentation, which is most important among all the 
>>> fragments originated from the aggregation packet.
>> I think this covers only case 2.
>>
>> Best regards,
>>
>> Nikola
[cut]

_______________________________________________
Audio/Video Transport Working Group
[e-mail]
https://www1.ietf.org/mailman/listinfo/avt