RE: [AVT] RTP streaming and adaptation to AVC of an SVC temporal scalable bitstream - Packet loss

[<[e-mail]>]

Daniele,

My understanding is that the NAL unit header of a NAL unit contained in a single NAL unit packet is considered as the payload header. However, for an aggregation packet, the "NAL unit header" of the aggregation packet itself is considered as the payload header, but not the NAL unit headers of individual NAL units contained in the aggregation packet. Otherwise we have to understand that the payload header is interleaved with payload data. 

The SEI message based solution can be used together with normal RTP sequence number to detect loss of a part of a picture. So is true also for the PACSI + TL0PicIdx solution mentioned by Thomas. 
 
However, you are right that a temporal_id specific RTP sequence number would be smoother as parsing of sub-sequence information SEI message is not required. But to me the complexity reduction is not much. Furthermore, having such layer specific RTP sequence numbers would make the payload structure pretty ugly, because general SVC use cases with other scalability dimensions, each combination of temporal_id (3 bits), dependency_id (3 bits) and quality_id (5 bits) then needs to have their specific RTP sequence numbers. The total number is up to 8x8x32. 
 
BR, YK

________________________________

	From: ext daniele renzi (bsoft) [mailto:[e-mail]] 
	Sent: Monday, August 20, 2007 6:48 PM
	To: Wang Ye-Kui (Nokia-NRC/Tampere)
	Cc: [e-mail]; [e-mail]; [e-mail]
	Subject: Re: [AVT] RTP streaming and adaptation to AVC of an SVC temporal scalable bitstream - Packet loss
	
	
	Dear Ye Kui,
	 
	sorry for not being clear.
	 
	Currently the adapter parses the NALU header to get all the information it needs, e.g. temporal_id.
	 
	When I mentioned the RTP payload header I meant also the NALU header, as, according to RFC-3984: 
	"All NAL units consist of a single NAL unit type octet, which also co-serves as the payload header of this RTP payload format".
	 
	Concerning the packet loss handling in the SEI message based solution, if I understood correctly the sub_seq_frame_num is used to detect a reference picture loss in the sub-sequence.
	However, this way to detect a loss looks to me inefficient (for our scenario) in the case where a packet carrying a slice which is just a sub-part of a picture gest lost, as it is based on sub_seq_frame_num gap detection and sub_seq_frame_num is the same for any slice in the same picture.
	 
	For our purposes this solution could be equivalent to simply delimiting an access unit by the marker_bit and inferring that a lost packet belongs to a layer by simply assuming that any packet in that access unit had the same temporal_id, which we can get either from prefix NALU Units or SEI message.
	Maybe the SEI message based solution could look more useful than the one based on prefix NALUs if we consider the SEI message as a better delimiter of an access unit and consequently of a temporal layer.
	 
	I still think that a parameter simulating the RTP sequence number inside the NALU header information and specific for any temporal layer would have been the smoother solution to keep the single RTP session mode.
	 
	Please correct me if my understanding is somewhere wrong.
	 
	Sorry for being a little long-winded with my emails...
	 
	Thanks a lot
	 
	BR,
	 
	Daniele
	 
	 
	Daniele Renzi
	bSoft -- www.bsoft.info <http://www.bsoft.info> 
	[phone]  (tel/fax)

		----- Original Message ----- 
		From: [e-mail] 
		To: [e-mail] ; [e-mail] ; [e-mail] 
		Cc: [e-mail] 
		Sent: Monday, August 20, 2007 3:35 PM
		Subject: RE: [AVT] RTP streaming and adaptation to AVC of an SVC temporal scalable bitstream - Packet loss

		Dear Daniele,
		 
		So you were assuming that the adapter does not parse anything else than RTP payload header as specified in the payload format. Then how could it find out which packet contains prefix NAL unit to be discarded? To do the adaption, parsing more than RTP payload header is needed anyway. In that case, parsing of a certain SEI message does not impose much burden in addition. In the sub-sequence information SEI message based solution, the adapter parses an sub-sequence information SEI message to detect whether an earlier packet containing a slice to be included in the outcoming stream was lost, then knows how to set the RTP sequence number of the current outgoing packet. 
		 
		BR, YK


________________________________

			From: ext daniele renzi (bsoft) [mailto:[e-mail]] 
			Sent: Monday, August 20, 2007 1:12 PM
			To: Wang Ye-Kui (Nokia-NRC/Tampere); [e-mail]; [e-mail]
			Cc: [e-mail]
			Subject: Re: [AVT] RTP streaming and adaptation to AVC of an SVC temporal scalable bitstream - Packet loss
			
			
			Dear Ye-Kui,
			 
			thanks for your help.
			 
			Just one (two) more question(s).
			In the SEI message based solution, how is a packet loss handled?
			Isn't it the same as in the prefix NALUs based solution, that is by the RTP sequence number?
			It seems that the problem would persist if for example the SEI message gets lost, but I could be wrong.
			 
			From RFC-3984 and RFC-3550 I suppose that the sequence number wouldn't be anyway specific for the single sub-sequence (temporal layer).
			 
			So far I can't see any other solution than multiplexing different RTP sessions for different temporal layers (or sub-sequences), unless a layer-specific sequence number was present in the RTP payload when using a single RTP session.
			 
			Thanks.
			 
			Best regards,
			 
			Daniele
			 
			Daniele Renzi
			bSoft -- www.bsoft.info <http://www.bsoft.info> 
			[phone]  (tel/fax)
			 
			----- Original Message ----- 
			From: <[e-mail] <mailto:[e-mail]> >
			To: <[e-mail] <mailto:[e-mail]> >; <[e-mail] <mailto:[e-mail]> >; <[e-mail] <mailto:[e-mail]> >
			Cc: <[e-mail] <mailto:[e-mail]> >
			Sent: Sunday, August 19, 2007 11:16 PM
			Subject: RE: [AVT] RTP streaming and adaptation to AVC of an SVC temporal scalable bitstream - Packet loss
			
			

			Yet another solution is to use AVC itself instead of SVC (you can also use RFC 3984 instead of the SVC RTP payload draft), as you need only temporal scalability. This requires the use of sub-sequence information SEI messages. The sub_seq_layer_num indicates the temporal layer. You set each sub-sequence layer (i.e. temporal layer) as one sub-sequence, then the sub_seq_frame_num indicates the frame number of each reference frame inside a temporal layer. 
			
			In the prefix NAL unit plus PACSI with TL0PicIndex solution, the adapter needs to parse prefix NAL units, and the outcoming stream can only be of temporal_id equal to 0 (i.e. the lowest temporal layer). In the sub-seqence informtion SEI message based solution, the adapter needs to parse sub-sequence information SEI messages, and the outcoming stream can be of any lower temporal layers. 
			
			BR, YK
			
			>-----Original Message-----
			>From: ext daniele renzi (bsoft) [mailto:[e-mail]] 
			>Sent: Sunday, August 19, 2007 8:55 PM
			>To: Wang Ye-Kui (Nokia-NRC/Tampere); [e-mail] <mailto:[e-mail]> ; Thomas Schierl
			>Cc: [e-mail] <mailto:[e-mail]> 
			>Subject: Re: [AVT] RTP streaming and adaptation to AVC of an 
			>SVC temporal scalable bitstream - Packet loss
			>
			>Dear Ye-Kui, Colin, Thomas, all,
			>
			>many thanks for your clarifications.
			>
			>Anyway, I'd like to define precisely the scenario.
			>Indeed, as Ye-Kui specified, the SVCtoAVC adapter assign new 
			>sequence numbers to the outgoing stream.
			>Here is the problem: if a packet is lost in the incoming 
			>stream, it would be good if the adapter reported this anyway 
			>to the receiver, even though the packet loss was in a 
			>different RTP *segment* (I'm not sure if that can be defined 
			>as a *session*...), as in any case there has been a loss 
			>between the sender and the receiver that should be handled by 
			>the receiver itself.
			>But the adapter cannot say whether this loss affected the base 
			>layer or an enhancement layer, as the prefix NALU could have 
			>been lost and with it the scalability information 
			>(temporal_id). Then it could assert that the sequence number 
			>gap in the incoming stream is due to a loss in the enhancement 
			>layer (then do nothing) even when this isn't true, or viceversa.
			>
			>We're trying to get a solution (maybe different than forcing 
			>the insertion of a sequence number gap in the outgoing stream) 
			>to make the receiver able to handle a loss in both the RTP *segments*.
			>
			>I'll try to evaluate the Thomas' proposal and have a better 
			>look to RFC-3550 and draft-ietf-avt-topologies-06.txt.
			>
			>Thanks again.
			>
			>Best regards,
			>
			>Daniele
			>
			>
			>Daniele Renzi
			>bSoft -- www.bsoft.info <http://www.bsoft.info> 
			>[phone]  (tel/fax)
			>
			>----- Original Message -----
			>From: <[e-mail] <mailto:[e-mail]> >
			>To: <[e-mail] <mailto:[e-mail]> >
			>Cc: <[e-mail] <mailto:[e-mail]> >; <[e-mail] <mailto:[e-mail]> >
			>Sent: Sunday, August 19, 2007 7:34 AM
			>Subject: RE: [AVT] RTP streaming and adaptation to AVC of an 
			>SVC temporalscalable bitstream - Packet loss
			>
			>
			>
			>OK, forget about my naïve question, because I found the 
			>following sentence
			>in RFC 3550, "If multiple data packets are re-encoded into one, or vice
			>versa, a translator MUST assign new sequence numbers to the outgoing
			>packets."
			>
			>BR, YK
			>
			>>-----Original Message-----
			>>From: Wang Ye-Kui (Nokia-NRC/Tampere)
			>>Sent: Sunday, August 19, 2007 12:59 AM
			>>To: 'ext Colin Perkins'
			>>Cc: [e-mail] <mailto:[e-mail]> ; [e-mail] <mailto:[e-mail]> 
			>>Subject: RE: [AVT] RTP streaming and adaptation to AVC of an
			>>SVC temporalscalable bitstream - Packet loss
			>>
			>>
			>>Hi Colin,
			>>
			>>Thanks for your clarification. But according to the following
			>>sentence copied from Daniele's email,
			>>
			>>"... then the SVCtoAVC adapter doesn't know whether this loss
			>>has to be signaled to the receiver, i.e. whether it must
			>>insert a sequence number gap in the outcoming RTP stream...",
			>>
			>>the SVCtoAVC adapter uses a different RTP sequence number
			>>value space for the outcoming RTP steam than the incoming RTP
			>>stream. Is this what a translator can do? It is not clear how
			>>the SVCtoAVC adapter handles the CC, SSRC and CSRC fields and
			>>RTCP traffic, though.
			>>
			>>BR, YK
			>>
			>>>-----Original Message-----
			>>>From: ext Colin Perkins [mailto:[e-mail]]
			>>>Sent: Saturday, August 18, 2007 9:07 PM
			>>>To: Wang Ye-Kui (Nokia-NRC/Tampere)
			>>>Cc: [e-mail] <mailto:[e-mail]> ; [e-mail] <mailto:[e-mail]> 
			>>>Subject: Re: [AVT] RTP streaming and adaptation to AVC of an SVC
			>>>temporalscalable bitstream - Packet loss
			>>>
			>>>On 18 Aug 2007, at 18:36, <[e-mail] <mailto:[e-mail]> > wrote:
			>>>> In your example the SVCtoAVC adapter is an RTP mixer, which
			>>>terminates
			>>>> the RTP session between the sender and itself and restarts
			>>>another RTP
			>>>> session between itself and the receiver.
			>>>> Therefore the RTP sequence number needs to be updated for the base
			>>>> layer packets anyway.
			>>>
			>>>If the SVCtoAVC adapter is a transcoder from an SVC stream to an AVC
			>>>stream, it will be an RTP translator, not an RTP mixer.
			>>>Neither an RTP translator or a RTP mixer terminate the RTP
			>>session. RFC
			>>>3550 and draft-ietf-avt-topologies-06.txt discuss this in 
			>more detail.
			>>>
			>>>--
			>>>Colin Perkins
			>>>http://csperkins.org/
			>>>
			>>>
			>>>
			>
			>
			>
			>-- 
			>No virus found in this incoming message.
			>Checked by AVG Free Edition.
			>Version: 7.5.484 / Virus Database: 269.12.0/961 - Release 
			>Date: 19/08/2007
			>07:27
			>
			>
			>
			
			
			
			-- 
			No virus found in this incoming message.
			Checked by AVG Free Edition. 
			Version: 7.5.484 / Virus Database: 269.12.0/961 - Release Date: 19/08/2007 07:27
			
			

		________________________________

				No virus found in this incoming message.
		Checked by AVG Free Edition. 
		Version: 7.5.484 / Virus Database: 269.12.0/961 - Release Date: 19/08/2007 07:27
		


_______________________________________________
Audio/Video Transport Working Group
[e-mail]
https://www1.ietf.org/mailman/listinfo/avt