Re: [AVT] Audio video synchronization-somne queries

[Magnus Westerlund <[e-mail]>]

Hi Shobana,

See inline:

shobana meenakshi wrote:
> Hi all
>  
> As I am new to implementing RTSP/RTCP protocol, I thought I would seek help from this mailing list.
> 
> I would like to explain my understanding here with respect to audio video synchronization after PAUSE/RESUME and during seek.Request you to correct me if Iam wrong.There are ideally 2 ways the streaming server gives the client,synchronization information for audio and video.
> 
> 1. RTSP PLAY Response
> If RTSP is used as the control channel, after an RTSP PAUSE and PLAY the server sends a play response before sending the first packet after resume in which it tells the RTP time stamp for both audio and video which would be in synch.
> Ideally the implementation on the Client for the audio video synchronization would be , after getting the first packet after resume calculate the offset for the first packet's RTP timestamp from the reference RTPtimestamp  got from the PLay response in both the audio and video threads seperately and increment the presentation time by that offset(after dividing it by the RTp timestamp frequency say 44100 for audio) so that they would be in synch.
> Am I right in this understanding?

I don't know, as can't clearly follow this. However RTSP does provide a 
Normal Play Time (NPT) and the corresponding RTP Timestamps for that 
point in time. Thus one can calculate a local wall clock time for 
playout of the different packets based on these reference.

> Now given this understanding is right, we won't need the wallclock synchronization at all as the RTP timestamp is purely based on the number of samples sent in the packet in which case we won't need the RTCP implementation at all.
> What am I missing here?

Correct, for synchronization of audio and video from the same source 
there are no need for wall clock synchronization.

>  
> 2.RTCP sender report
> The RTP sender report sends the NTP timestamp which would be synchronized for both server and client and for both audio and video and an RTP timestamp associated with it.Ideally after pause and resume after we get the RTCP sender report, we take the RTP timesatamp sent in the RTCP sender report use that as a reference calculate the offset of the first audio and video packets after resume from the reference RTP timestamp and calculate the presentation time in the same way as explained above.

The error you have made here is assuming that sender and receiver would 
be time synchronized. This is not a requirement. RTCP synchronization 
only relies on that the sender has been using the same clock for all 
media it needs to send synchronized. The NTP formated clock has no 
requirement on it being NTP derived and with any tolerance.

> 
> If this is correct what I don't understand is 
>  
> 1.How is the NTP time associated with the RTP timestamp if RTp timestamp is purely based on the sample count in the packet?

If comparing with RTSP where the RTP timestamp for a fixed other clock 
is provide, RTCP is synchronized based on a common time reference 
expressed in NTP format. This can be converted into the corresponding 
timestamps at a particular point in time.

RTSP:

Media A
------------TS_A----------

NPT
------------NPT-----------

Media B
------------TS_B----------

All clocks are provided for the same time.

RTCP

Media A

---TS_A------------------
    ^
    |
    V     NTP Format Clock
---NTP_A-----NTP_B-------
              ^
              |
Media B      V
-------------TS_B--------

Here each media is provided independently a mapping between TS and NTP 
formated reference clock allowing a receiver to calculate the sync.


>  
> In general I have the following queries regarding how the server sends the RTP time stamp in the play response as well as in the RTCP sender report.
>  
> 1.Suppose the RTPtimestamp for audio=12345 and RTP Timestamp for Video=123456 at the point they were paused.
> After Resume the first audio and video packet should have an RTP timestamp of say audio=12360 and video=123486 ideally if we assume that the packets sent has 15 audio samples and 30 video samples respectively.Is my understanding right or is the duration of PAUSE has to do anything with the RTPtimestamps(in the sense would the server calculate the RTPtimestamp of the first packet based on what it would have been if we did not pause at all based on the wall clocktime?)

The duration in pause state do have impact on the TS advancement for the 
first packet after a pause period. Please review Appendix B in 
draft-ietf-mmusic-rfc2326bis-10.txt.

Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone [phone]
Torshamsgatan 23           | Fax   [phone]
S-164 80 Stockholm, Sweden | mailto: [e-mail]


_______________________________________________
Audio/Video Transport Working Group
[e-mail]
https://www1.ietf.org/mailman/listinfo/avt