Re: [rtcweb] Proposal for dealing with CNAMEs and MSIDs for synchronization

[Magnus Westerlund <[e-mail]>]

On 2013-02-14 05:23, Timothy B. Terriberry wrote:
> Harald Alvestrand wrote:
>> I don't understand what the problem is here. I may be dumb.
>> If syncing is important to the guy writing the service, (he can't live
>> with them being unsynchronized), he should put them in the same stream.
> 
> The argument goes that if we leave this unspecified, web authors will
> test against browser A which does X, and not against browser B which
> does Y, and users of browser B will suffer when things don't work like
> they did in browser A.
> 
> Personally I'm not convinced this argument applies here. For cases like
> "Do the contents of this table element show up at all when I fail to
> close my <tr>?" it's pretty important, but in reality resynchronization
> is only going to affect things so much... if things get more than a few
> 100 ms out of sync on the wire, the experience is going to be terrible
> no matter what you do (and I'd call them "bad" long before that point).
> 
> The goal of pixel-perfect rendering for webpages might be reasonable,
> but sample-accurate rendering of real-time media will never be possible
> given the limitations of real-world networks, and whether you're forcing
> synchronization or not, the media will still play.

My understanding of the underlying issues why this is important is
discussed in
https://datatracker.ietf.org/doc/draft-burman-rtcweb-mmusic-media-structure/
See Section 4.1.3

Lets start with two media streams from end-point that shares media
sources. Here we encounter another open issue, namely the relation
between media sources, a particular encoding and MediaStreamTrack. To my
knowledge it is not agreed on what should happen if I have two
MediaStreams, each with an audio and video MediaStreamTrack that are the
same media source, i.e. microphone and video camera. Even if no
configuration for encoding has been done, will the peer receive two
copies or are these optimized to send only two RTP streams, one with the
audio, one with the video that is used in both?

If they are then they MUST share the same CNAME in both media streams
all the MediaStreamTracks.

If the audio MediaStreamTrack in each MediaStream actually gets
different encodings, which I think is a waste unless intended due to
encoding requirements, but then we are in fact into Simulcast. IF one
has two different encodings and both are sent, then playing the same
audio source twice without sample accurate synchronization you have just
created an echo effect.

Even if the JS application switches back and forth between the media
streams there could be playback overlapping unless they are in sync.

> 
>> The practical issue I can think of would be a source that had its own
>> clock, not the system clock, but is otherwise a local source. Having the
>> same CNAME would require resynchronizing that source. I don't know if
>> any such sources exist in practice, but again, I don't want to outlaw
>> them.
> 
> I can certainly think of cases where two local streams with, e.g., the
> same sampling rate would advance at different rates measured against a
> global wallclock. For example, a gUM stream running against the
> microphone's clock vs. a <video> tag playing back against the soundcard
> output clock.

Yes, but that should be dealt with locally. It is after all detectable
by using the global wallclock and can be compensated for. In fact RTCP
synchronization mechanism deals with the relative clock drift of
sampling clocks compared to the senders wall clock by updating the
offset in new RTCP Sender Reports.

This might not be implemented correctly in many implementations, but
that is an implementation bug, not a protocol issue. Same goes for the
local playback with the above characteristics.

> 
> Then AFAICT forcing the use of the same CNAME means we will always add
> some additional latency in the receiver to perform the
> resynchronization. In the above situation, I'm not sure that's always
> the best choice.

I think it is important that we are clear on that all media sources that
comes from the same synchronization context must be using the same
CNAME. This implies that media mixing entities track the incoming clocks
and keep the same off-set over multiple RTP streams from the same
synchronization context to maintain synchronization of that context
across the mixing/relaying/video switching.

Yes, it may incur delay, depending on what media operation you are
doing. I think the requirement we are discussing is what the sender MUST
do so that the receiver can ensure synchronization if they want to.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  [phone]
Färögatan 6                | Mobile [phone]
SE-164 80 Stockholm, Sweden| mailto: [e-mail]
----------------------------------------------------------------------