Re: [AVT] Working Group Last Call: draft-ietf-avt-rfc3047-bis-06.txt

[Randell Jesup <[e-mail]>]

"Even, Roni" <[e-mail]> writes:
>> 3.2) "An application MAY switch bit rates
>>    from packet to packet by defining two payload type values and
>>    switching between them."
>> -> it is not limited to 2

>[RE:] It is from the RFC3047 that had two bit rates. I will change to "An
>application MAY switch bit rates and clock rates from packet to packet by
>defining different payload type value and switching between them."

I haven't been following this draft, but please check the AVT archives -
mixing clock rates in the same RTP stream (by switching between payloads
with different RTP timestamp rates) causes some real problems.

While in a perfect world with no packet loss this may seem ok, it's really
not. 

Consider the case of synchronization between two streams.  Normally this is
done via RTCP packets which provide RTP timestamp -> NTP time translation.
Now, if the RTP timestamp rate can change from one packet to the next, 
properly adjusting the inter-stream synchronization becomes impossible
if the wrong packets are lost.

For example:
m=audio 4567 RTP/AVP 99 100 101
a=rtpmap:99 foo/16000
a=rtpmap:100 foo/32000
a=rtpmap:101 telephone-event/8000

Payload   Sequence  Timestamp
99          0          0
99          1          320
99          2          640
<packets 3-8 lost>
100         9          3520

Now, you might be able to infer that packets 3-5 were type 99, and 6-8 were 
type 100.  But a) this is a tricky calculation to make, and prone to error
as you add codecs and bitrates, and b) there might be multiple solutions to
the problem - especially if there were a telephone-event packet in there.
If you can't figure out what the timestamp -> NTP relationship is, your
synchronization is screwed.

Even worse, you need that TS->NTP data even in order to play back packet 9
at the correct time, even in a single-stream case.

I have *strong* objections to mixing timestamp rates on a packet-by-packet
basis without some strong recommedations or guidelines on how to handle
all the edge cases.  If we could, I'd bump all the codecs up to the
lowest-common-multiple, though in some odd cases that could be too large,
at least in theory.  However, I'm certain huge numbers of endpoints would
barf on PCMU with a 32000 timestamp rate.


This was discussed in June of 2007 under the Subject: 
Re: [AVT] I-D ACTION:draft-ietf-avt-rtp-speex-01.txt

See my messages of June 20, 2007 for example:
(">>" is from me, ">" is Magnus' reply, and then my reply to him)

  >> The overloading of sample rate onto the timestamp rate for RTP for (most)
  >> audio codecs causes some real problems with certain cases.  Switching
  >> (in-call) between codecs with different sample rates -
  >> overloading means that playout time is tricky to calculate, and doubly so
  >> if there was a lost packet at the boundary.  (It also could significantly
  >> upset jitter buffers and the like, unless you do a full reset on codec
  >> shift.)  It may also complicate related RFCs like 2833, since 2833 is
  >> normally intermixed with regular codecs:
  >> m=audio 4321 RTP/AVP 98 99 100 101
  >> a=rtpmap:98 ILBC/8000
  >> a=rtpmap:99  G7221/16000
  >> a=rtpmap:100 G7221/32000
  >> a=rtpmap:101 telephone-event/8000
  >> (a=fmtp's would be needed too)
  >> Ok: what's the timestamp rate?  :-)  And when can it change?
  >
  >For this case to have a chance to work you need three telephone-event lines
  >like this:
  >  a=rtpmap:101 telephone-event/8000
  >  a=rtpmap:102 telephone-event/16000
  >  a=rtpmap:103 telephone-event/32000
  >
  >So that the packets sent is sent at the same timestamp rate as used by the
  >main codec.
  
  So long as none of the events span a timestamp rate change...
  
  I have a strong suspicion that including the above is likely to cause
  compatibility problems in practice (in addition to being lengthy, plus the
  related fmtp lines, and so adding to the overall SIP/UDP problem).
  
  Perhaps the real issue here comes down to a media stream is really a form
  of multiplexed channel.  Some payload "switches" are real switches, others
  are just passing additional streams of related data (like 2833 DTMF data).
  There are no limits on when payloads can switch, or on overlap between
  different payloads.  You *could* create a stream with 10 G.711 payloads,
  with all 10 arriving at all times, and each one being a different language.
  This might even be useful for RTSP, IPTV, etc.
  
  Now in practice devices won't overlap or even switch often, at least for
  VoIP devices, with rare exceptions like 2833.  Switches are usually to
  respond to network (or processor) impairments, quality, etc.  The probably
  works moderately well today in narrowband VoIP devices.  Mixing wideband
  (G.722/etc) into the mix really does confuse things if the device doesn't
  pick one at call setup and stick with it (or others of the same timestamp
  rate).
  
  Either we need to provide guidance on how to handle timestamp shifts, or
  we need to move away from them (tough), or we need to move away from
  timestamp rate == sample rate.  (Tough, but will help us in the long run.)

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[e-mail]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

_______________________________________________
Audio/Video Transport Working Group
[e-mail]
https://www.ietf.org/mailman/listinfo/avt