Re: [rtcweb] Offer/answer for heterogeneous encode/decode

["Paul Giralt (pgiralt)" <[e-mail]>]

On Nov 22, 2013, at 5:11 PM, Justin Uberti <[e-mail]<mailto:[e-mail]>> wrote:

On Fri, Nov 22, 2013 at 11:57 AM, Paul Giralt <[e-mail]<mailto:[e-mail]>> wrote:
I may be missing something but I don't know that this is entirely correct. >From 3264:

   The list of media formats for each media stream conveys two pieces of
   information, namely the set of formats (codecs and any parameters
   associated with the codec, in the case of RTP) that the offerer is
   capable of sending and/or receiving (depending on the direction
   attributes), and, in the case of RTP, the RTP payload type numbers
   used to identify those formats.

To me, this means that a capability in an offer or answer (with a direction attribute of sendrecv) means that you are capable of both sending and receiving for that particular capability. I don't see a way where you can specify (within a single m= line) that you are capable of receiving on two of the specified payload types, but only capable of sending on one of the two. In other words, if an offer or answer specifies both H.264 and VP8 with a sendrecv attribute, this implies that you are capable of both sending and receiving both.

You could, of course, specify two separate m= lines where you specify both codecs as recvonly and the codec you support for sending as sendonly, but this is very complicated because now you have two separate m= lines for what really should be one bi-directional stream.

Someone please correct me if I'm misunderstanding this.


The sender has full discretion of which codec to actually use. Assuming the remote side has offered both codec A and codec B (indicating that they are willing to receive either), the local side can choose to send either A or B, and can switch at any time.

>From later on in the paragraph from 3264 that you quoted:

   If multiple formats are listed, it
   means that the offerer is capable of making use of any of those
   formats during the session.  In other words, the answerer MAY change
   formats in the middle of the session, making use of any of the
   formats listed, without sending a new offer.

and later:




   In all cases, the formats in the "m=" line MUST be listed in order of
   preference, with the first format listed being preferred.  In this
   case, preferred means that the recipient of the offer SHOULD use the
   format with the highest preference that is acceptable to it.

I certainly think that "is acceptable to it" covers "has an encoder for it".



While you’re right that it would work for this scenario, my point was really that Offer/Answer is not really asymmetric as implied earlier. Take for example the hypothetical case where you are only able to decode VP8 but only able to encode H.264. If I offer VP8 as my only codec (because it’s the only thing I’m able to decode therefore I never want anyone to send me anything other than VP8) I cannot send H.264 in the offer because that implies I’m able to decode it. The other side then wants to say it can only receive H.264 so it would have to send back an answer with only H.264. I guess there’s nothing really inherently stopping you from doing this because as far as I can tell, 3264 only says the answer has to be a subset of the offer for multicast streams, however how would the answering side know that the offering side is even capable to receiving H.264? Perhaps Offer/Answer can technically be asymmetric, but it doesn’t seem practical to use it this way because you cannot really indicate your send and receive capabilities independent of each other.

-Paul