Re: [rtcweb] codec and connection negotiation

[Sohel Khan <[e-mail]>]

Matt, Thank you for nicely presenting various sets of session capability transfer methods.
I like X1 and X2, and also Y1 and Y2.  Although I do not have any opinion on X3, it is okay to add this use case in the document. Diverse methods create innovation opportunities although create interoperability challenges. 

Although in a utopian network, it is tempting to consider that no-middle box (e.g., network transcoder) should exist, unfortunately, not all end system support the same sets of codec in the today's network. Therefore, a network transcoding system may be needed; thus, the need of servers in the network.


Apart from above method selections, in details level, we will need some important parameters. I am writing here although this thread may not be the right forum.
One important point is to ensure we have a field or parameter, or arrangement order to depict media capability preference. For example, in SDP, the first codec in the list is preferred. This is helpful for both network (e.g., transcoding mediation) and end systems, in which those support multiple codecs but would like to select the default codec. In addition, we have to consider a parameter for "force: no <parameter>" to explicitly mention unwanted media capability. For example, an end user or network may like to preclude a "bad-codec". In this case, the end system or network should be able to negotiate "force: no bad-codec".  

In DTMF codec negotiation, we will need to ensure that a parameter clearly indicates which DTMF transfer method is supported. Currently, RFC 2833  telephone-event expresses intention of out-of-band RTP DTMF digit transfer. In my opinion, we do not have a tool to deny SIP INFO-based DTMF digit transfer.  Some transcoder in the network or end user codec may prefer not to use INFO-based DTMF digit transfer. In this case, a parameter such as "force: no info-dtmf-event" will be helpful.

Regards,
Sohel Khan


________________________________
From: Matthew Kaufman <[e-mail]>
To: "[e-mail]" <[e-mail]>
Sent: Friday, August 5, 2011 2:33 PM
Subject: [rtcweb] codec and connection negotiation

I put this together to try to help myself, and perhaps others, understand the various ways in which codec and connection negotiation might be decomposed and then put together for RTCWEB applications. It is a bit of a stream of consciousness, but hopefully we can get some good discussion provoked.

A. How to encode codec capabilities or choices

A1 - Don't provide a canonical way of encoding codec settings. The choice of which codec and what parameters is encoded in an ad-hoc way that requires no standardization.

A2 - Encode codec settings using SDP. The choice of which codec and what parameters is encoded using the syntax and semantics of SDP and reuses the SDP standardization work.

A3 - Encode codec settings using a JSON encoding of SDP. The choice of which codec and what parameters is using the semantics of SDP but a JSON syntax. Reuses the SDP standardization plus new standards around how to encode SDP in JSON.

B. How to expose codec capabilities and controls

B1. Expose codec capabilities and settings via individual Javascript APIs. For example one might say "camera.encodeMode = "H.264"; camera.encodeWidth = 640;" or "if (camera.canEncode("H.264")) ..."

B2. Expose codec capabilities and settings via a combined Javascript API that concatenates all the capabilities and settings into a single object. Calling "camera.encodeCapabilities()" would return a large object that is a full enumeration of what it can do.

B3. Expose codec capabilities and settings via a combined Javascript API that produces and consumes SDP strings.

B4. Don't expose codec capbilities and settings via Javascript, rather handle this outside of Javascript.

C. How to agree on which codec and settings to use

C1. The server receives information about the capabilities, decides what settings should be used, and communicates them to the endpoints

C2. The endpoints agree using offer-answer, using the server as a communication channel

C3. The endpoints agree using offer-answer, directly between the two endpoints


(There's obviously a few additional nuances, like whether to use something like RFC 5939 SDP capability negotiation, that aren't well-captured above.)

Now, with the above, we find that some of the models that have been discussed are combinations of the above choices.

X1 - Use A2, B3, C2 - The endpoints generate SDP for offer-answer by firing an SDP event, having it sent via a server to the far end, where it is injected as SDP into the Javascript.

X2 - Use A1, B1, C1 - The endpoints run Javascript that inspects their capabilities, sends that to the server which decides what is best, and sends back information that the Javascript uses to set up the encoders.

X3 - Use A2, B4, C3 - The endpoints establish a communication channel using ICE and an out-of-band channel, they then use SDP inside of SIP to directly negotiate using offer-answer, the server has no information about what was chosen and does not participate.

We also find that almost all the models can map, though with varying degrees of pain, to others.

For instance:

Y1 - Use A2, B1, C2 - The endpoints run Javascript that inspects their capabilities and encodes it into an SDP string. This is sent via the server to the far end, where the SDP is parsed in Javascript and the individual APIs called to implement SDP offer-answer.

Y2 - Use A1, B3, C1 - The endpoints generate SDP for offer-answer by firing an SDP event. The endpoint then runs Javascript that extracts from the SDP all the individual parameters and re-encodes them using an ad-hoc scheme. The server determines what each end should do and sends ad-hoc messages to the endpoints which then generate the SDP answer that causes the desired outcome.

It should be noted that extracting a full set of capabilities when only offer-answer is available via the API is particularly painful, as it might be necessary to explore a very wide space of fake offers to get answers that clarify things like "can do G.722 audio but only if not doing H.264 video".

We might be able to learn from some previous experiences:

1. Web-based email. Web browsers run Javascript that uses ad-hoc signaling to/from the web site in order to send and receive email. The browser does not have SMTP, POP, or IMAP implementations, or even know how to parse RFC822 headers on its own. This argues for A1, B1 (or maybe B2), C1.

2. Web-based image display. Web browsers send an "Accept" header via HTTP indicating which MIME types they can handle. Knowing whether a browser can do JPEG is as simple as looking for "image/jpeg" in the header. This is probably at the wrong layer for sophisticated web applications (as this isn't about what comes back over HTTP, but what can be supported over a separate channel), but might be appropriate if we could agree on a small number of supported codecs and simply have the browser tell the server in a similar manner whether or not it can do RTCWEB.

3. SIP. SIP endpoints use SDP directly between the endpoints (though possibly with intermediaries that rarely change the SDP) to do offer-answer. This is most like A2, B3 (or B4), C3 (or C2).

4. MGCP. MGCP uses SDP for both capabilities and settings, but the server is in control. This is most like A2, B3, C1.

5. H.323 (H.245) uses a protocol other than SDP for both capabilities and settings, but the server is in control. This is like A3 (or some other encoding), B2, C1

As for connection negotiation, most of the separation previously also applies... again whether to put the ICE candidates into SDP or not, whether to have APIs that take blobs or individual calls, etc.

But the other issue that arises when we start talking about connection negotiation is that if you choose SDP offer-answer for codec negotiation, and SDP for ICE candidates, there is the temptation to combine the SDP for the two, thus conflating the process of establishing a logical channel over which various types of media might flow, and establishing just which media should be sent over that channel.

Matthew Kaufman

_______________________________________________
rtcweb mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/rtcweb