Re: [rtcweb] Low Level Javascript API Proposal avail on the webrtc list

[Roman Shpount <[e-mail]>]

On Wed, Oct 5, 2011 at 12:25 AM, Matthew Kaufman
<[e-mail]>wrote:

> How about I rephrase?
>
> If I generate an offer by writing SDP code in C++ using Win32 APIs to
> allocate sockets and send and receive packets, and I send a SIP packet that
> gets multiple answers in the same dialog, how do I process those answers
> using the Win32 API? See how silly that sounds?
>
> Seriously though, if you need an additional local endpoint because of
> forking, you need to create a second connection object, which will then give
> you an onCandidates() callback, as I understand this API proposal. The other
> way to do it is to have the connection object be able to bind to multiple
> possible far endpoints, but that's not exactly what this API says, and it
> isn't clear that it is needed. (Because it isn't how other SIP devices use
> their network stacks to solve this problem either).
>
>
I think you are completely missing the point: If I build similar thing using
Win32 API, I would re-initialize codec and  RTP states and re-do the ICE
candidate selection based on the new answer. This particular API has methods
to add and remove streams and candidates, but no method to start a
re-negotiation. Don't assume something is silly just because you did not
understand it.


> I don't see why you can't create a second connection object (or extend this
> API to allow cloning), and add streams and candidates to that connection
> object. Should be possible to add the same stream to two different
> connection objects, too.
>
>
Each connection object has an associated local port on which it is listening
for incoming media and ICE STUN messages. What I need is to be able to reuse
the same port to connect to multiple different sets of remote candidates.
Remote IP/port should be sufficient to disambiguate different streams.



>
> 6. How do we deal with CODEC negotiated parameters?
>
>
> I believe that the codec parameters are exposed by the codec object, at
> least that's how I read this spec.
>
>
> It is possible to have codec level parameter that needs to be negotiated
> via offer/answer exchange.
>
>
> Maybe. Or maybe there's an offer-answer at the server that picks the right
> answer and then sends that down as a setting to the browser.
>
>
>  There are also codec level parameters that are affected by stream level
> parameters (like iLBC mode=30 that only makes sense with ptime dividable by
> 30).
>
>
> That's just a bug with how RTP works.
>
>
>  Is it going to be implemented in JavaScript?
>
>
> I suppose it might need to be, if you really need to select
> connection-level things to match codec-level things and vice versa. Or
> control it up at the server and then send the right settings down.
>
>
>  This might not be the best idea, since this will require JavaScript to
> have a detailed knowledge about each codec implementation provided by the
> browser. For third party call control, it would be more robust to provide a
> way to return actual codec parameters selected by the browser so that they
> can be returned in the answer.
>
>
> Except that we often don't want the codec parameters to be "selected by the
> browser". We want to know what the browser can do and just set them a
> particular way.
>
> I've already suggested that the format of the "detailed knowledge" can
> actually just reuse the parameter strings that are already defined for use
> in SDP.
>
>
Once again, you are missing the point completely. Part of the problem here
is that typically codec information in offer/answer exchange does not
provide the list of all the codec level parameters supported by the codec.
The fact that, for instance SILK implementation supports FEC or DTX does not
have to be present in the offer to be present in the answer. This means
Codec object will need to expose a more information then typically provided
in SDP (ie all the supported codec parameters vs just the desired one). This
means RTC will need to define fields for this object for each codec it will
support and cannot simply reuse MMUSIC/PAYLOAD standards.

This also brings about the whole new problem -- asymmetric codecs and codec
parameters. There are a lot of valid use cases when you want to use
different codecs in different directions (different hardware optimized
codecs in different mobile devices). There are cases when different payload
types are used for the same codecs in different directions (It can also be
considered an RTP bug). You might want to use different video resolutions
due to device camera and display limitations. You might want to use
different codec bandwidths due to difference in up and down stream network
capacity. There is no way to handle this now. I would suggest that the
better model for third party call control or inbound call would be get the
offer from remote side, process it on the server or in javascript, pass it
to the RTC stack in the browser, get the answer from the RTC stack in the
browser, process it (if needed) in javascript or on the server and send it
to the remote side.

Yet another codec negotiation deficiency that I see here is that you cannot
specify multiple codecs. It is fairly typical to have audio codec, comfort
noise codec, and DTMF to be used at the same time for a single stream.

Final comment about the draft is that it is missing a lot of parameters
about codecs. It is missing things like media type, transport type, clock
rates, payload types, crypto attributes, etc. It would be interesting to see
which attributes would end up in which of the objects.
______________
Roman Shpount