Re: [rtcweb] Low Level Javascript API Proposal avail on the webrtc list

[Neil Stratford <[e-mail]>]

On Wed, Oct 5, 2011 at 7:21 AM, Roman Shpount <[e-mail]> wrote:

>
> On Wed, Oct 5, 2011 at 12:25 AM, Matthew Kaufman <
> [e-mail]> wrote:
>
>> How about I rephrase?
>>
>> If I generate an offer by writing SDP code in C++ using Win32 APIs to
>> allocate sockets and send and receive packets, and I send a SIP packet that
>> gets multiple answers in the same dialog, how do I process those answers
>> using the Win32 API? See how silly that sounds?
>>
>> Seriously though, if you need an additional local endpoint because of
>> forking, you need to create a second connection object, which will then give
>> you an onCandidates() callback, as I understand this API proposal. The other
>> way to do it is to have the connection object be able to bind to multiple
>> possible far endpoints, but that's not exactly what this API says, and it
>> isn't clear that it is needed. (Because it isn't how other SIP devices use
>> their network stacks to solve this problem either).
>>
>>
> I think you are completely missing the point: If I build similar thing
> using Win32 API, I would re-initialize codec and  RTP states and re-do the
> ICE candidate selection based on the new answer. This particular API has
> methods to add and remove streams and candidates, but no method to start a
> re-negotiation. Don't assume something is silly just because you did not
> understand it.
>

I'm sure if it is required we can add the ability to re-start ICE
negotiation with a new set of farCandidates once the JS decides what to do
as a result of the fork. (I'm not a great supporter of the need for forking,
but if there are real use cases then lets add it.)

...

Once again, you are missing the point completely. Part of the problem here
> is that typically codec information in offer/answer exchange does not
> provide the list of all the codec level parameters supported by the codec.
> The fact that, for instance SILK implementation supports FEC or DTX does not
> have to be present in the offer to be present in the answer. This means
> Codec object will need to expose a more information then typically provided
> in SDP (ie all the supported codec parameters vs just the desired one). This
> means RTC will need to define fields for this object for each codec it will
> support and cannot simply reuse MMUSIC/PAYLOAD standards.
>

I'm not sure I understand why this is such a problem. XMPP/Jingle defines
it's own mapping, which is a common VoIP protocol that is widely supported.
I'd much rather that the JS API is as expressive as possible in codec
specific matters - for example even to the point of providing callbacks to
the JS on certain codec level events, such as decoding a DTMF digit.


> This also brings about the whole new problem -- asymmetric codecs and codec
> parameters. There are a lot of valid use cases when you want to use
> different codecs in different directions (different hardware optimized
> codecs in different mobile devices). There are cases when different payload
> types are used for the same codecs in different directions (It can also be
> considered an RTP bug). You might want to use different video resolutions
> due to device camera and display limitations. You might want to use
> different codec bandwidths due to difference in up and down stream network
> capacity. There is no way to handle this now. I would suggest that the
> better model for third party call control or inbound call would be get the
> offer from remote side, process it on the server or in javascript, pass it
> to the RTC stack in the browser, get the answer from the RTC stack in the
> browser, process it (if needed) in javascript or on the server and send it
> to the remote side.
>

Again, if it's required I don't see why we can't support asymmetric codecs
in the low level API. In my view a lot of this codec manipulation is much
better controlled by the JS which has application specific knowledge about
how to adapt to given situations than the browser itself. How would you
signal to the browser to dynamically lower framerate rather than image size,
or colour depth depending on the application use case? Giving the
application direct control over codec parameters seems a better option.


> Yet another codec negotiation deficiency that I see here is that you cannot
> specify multiple codecs. It is fairly typical to have audio codec, comfort
> noise codec, and DTMF to be used at the same time for a single stream.
>

Agreed - this is an oversight. Though I don't see any mention of DTMF in the
current PeerConnection draft either. I would propose in the low level API
that we add a DTMF codec with a callback on events and a function to
transmit. How would you do this in the PeerConnection API? How would you
specify to generate in-band DTMF on the ULAW codec rather than send an
RFC2833 digit?


> Final comment about the draft is that it is missing a lot of parameters
> about codecs. It is missing things like media type, transport type, clock
> rates, payload types, crypto attributes, etc. It would be interesting to see
> which attributes would end up in which of the objects.


Agreed. These all need to be added - if you have suggestions it would be
great to include them.

Neil