Return-Path: X-Original-To: rtcweb@ietfa.amsl.com Delivered-To: rtcweb@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C60FB21F8A7A for ; Wed, 5 Oct 2011 03:57:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.9 X-Spam-Level: X-Spam-Status: No, score=-2.9 tagged_above=-999 required=5 tests=[AWL=0.076, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CgYa4juzkb3m for ; Wed, 5 Oct 2011 03:57:05 -0700 (PDT) Received: from mail-iy0-f172.google.com (mail-iy0-f172.google.com [209.85.210.172]) by ietfa.amsl.com (Postfix) with ESMTP id 9733021F85F2 for ; Wed, 5 Oct 2011 03:57:05 -0700 (PDT) Received: by iaby26 with SMTP id y26so2061298iab.31 for ; Wed, 05 Oct 2011 04:00:13 -0700 (PDT) MIME-Version: 1.0 Received: by 10.231.0.133 with SMTP id 5mr4161840ibb.22.1317812412946; Wed, 05 Oct 2011 04:00:12 -0700 (PDT) Sender: neils@vipadia.com Received: by 10.231.200.146 with HTTP; Wed, 5 Oct 2011 04:00:12 -0700 (PDT) In-Reply-To: References: <545388B3-3189-4291-BD1D-52898B888F3E@phonefromhere.com> <4E8BC0ED.2020001@skype.net> <4E8BDC3B.7000501@skype.net> Date: Wed, 5 Oct 2011 12:00:12 +0100 X-Google-Sender-Auth: dJrE-cZgwQwgrBTypYfTZytVsCM Message-ID: From: Neil Stratford To: Roman Shpount Content-Type: multipart/alternative; boundary=0015177407b8c118ba04ae8b1c19 Cc: rtcweb@ietf.org Subject: Re: [rtcweb] Low Level Javascript API Proposal avail on the webrtc list X-BeenThere: rtcweb@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Real-Time Communication in WEB-browsers working group list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 10:57:06 -0000 --0015177407b8c118ba04ae8b1c19 Content-Type: text/plain; charset=ISO-8859-1 On Wed, Oct 5, 2011 at 7:21 AM, Roman Shpount wrote: > > On Wed, Oct 5, 2011 at 12:25 AM, Matthew Kaufman < > matthew.kaufman@skype.net> wrote: > >> How about I rephrase? >> >> If I generate an offer by writing SDP code in C++ using Win32 APIs to >> allocate sockets and send and receive packets, and I send a SIP packet that >> gets multiple answers in the same dialog, how do I process those answers >> using the Win32 API? See how silly that sounds? >> >> Seriously though, if you need an additional local endpoint because of >> forking, you need to create a second connection object, which will then give >> you an onCandidates() callback, as I understand this API proposal. The other >> way to do it is to have the connection object be able to bind to multiple >> possible far endpoints, but that's not exactly what this API says, and it >> isn't clear that it is needed. (Because it isn't how other SIP devices use >> their network stacks to solve this problem either). >> >> > I think you are completely missing the point: If I build similar thing > using Win32 API, I would re-initialize codec and RTP states and re-do the > ICE candidate selection based on the new answer. This particular API has > methods to add and remove streams and candidates, but no method to start a > re-negotiation. Don't assume something is silly just because you did not > understand it. > I'm sure if it is required we can add the ability to re-start ICE negotiation with a new set of farCandidates once the JS decides what to do as a result of the fork. (I'm not a great supporter of the need for forking, but if there are real use cases then lets add it.) ... Once again, you are missing the point completely. Part of the problem here > is that typically codec information in offer/answer exchange does not > provide the list of all the codec level parameters supported by the codec. > The fact that, for instance SILK implementation supports FEC or DTX does not > have to be present in the offer to be present in the answer. This means > Codec object will need to expose a more information then typically provided > in SDP (ie all the supported codec parameters vs just the desired one). This > means RTC will need to define fields for this object for each codec it will > support and cannot simply reuse MMUSIC/PAYLOAD standards. > I'm not sure I understand why this is such a problem. XMPP/Jingle defines it's own mapping, which is a common VoIP protocol that is widely supported. I'd much rather that the JS API is as expressive as possible in codec specific matters - for example even to the point of providing callbacks to the JS on certain codec level events, such as decoding a DTMF digit. > This also brings about the whole new problem -- asymmetric codecs and codec > parameters. There are a lot of valid use cases when you want to use > different codecs in different directions (different hardware optimized > codecs in different mobile devices). There are cases when different payload > types are used for the same codecs in different directions (It can also be > considered an RTP bug). You might want to use different video resolutions > due to device camera and display limitations. You might want to use > different codec bandwidths due to difference in up and down stream network > capacity. There is no way to handle this now. I would suggest that the > better model for third party call control or inbound call would be get the > offer from remote side, process it on the server or in javascript, pass it > to the RTC stack in the browser, get the answer from the RTC stack in the > browser, process it (if needed) in javascript or on the server and send it > to the remote side. > Again, if it's required I don't see why we can't support asymmetric codecs in the low level API. In my view a lot of this codec manipulation is much better controlled by the JS which has application specific knowledge about how to adapt to given situations than the browser itself. How would you signal to the browser to dynamically lower framerate rather than image size, or colour depth depending on the application use case? Giving the application direct control over codec parameters seems a better option. > Yet another codec negotiation deficiency that I see here is that you cannot > specify multiple codecs. It is fairly typical to have audio codec, comfort > noise codec, and DTMF to be used at the same time for a single stream. > Agreed - this is an oversight. Though I don't see any mention of DTMF in the current PeerConnection draft either. I would propose in the low level API that we add a DTMF codec with a callback on events and a function to transmit. How would you do this in the PeerConnection API? How would you specify to generate in-band DTMF on the ULAW codec rather than send an RFC2833 digit? > Final comment about the draft is that it is missing a lot of parameters > about codecs. It is missing things like media type, transport type, clock > rates, payload types, crypto attributes, etc. It would be interesting to see > which attributes would end up in which of the objects. Agreed. These all need to be added - if you have suggestions it would be great to include them. Neil --0015177407b8c118ba04ae8b1c19 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Wed, Oct 5, 2011 at 7:21 AM, Roman Shpount <roman@telurix.com<= /a>> wrote:

On Wed, Oct 5, 2011 at 12:= 25 AM, Matthew Kaufman <matthew.kaufman@skype.net> w= rote:
=20 =20 =20
How about I rephrase?

If I generate an offer by writing SDP code in C++ using Win32 APIs to allocate sockets and send and receive packets, and I send a SIP packet that gets multiple answers in the same dialog, how do I process those answers using the Win32 API? See how silly that sounds?

Seriously though, if you need an additional local endpoint because of forking, you need to create a second connection object, which will then give you an onCandidates() callback, as I understand this API proposal. The other way to do it is to have the connection object be able to bind to multiple possible far endpoints, but that's not exactly what this API says, and it isn't clear that = it is needed. (Because it isn't how other SIP devices use their network stacks to solve this problem either).

=

I think you are completely missing the point: If I build sim= ilar thing using Win32 API, I would re-initialize codec and=A0 RTP states a= nd re-do the ICE candidate selection based on the new answer. This particul= ar API has methods to add and remove streams and candidates, but no method = to start a re-negotiation. Don't assume something is silly just because= you did not understand it.

I'm sure if it is required= we can add the ability to re-start ICE negotiation with a new set of farCa= ndidates once the JS decides what to do as a result of the fork. (I'm n= ot a great supporter of the need for forking, but if there are real use cas= es then lets add it.)
=A0
...

= Once again, you are missing the point completely. Part of the problem here = is that typically codec information in offer/answer exchange does not provi= de the list of all the codec level parameters supported by the codec. The f= act that, for instance SILK implementation supports FEC or DTX does not hav= e to be present in the offer to be present in the answer. This means Codec = object will need to expose a more information then typically provided in SD= P (ie all the supported codec parameters vs just the desired one). This mea= ns RTC will need to define fields for this object for each codec it will su= pport and cannot simply reuse MMUSIC/PAYLOAD standards.

I'm not sure I understand why this is = such a problem. XMPP/Jingle defines it's own mapping, which is a common= VoIP protocol that is widely supported. I'd much rather that the JS AP= I is as expressive as possible in codec specific matters - for example even= to the point of providing callbacks to the JS on certain codec level event= s, such as decoding a DTMF digit.
=A0
This also brings about the wh= ole new problem -- asymmetric codecs and codec parameters. There are a lot = of valid use cases when you want to use different codecs in different direc= tions (different hardware optimized codecs in different mobile devices). Th= ere are cases when different payload types are used for the same codecs in = different directions (It can also be considered an RTP bug). You might want= to use different video resolutions due to device camera and display limita= tions. You might want to use different codec bandwidths due to difference i= n up and down stream network capacity. There is no way to handle this now. = I would suggest that the better model for third party call control or inbou= nd call would be get the offer from remote side, process it on the server o= r in javascript, pass it to the RTC stack in the browser, get the answer fr= om the RTC stack in the browser, process it (if needed) in javascript or on= the server and send it to the remote side.

Again, if it's required I don't se= e why we can't support asymmetric codecs in the low level API. In my vi= ew a lot of this codec manipulation is much better controlled by the JS whi= ch has application specific knowledge about how to adapt to given situation= s than the browser itself. How would you signal to the browser to dynamical= ly lower framerate rather than image size, or colour depth depending on the= application use case? Giving the application direct control over codec par= ameters seems a better option.
=A0
Yet another codec negotiation= deficiency that I see here is that you cannot specify multiple codecs. It = is fairly typical to have audio codec, comfort noise codec, and DTMF to be = used at the same time for a single stream.

Agreed - this is an oversight. Though I do= n't see any mention of DTMF in the current PeerConnection draft either.= I would propose in the low level API that we add a DTMF codec with a callb= ack on events and a function to transmit. How would you do this in the Peer= Connection API? How would you specify to generate in-band DTMF on the ULAW = codec rather than send an RFC2833 digit?
=A0
Final comment about the draft= is that it is missing a lot of parameters about codecs. It is missing thin= gs like media type, transport type, clock rates, payload types, crypto attr= ibutes, etc. It would be interesting to see which attributes would end up i= n which of the objects.=A0

Agreed. These all need to be added - if you have sugges= tions it would be great to include them.

Neil=A0
--0015177407b8c118ba04ae8b1c19--