Re: [rtcweb] Additional requirement - audio-only communication

[Justin Uberti <[e-mail]>]

On Fri, Aug 26, 2011 at 10:23 AM, Stefan Håkansson LK <
[e-mail]> wrote:

> On 2011-08-26 16:11, Justin Uberti wrote:
>
>>
>>
>> On Fri, Aug 26, 2011 at 3:14 AM, Stefan Håkansson LK
>> <stefan.lk.hakansson@ericsson.**com <[e-mail]>
>> <mailto:stefan.lk.hakansson@**ericsson.com<[e-mail]>>>
>> wrote:
>>
>>    FWIW,
>>
>>    this is how I would do the case of the caller (initiator) wanting to
>>    receive only audio, but willing to send audio and video with the
>>    current API proposal:
>>
>>    1. Generate an audiovisual mediastream (getUserMedia)
>>    2. create PeerConnection, add the above mediastream (addStream)
>>    3. combine the SDP received from PeerConnection with the following
>>    fields "This is an invitation to a communication", "sending audio
>>    and video", "want to receive only audio" into a message sent to the
>>    peer.
>>    4. the app in the peer (same app!) receives the message, reads the
>>    fields, (given user agreement to start the communication) creates a
>>    PeerConnection, feeds the received SDP into the PC, generates an
>>    *audio only* stream (getUserMedia) and adds it to the PC
>>    5. A couple of SDP exchanges later the application is working as
>>    intended. (there will be onaddstream's, you will attach those
>>    streams to video/audio elements etc.)
>>
>>    Quite simple! And no need to have the application read, understand,
>>    or modify, the SDPs - they are opaque. (Then, of course, the
>>    mediastreams must be mapped to RTP sessions and such stuff, but I'm
>>    sure that is solvable and should not be visible in the API IMHO.)
>>
>>
>> Maybe I misunderstand, but it sounds like with these fields you are
>> defining a new signaling protocol, which I think we want to avoid.
>>
>
> That's not how I see it. It is the same web app in both peers, and they can
> communicate inbetween them in any way the app developer see fit (using e.g.
> XHR via the web server). There is already a need for a mechanism to pass the
> SDPs between the browsers, and that mechanism could be used to pass other
> data.
>
> So no new protocol needed!


How will this work if it is not the same web app in both peers, i.e. an
interop scenario? Clearly if it is the same app we can pass along whatever
meta-info we want to instruct the remote side on what to do.


>
>
>
>> Going back to your step 3 - here the initiator can of course munge the
>> generated SDP and discard the m=video section, which should work, but in
>> other cases this munging may cause things to get out of sync since the
>> initiator browser's understanding of the offer now differs from reality.
>>
>
> I would really like to avoid that the web app has to read and modify the
> SDP except for in exceptional cases.
>
> In the current API there is also no need (at least in this case) since all
> streams are set up unidirectional - all that is needed is that the
> application in the browser of the callee gets to know it should add an
> audio-only stream (and it can be told that as I outlined above).
>
>  What we really want is some way to either a) an API to tell the
>> initiator browser to produce an offer with no m=video, or b) a way for
>> the application to customize or generate the offer and feed it back to
>> the browser. b) is definitely more complex, but a) could end up causing
>> us to add a lot of knobs to the API.
>>
>>
>>    Stefan
>>
>>
>>
>>
>>
>>
>>    on 2011-08-26 06:48, Justin Uberti wrote:
>>
>>
>>
>>        On Thu, Aug 25, 2011 at 8:01 PM, Timothy B. Terriberry
>>        <[e-mail] <mailto:tterriberry@mozilla.**com<[e-mail]>
>> >
>>        <mailto:tterriberry@mozilla.__**com
>>
>>        <mailto:tterriberry@mozilla.**com <[e-mail]>>>>
>> wrote:
>>
>>            Justin Uberti wrote:
>>
>>                I think it makes sense for the browser to emit
>> capabilities,
>>                which could
>>
>>
>>            I agree there's clearly some gaps here that need to be filled.
>>
>>
>>                then be used by the web app to generate a SDP offer or
>>        answer. This
>>
>>
>>                The original problem that started this email is one
>> specific
>>                example -
>>                if the callee application wants to only receive audio, the
>>                application
>>                can generate an audio-only SDP based on the offer, the
>>        browser
>>
>>
>>            I think the Harald's original problem was the other way
>>        around: the
>>            _caller_ wants to only receive audio, and needs to generate
>>        an SDP
>>            _offer_ that says that, even if the browser is capable of
>>        receiving
>>            video. I don't think that invalidates your point, though.
>>
>>
>>                capabilities, and the desired app behavior - without any
>> new
>>                APIs in the
>>                browser.
>>
>>
>>            But I'm not sure what you mean by "without any new APIs"...
>>        in your
>>            approach, something has to be able to enumerate the
>>        capabilities in
>>            sufficient detail for the webapp to generate SDP by itself.
>>        I don't
>>            think there are any existing APIs that go that far.
>>
>>
>>        I meant, without specific APIs for that specific use case (i.e.
>>        "create
>>        an audio-only offer"). We would need some sort of
>>        GetCapabilities API
>>        that returned a blob of all the session description options the
>>        browser
>>        supported, which probably could be formatted as a uber SDP offer
>>        if that
>>        made parsing simplest.
>>
>>
>>            You also need an API to tell the browser what to actually
>>        do. The
>>            current PeerConnection approach is passing in the offer or the
>>            answer. If you're generating the answer, you need some way
>>        to tell
>>            your browser what you answered. For the "please don't send
>>        me video"
>>            case this is not an issue... it'll simply never arrive. If
>>        you want
>>            to change what the local browser is sending out, however,
>>        then it is.
>>
>>
>>        Yes, you need a "HandleLocalDescription" and
>>        "HandleRemoteDescription"
>>        API, instead of just a single OnSignalingMessage API. The
>>        deviation from
>>        the current flow is fairly minor, you just have 2 additional
>>        states in
>>        the state machine.
>>
>>
>>            I do agree it eliminates the need for an API to tell the
>> browser
>>            what kind of SDP to generate, but it also seems like it
>>        imposes a
>>            pretty big burden on application developers: even if you
>>        keep the
>>            currently-proposed PeerConnection ability to generate SDP as
>> the
>>        "simple API", the moment you want to do something slightly more
>>            complex, you have to add code to generate the appropriate
>>        SDP, which
>>            necessarily involves figuring out all the capabilities of the
>>            various browsers on various platforms. Maybe I'm naive in
>>        thinking
>>            that seems like an awful lot of work just to say, "Please
>>        don't send
>>            me video."
>>
>>
>>        It's a fair point, there is more complexity involved in
>>        generating the
>>        offer, but in our experience it is manageable. I suppose it
>>        depends on
>>        how many APIs we decide we need to control the generation of SDP
>>          (i.e.
>>        use cases other than no-video). If we decide we need to control
>>        crypto,
>>        resolution preference, codec preference, we may find this
>>        approach simpler.
>>
>>
>>            ______________________________**_____________________
>>            rtcweb mailing list
>>        [e-mail] <mailto:[e-mail]> <mailto:[e-mail]
>>
>>        <mailto:[e-mail]>>
>>
>>        https://www.ietf.org/mailman/_**___listinfo/rtcweb<https://www.ietf.org/mailman/____listinfo/rtcweb>
>>        <https://www.ietf.org/mailman/**__listinfo/rtcweb<https://www.ietf.org/mailman/__listinfo/rtcweb>
>> >
>>        <https://www.ietf.org/mailman/**__listinfo/rtcweb<https://www.ietf.org/mailman/__listinfo/rtcweb>
>>        <https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>> >>
>>
>>
>>
>>    ______________________________**___________________
>>    rtcweb mailing list
>>    [e-mail] <mailto:[e-mail]>
>>    https://www.ietf.org/mailman/_**_listinfo/rtcweb<https://www.ietf.org/mailman/__listinfo/rtcweb>
>>    <https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>> >
>>
>>
>>
>