Re: [rtcweb] Proposal for a JS API for NoPlan (adding multiple sources without encoding them in SDP)

[Emil Ivov <[e-mail]>]

On 17.06.13, 20:57, Martin Thomson wrote:
> Maybe you'd expect me to be more supportive of something that looked
> so much like CU-RTC-Web.

No, but this doesn't mean you can't at least be constructive ...

> It inherits all the worst properties of JSEP
> (Offer/Answer, SDP editing) with a partial implementation of a clean
> API.
>
> It's comment 22-lite.  It's an abomination.  If you are going to do
> this, do it properly.

So here's your chance: could you please share your view of how this 
would look if done properly?

Emil

>
> On 17 June 2013 05:57, Peter Thatcher <[e-mail]> wrote:
>> Google is in full support of "Plan B" for encoding multiple media sources in
>> SDP, and would like to see the Plan A vs. Plan B decision resolved soon.
>> Recently, though, a third option, called "NoPlan" has been proposed, but it
>> lacked the details of what a JS API would look like for NoPlan.  Cullen
>> asked to see such an API proposal, and so I have worked with Emil to make
>> one.  He will be adding it to the NoPlan draft, but I will also include it
>> in this email.
>>
>> Again, Google is in full support of "Plan B".  But if Plan A vs. Plan B
>> cannot be decided, then we support NoPlan with the following additions to
>> the WebRTC JS API as an option that allows implementing either Plan A or
>> Plan B  in Javascript.  And even if Plan A vs. Plan B is resolved, these API
>> additions would still be a big improvement for those WebRTC applications
>> that don't use SDP for signalling.
>>
>> It is a bit long because I have added many comments and examples, but the
>> actually additions only include two new methods on PeerConnection and a few
>> new dictionaries.  So don't be overwhelmed :).
>>
>>
>>
>> Intro: This follows the model of createDataChannel, which has a JS method on
>> PeerConnection that makes it possible to add data channels without going
>> through SDP.  Furthermore, just like createDataChannel allows 2 ways to
>> handle neogitation (the "I know what I'm doing;  Here's what I want to send;
>> Let me signal everything" mode and the "please take care of it for me;  send
>> an OPEN message" mode), this also has 2 ways to handle negotiation (the "I
>> know what I'm doing; Here's what I want to send; Let me signal everything"
>> mode and the "please take care of it for me;  send SDP back and forth"
>> mode).
>>
>> Following the success of createDataChannel, this allows simple applications
>> to Just Work and more advanced applications to easily control what they need
>> to.  In particular, it's possible to use this API to implement either Plan A
>> or Plan B.
>>
>> // The following two method are added to RTCPeerConnection
>> partial interface RTCPeerConnection {
>>   // Create a stream that is used to send a source stream.
>>   // The MediaSendStream.description can be used for signalling.
>>   // No media is sent until addStream(MediaSendStream) is called.
>>   LocalMediaStream createLocalStream(MediaStream sourceStream);
>>
>>   // Create a stream that is used to receive media from the remote side,
>>   // given the parameters signalled from MedaiSendStream.description.
>>   MediaStream createRemoteStream(MediaStreamDescription description);
>> }
>>
>>
>> interface LocalMediaStream implements MediaStream {
>>    // This can be changed at any time, but especially before calling
>>    // PeerConnection.addStream
>>    attribute MediaStreamDescription description;
>> }
>>
>>
>> // Represents the parameters used to either send or receive a stream
>> // over a PeerConnection.
>> dictionary MediaStreamDescription {
>>    MediaStreamTrackDescription[] tracks;
>> }
>>
>>
>> // Represents the parameters used to either send or receive a track over //
>> a PeerConnection.  A track has many "flows", which can be grouped
>> // together.
>> dictionary MediaStreamTrackDescription {
>>    // Same as the MediaStreamTrack.id
>>    DOMString id;
>>
>>    // Same as the MediaStreamTrack.kind
>>    DOMString kind;
>>
>>    // A track can have many "flows", such as for Simulcast, FEC, etc.
>>    // And they can be grouped in arbitrary ways.
>>    MediaFlowDescription[] flows;
>>    MediaFlowGroup[] flowGroups;
>> }
>>
>> // Represents the parameters used to either send or receive a "flow"
>> // over a PeerConnection.  A "flow" is a media that arrives with a
>> // single, unique SSRC.  One to many flows together make up the media
>> // for a track.  For example, there may be Simulcast, FEC, and RTX
>> // flows.
>> dictionay MediaFlowDescription {
>>    // The "flow id" must be unique to the track, but need not be unique
>>    // outside of the track (two tracks could both have a flow with the
>>    // same flow ID).
>>    DOMString id;
>>
>>    // Each flow can go over its own transport.  If the JS sets this to a
>>    // transportId that doesn't have a transport setup already, the
>>    // browser will use SDP negotiation to setup a transport to back that
>>    // transportId.  If This is set to an MID in the SDP, then that MID's
>>    // transport is used.
>>    DOMString transportId;
>>
>>    // The SSRC used to send the flow.
>>    unsigned int ssrc;
>>
>>    // When used as receive parameters, this indicates the possible list
>>    // of codecs that might come in for this flow.  For exmample, a given
>>    // receive flow could be setup to receive any of OPUS, ISAC, or PCMU.
>>    // When used as send parameters, this indicates that the first codec
>>    // should be used, but the browser can use send other codecs if it
>>    // needs to because of either bandwidth or CPU constraints.
>>    MediaCodecDescription[] codecs;
>> }
>>
>>
>> dictionary MediaFlowGroup {
>>    DOMString type;  // "SIM" for Simulcast, "FEC" for FEC, etc
>>    DOMString[] flowids;
>> }
>>
>> dictionary MediaCodecDescription {
>>    unsigned byte payloadType;
>>    DOMString name;
>>    unsigned int? clockRate;
>>    unsigned int? bitRate;
>>    // A grab bag of other fmtp that will need to be further defined.
>>    MediaCodecParam[] params;
>> }
>>
>> dictionary MediaCodecParam {
>>    DOMString key;
>>    DOMString value;
>> }
>>
>>
>> Notes:
>>
>> - When LocalMediaStreams are added using addStream, onnegotiatedneeded is
>> not called, and those streams are never reflected in future SDP exchanges.
>> Indeed, it would be impossible to put them in the SDP without first
>> resolving if that would be Plan A SDP or Plan B SDP.
>>
>> - Just like piles of attributes would need to be defined for Plan A and for
>> Plan B, similar attributes would need to be defined here (Luckily,  much
>> work has already been done figuring out what those parameters are :).
>>
>>
>> Pros:
>>
>> - Either Plan A or Plan B or could be implemented in Javascript using this
>> API
>> - It exposes all the same functionality to the Javascript as SDP, but in a
>> much nicer format that is much easier to work with.
>> - Any other signalling mechanism, such as Jingle or CLUE could be
>> implemented using this API.
>> - There is almost no risk of signalling glare.
>> - Debugging errors with misconfigured descriptions should be much easier
>> with this than with large SDP blobs.
>>
>>
>> Cons:
>>
>> - Now there are two slightly different ways to add streams: by creating a
>> LocalMediaStream first, and not.  This is, however, analogous to setting
>> "negotiated: true" in createDataChannel.  On way is "Just Work", and the
>> other is more advanced control.
>>
>> - All the options in MediaCodecDescription are a bit complicated.  Really,
>> this is only necessary because Plan A requires being able to specify codec
>> parameters per SSRC, and set each flow on different transports.  If we did
>> not have this requirement, we could simplify.
>>
>>
>> Example Usage:
>>
>> // Imagine I have MyApp, handles creating a PeerConnection,
>> // signalling, and rendering streams.  This is how the new API could be
>> // used.
>> var peerConnection = MyApp.createPeerConnection();
>>
>> // On sender side:
>> var stream = MyApp.getMediaStream();
>> var localStream = peerConnection.createSendStream(stream);
>> sendStream.description = MyApp.modifyStream(localStream.description)
>> MyApp.signalAddStream(localStream.description, function(response)) {
>>    if (!response.rejected) {
>>      // Media will not be sent.
>>      peerConnection.addStream(localStream);
>>    }
>> }
>>
>> // On receiver side:
>> MyApp.onAddStreamSignalled = function(streamDescription) {
>>    var stream = peerConnection.createReceiveStream(streamDescription);
>>    MyApp.renderStream(stream);
>> }
>>
>>
>> // In this exchange, the MediaStreamDescription signalled from the
>> // sender to the receiver may have looked something like this:
>>
>> {
>>    tracks: [
>>    {
>>      id: "audio1",
>>      kind: "audio",
>>      flows: [
>>      {
>>        id: "main",
>>        transportId: "transport1",
>>        ssrc: 1111,
>>        codecs: [
>>        {
>>          payloadType: 111,
>>          name: "opus",
>>          // ... more codec details
>>        },
>>        {
>>          payloadType: 112,
>>          name: "pcmu",
>>          // ... more codec details
>>        }]
>>     }]
>>   },
>>   {
>>      id: "video1",
>>      kind: "video",
>>      flows: [
>>      {
>>        id: "sim0",
>>        transportId: "transport2",
>>        ssrc: 2222,
>>        codecs: [
>>        {
>>          payloadType: 122,
>>          name: "vp8"
>>          // ... more codec details
>>        }]
>>     },
>>     {
>>       id: "sim1",
>>       transportId: "transport2",
>>       ssrc: 2223,
>>       codecs: [
>>       {
>>         payloadType: 122,
>>         name: "vp8",
>>         // ... more codec details
>>       }]
>>     },
>>     {
>>       id: "sim2",
>>       transportId: "transport2",
>>       ssrc: 2224,
>>       codecs: [
>>       {
>>         payloadType: 122,
>>         name: "vp8",
>>         // ... more codec details
>>       }]
>>     },
>>
>>     {
>>       id: "sim0fec",
>>       transportId: "transport2",
>>       ssrc: 2225,
>>       codecs: [
>>       {
>>         payloadType: 122,
>>         name: "vp8",
>>         // ...
>>       }]
>>     }],
>>     flowGroups: [
>>     {
>>       semantics: "SIM",
>>       ssrcs: [2222, 2223, 2224]
>>     },
>>     {
>>       semantics: "FEC",
>>       ssrcs: [2222, 2225]
>>     }]
>>   }]
>> }
>>
>>
>> Constructive feedback is welcome :).
>>
>> _______________________________________________
>> rtcweb mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/rtcweb
>>
> _______________________________________________
> rtcweb mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

-- 
https://jitsi.org