Re: [rtcweb] Proposal for a JS API for NoPlan (adding multiple sources without encoding them in SDP)

[Martin Thomson <[e-mail]>]

Maybe you'd expect me to be more supportive of something that looked
so much like CU-RTC-Web.  It inherits all the worst properties of JSEP
(Offer/Answer, SDP editing) with a partial implementation of a clean
API.

It's comment 22-lite.  It's an abomination.  If you are going to do
this, do it properly.

On 17 June 2013 05:57, Peter Thatcher <[e-mail]> wrote:
> Google is in full support of "Plan B" for encoding multiple media sources in
> SDP, and would like to see the Plan A vs. Plan B decision resolved soon.
> Recently, though, a third option, called "NoPlan" has been proposed, but it
> lacked the details of what a JS API would look like for NoPlan.  Cullen
> asked to see such an API proposal, and so I have worked with Emil to make
> one.  He will be adding it to the NoPlan draft, but I will also include it
> in this email.
>
> Again, Google is in full support of "Plan B".  But if Plan A vs. Plan B
> cannot be decided, then we support NoPlan with the following additions to
> the WebRTC JS API as an option that allows implementing either Plan A or
> Plan B  in Javascript.  And even if Plan A vs. Plan B is resolved, these API
> additions would still be a big improvement for those WebRTC applications
> that don't use SDP for signalling.
>
> It is a bit long because I have added many comments and examples, but the
> actually additions only include two new methods on PeerConnection and a few
> new dictionaries.  So don't be overwhelmed :).
>
>
>
> Intro: This follows the model of createDataChannel, which has a JS method on
> PeerConnection that makes it possible to add data channels without going
> through SDP.  Furthermore, just like createDataChannel allows 2 ways to
> handle neogitation (the "I know what I'm doing;  Here's what I want to send;
> Let me signal everything" mode and the "please take care of it for me;  send
> an OPEN message" mode), this also has 2 ways to handle negotiation (the "I
> know what I'm doing; Here's what I want to send; Let me signal everything"
> mode and the "please take care of it for me;  send SDP back and forth"
> mode).
>
> Following the success of createDataChannel, this allows simple applications
> to Just Work and more advanced applications to easily control what they need
> to.  In particular, it's possible to use this API to implement either Plan A
> or Plan B.
>
> // The following two method are added to RTCPeerConnection
> partial interface RTCPeerConnection {
>  // Create a stream that is used to send a source stream.
>  // The MediaSendStream.description can be used for signalling.
>  // No media is sent until addStream(MediaSendStream) is called.
>  LocalMediaStream createLocalStream(MediaStream sourceStream);
>
>  // Create a stream that is used to receive media from the remote side,
>  // given the parameters signalled from MedaiSendStream.description.
>  MediaStream createRemoteStream(MediaStreamDescription description);
> }
>
>
> interface LocalMediaStream implements MediaStream {
>   // This can be changed at any time, but especially before calling
>   // PeerConnection.addStream
>   attribute MediaStreamDescription description;
> }
>
>
> // Represents the parameters used to either send or receive a stream
> // over a PeerConnection.
> dictionary MediaStreamDescription {
>   MediaStreamTrackDescription[] tracks;
> }
>
>
> // Represents the parameters used to either send or receive a track over //
> a PeerConnection.  A track has many "flows", which can be grouped
> // together.
> dictionary MediaStreamTrackDescription {
>   // Same as the MediaStreamTrack.id
>   DOMString id;
>
>   // Same as the MediaStreamTrack.kind
>   DOMString kind;
>
>   // A track can have many "flows", such as for Simulcast, FEC, etc.
>   // And they can be grouped in arbitrary ways.
>   MediaFlowDescription[] flows;
>   MediaFlowGroup[] flowGroups;
> }
>
> // Represents the parameters used to either send or receive a "flow"
> // over a PeerConnection.  A "flow" is a media that arrives with a
> // single, unique SSRC.  One to many flows together make up the media
> // for a track.  For example, there may be Simulcast, FEC, and RTX
> // flows.
> dictionay MediaFlowDescription {
>   // The "flow id" must be unique to the track, but need not be unique
>   // outside of the track (two tracks could both have a flow with the
>   // same flow ID).
>   DOMString id;
>
>   // Each flow can go over its own transport.  If the JS sets this to a
>   // transportId that doesn't have a transport setup already, the
>   // browser will use SDP negotiation to setup a transport to back that
>   // transportId.  If This is set to an MID in the SDP, then that MID's
>   // transport is used.
>   DOMString transportId;
>
>   // The SSRC used to send the flow.
>   unsigned int ssrc;
>
>   // When used as receive parameters, this indicates the possible list
>   // of codecs that might come in for this flow.  For exmample, a given
>   // receive flow could be setup to receive any of OPUS, ISAC, or PCMU.
>   // When used as send parameters, this indicates that the first codec
>   // should be used, but the browser can use send other codecs if it
>   // needs to because of either bandwidth or CPU constraints.
>   MediaCodecDescription[] codecs;
> }
>
>
> dictionary MediaFlowGroup {
>   DOMString type;  // "SIM" for Simulcast, "FEC" for FEC, etc
>   DOMString[] flowids;
> }
>
> dictionary MediaCodecDescription {
>   unsigned byte payloadType;
>   DOMString name;
>   unsigned int? clockRate;
>   unsigned int? bitRate;
>   // A grab bag of other fmtp that will need to be further defined.
>   MediaCodecParam[] params;
> }
>
> dictionary MediaCodecParam {
>   DOMString key;
>   DOMString value;
> }
>
>
> Notes:
>
> - When LocalMediaStreams are added using addStream, onnegotiatedneeded is
> not called, and those streams are never reflected in future SDP exchanges.
> Indeed, it would be impossible to put them in the SDP without first
> resolving if that would be Plan A SDP or Plan B SDP.
>
> - Just like piles of attributes would need to be defined for Plan A and for
> Plan B, similar attributes would need to be defined here (Luckily,  much
> work has already been done figuring out what those parameters are :).
>
>
> Pros:
>
> - Either Plan A or Plan B or could be implemented in Javascript using this
> API
> - It exposes all the same functionality to the Javascript as SDP, but in a
> much nicer format that is much easier to work with.
> - Any other signalling mechanism, such as Jingle or CLUE could be
> implemented using this API.
> - There is almost no risk of signalling glare.
> - Debugging errors with misconfigured descriptions should be much easier
> with this than with large SDP blobs.
>
>
> Cons:
>
> - Now there are two slightly different ways to add streams: by creating a
> LocalMediaStream first, and not.  This is, however, analogous to setting
> "negotiated: true" in createDataChannel.  On way is "Just Work", and the
> other is more advanced control.
>
> - All the options in MediaCodecDescription are a bit complicated.  Really,
> this is only necessary because Plan A requires being able to specify codec
> parameters per SSRC, and set each flow on different transports.  If we did
> not have this requirement, we could simplify.
>
>
> Example Usage:
>
> // Imagine I have MyApp, handles creating a PeerConnection,
> // signalling, and rendering streams.  This is how the new API could be
> // used.
> var peerConnection = MyApp.createPeerConnection();
>
> // On sender side:
> var stream = MyApp.getMediaStream();
> var localStream = peerConnection.createSendStream(stream);
> sendStream.description = MyApp.modifyStream(localStream.description)
> MyApp.signalAddStream(localStream.description, function(response)) {
>   if (!response.rejected) {
>     // Media will not be sent.
>     peerConnection.addStream(localStream);
>   }
> }
>
> // On receiver side:
> MyApp.onAddStreamSignalled = function(streamDescription) {
>   var stream = peerConnection.createReceiveStream(streamDescription);
>   MyApp.renderStream(stream);
> }
>
>
> // In this exchange, the MediaStreamDescription signalled from the
> // sender to the receiver may have looked something like this:
>
> {
>   tracks: [
>   {
>     id: "audio1",
>     kind: "audio",
>     flows: [
>     {
>       id: "main",
>       transportId: "transport1",
>       ssrc: 1111,
>       codecs: [
>       {
>         payloadType: 111,
>         name: "opus",
>         // ... more codec details
>       },
>       {
>         payloadType: 112,
>         name: "pcmu",
>         // ... more codec details
>       }]
>    }]
>  },
>  {
>     id: "video1",
>     kind: "video",
>     flows: [
>     {
>       id: "sim0",
>       transportId: "transport2",
>       ssrc: 2222,
>       codecs: [
>       {
>         payloadType: 122,
>         name: "vp8"
>         // ... more codec details
>       }]
>    },
>    {
>      id: "sim1",
>      transportId: "transport2",
>      ssrc: 2223,
>      codecs: [
>      {
>        payloadType: 122,
>        name: "vp8",
>        // ... more codec details
>      }]
>    },
>    {
>      id: "sim2",
>      transportId: "transport2",
>      ssrc: 2224,
>      codecs: [
>      {
>        payloadType: 122,
>        name: "vp8",
>        // ... more codec details
>      }]
>    },
>
>    {
>      id: "sim0fec",
>      transportId: "transport2",
>      ssrc: 2225,
>      codecs: [
>      {
>        payloadType: 122,
>        name: "vp8",
>        // ...
>      }]
>    }],
>    flowGroups: [
>    {
>      semantics: "SIM",
>      ssrcs: [2222, 2223, 2224]
>    },
>    {
>      semantics: "FEC",
>      ssrcs: [2222, 2225]
>    }]
>  }]
> }
>
>
> Constructive feedback is welcome :).
>
> _______________________________________________
> rtcweb mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/rtcweb
>