Re: [rtcweb] My Opinion: Why I think a negotiating protocol is a Good Thing

[Hadriel Kaplan <[e-mail]>]

On Oct 21, 2011, at 12:35 AM, Cullen Jennings wrote:
> 
> I'm happy to ignore T.38 as a "special case" but new codecs, video codecs in particular, need to define new parameters. Lets say a given codec defines some new parameters that show up in some normal spot in SDP, I don't see how the JS app will be able to negotiation without understanding what the new parameters mean. Lets say some new VP9 video codec defines a new maxFluffyDepth and the browser reports that it support 666. The other other side offers a maxFlufflyDepth=Yes. What does the Javascript code select? It might be that the based on these you even use a different parameter, say fluffyDepth=-6ft , to specify the interoperable solution. 

I'm not sure you're following what I meant, or else I'm not following you.  
If a new codec VP9 needed to indicate a "maxFluffyDepth" and it supports a value of "666", then when JS calls a getter on the API for available codec info, in the array-list of returned codecs it sees something like this:

returnedArrayOfLocalVideoCodecs = [
	{ "payload-type": 99, "name": "H263-2000", "clock" : 90000, "fmtp": "CIF=4;QCIF=2;F=1;K=1" },
	{ "payload-type": 100, "name": "VP9", "clock": 64000, "fmtp": "maxFluffyDepth=666" },
	...
]

(that's not actually exactly what I would do, BTW, but just for sake of simplicity)

JS could, if it so wished, only use entries in that array it understood.  OR, it could use any/all of them.  How it uses them exactly is app-specific.  It could stringify this into JSON, etc.

But let's just say it wants to turn this into SDP and use offer/answer.  Lucky for us there's a sdpForDummies.js library which does that, so this is all in that library.  The library knows nothing about codec VP9, but it knows how to produce attributes and such using the fields defined in the API, so it creates this:

[some SDP lines missing because I'm lazy]

m=video 1234 RTP/AVPF 99 100
a=rtpmap:99 H263-2000/90000
a=fmtp:99 CIF=4;QCIF=2;F=1;K=1
a=rtpmap:100 VP9/64000
a=fmtp:100 maxFluffyDepth=666

Then let's say it gets an SDP answer back of this:
m=video 5678 RTP/AVPF 100
a=rtpmap:100 VP9/64000
a=fmtp:100 maxFluffyDepth=Yes

So sdpForDummies.js understands the SDP offer/answer model, but not the codec chosen.  But it does what it does in every case: it parses it, sees the fmtp stuff and treats that as a opaque string, and calls a Browser API setter function like 
	MediaStream.setRemoteCodec(1, 100, 64000, "maxFluffyDepth=Yes"); 
	// the first arg=1 is the array index of codecs for "VP9"

You can say "well isn't that like an answer/offer just with SDP parsed out as Javascript integral data types?"  And I would say no it's not - the Browser's maintaining no offer/answer state, it's not handling forking, it's not handling reversion in case of failure, it's not parsing protocol messages (ie, SDP), and it's not making decisions unbeknownst to the JavaScript.  In short: it's an API rather than a protocol. 

Will there be some future case that could require the JS to change when the Browser does something new?  Of course.  But JS developers are incredibly fast at changing code and getting it deployed/used compared to timescales of Browser changes.  If the JS app developer wants to enable some new doohicky that really did new things differently in an incompatible way, then the JS developer will do it.  We don't need to tell them how to do their job.  We need to *let* them do their job, and their job is code.

-hadriel
p.s. as a side benefit, doing this might also keep us honest in MMUSIC about how we encode things and handle SDP offer/answer consistently across codecs.  :)