Re: [rtcweb] Alternative Proposal for Dynamic Codec Parameter Change (Was: Re: Resolution negotiation - a contribution)

Magnus Westerlund <> Mon, 30 April 2012 09:13 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 923EC21F8557 for <>; Mon, 30 Apr 2012 02:13:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -105.233
X-Spam-Status: No, score=-105.233 tagged_above=-999 required=5 tests=[AWL=-0.784, BAYES_00=-2.599, HELO_EQ_SE=0.35, J_CHICKENPOX_53=0.6, J_CHICKENPOX_55=0.6, J_CHICKENPOX_56=0.6, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id gJ2LqFDKNfzj for <>; Mon, 30 Apr 2012 02:13:44 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id 785B921F849A for <>; Mon, 30 Apr 2012 02:13:43 -0700 (PDT)
X-AuditID: c1b4fb30-b7b07ae000006839-3e-4f9e57c650ca
Authentication-Results: x-tls.subject="/CN=esessmw0197"; auth=fail (cipher=AES128-SHA)
Received: from (Unknown_Domain []) (using TLS with cipher AES128-SHA (AES128-SHA/128 bits)) (Client CN "esessmw0197", Issuer "esessmw0197" (not verified)) by (Symantec Mail Security) with SMTP id D1.C1.26681.6C75E9F4; Mon, 30 Apr 2012 11:13:42 +0200 (CEST)
Received: from [] ( by ( with Microsoft SMTP Server id; Mon, 30 Apr 2012 11:13:42 +0200
Message-ID: <>
Date: Mon, 30 Apr 2012 11:13:35 +0200
From: Magnus Westerlund <>
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:12.0) Gecko/20120420 Thunderbird/12.0
MIME-Version: 1.0
To: Harald Alvestrand <>
References: <> <> <> <> <>
In-Reply-To: <>
X-Enigmail-Version: 1.4.1
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-Brightmail-Tracker: AAAAAA==
Cc: "" <>
Subject: Re: [rtcweb] Alternative Proposal for Dynamic Codec Parameter Change (Was: Re: Resolution negotiation - a contribution)
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 30 Apr 2012 09:13:47 -0000

On 2012-04-25 18:13, Harald Alvestrand wrote:
> On 04/25/2012 04:35 PM, Magnus Westerlund wrote:
>> On 2012-04-25 05:20, Justin Uberti wrote:
>>> On Tue, Apr 24, 2012 at 10:25 AM, Magnus Westerlund
>>> <<>>
>>> wrote:
>>>      A) Using COP and using SDP based signaling for the dynamic changes are
>>>      two quite different models in relation to how the interaction happens.
>>>      For COP this all happens in the browser, normally initiated by the
>>>      browser's own determination that a COP request or notification is
>>>      needed. Harald's proposal appears to require that the JS initiate a
>>>      renegotiation. This puts a requirement on the implementation to listen
>>>      to the correct callbacks to know when changes happens, such as window
>>>      resize. To my knowledge there are not yet any proposals for how the
>>>      browser can initiate a JSEP renegotiation.
>>> Maybe I misunderstand what you are saying, but the application will
>>> definitely know when the browser size changes, and can then trigger a
>>> JSEP renegotiation by calling createOffer and shipping it off.
>> Yes, they can but that requires them to write code to do this. With COP
>> the browser can do it and it works well for all applications, not only
>> the ones that have implementors that are aware of the need to do this.
>> I am proposing a mechanism that makes the default behavior if you care
>> nothing about controlling or optimizing this behavior in your JS to be
>> good. In addition JS that wants to control it can affect the behavior
>> through generic mechanisms, such as controlling the video element. And
>> if clear need is shown and some functionality is missing, W3C can create
>> an API to provide explicit control.
> It's not at all clear to me that the browser being in control is right 
> for all cases.
> Renegotiating has a cost, and some of the costs are known only to the 
> application.
> We may possibly handle this via constraints like 
> "VideoEnumRenegotiationTendency" or something like that - when you get 
> the proposal together more, I'd like to see the controls you're proposing.
>>> I have the opposite concern - how can the browser know when the
>>> application makes a change, for instance to display a participant in a
>>> large view versus a small view? This may be possible if using a<video/>
>>> for display, but I don't think it will be possible when using WebGL for
>>> rendering, where the size of the view will be dictated only by the
>>> geometry of the scene.
>> What I understand from my colleagues is that all rendering of video have
>> to go through a video element, even if not displayed directly. Thus the
>> application can control the size of the video element used and possibly
>> consumed by WebGL to be included in rendering. Thus the application will
>> have control over this even when COP is being used.
> We have on the "list of things that need doing" the connection of a 
> mediastream directly to a canvas, a WebGL element or being captured to a 
> file, so don't gamble too much on the video element to stay in the path.
> Example code for capturing video to a canvas (taken from a real app):
>         video.src = webkitURL.createObjectURL(s);
>         canvas.getContext("2d").drawImage(video, 0, 0, 300, 300, 0, 0, 
> 300, 300);

According to my Colleague Per-Erik you can use video.width and
video.height to set the resolution the video element should have.


Thus the code you presented should be something like.

video.src = webkitURL.createObjectURL(s);
video.width = 300;
video.height = 300;
canvas.getContext("2d").drawImage(video, 0, 0, 300, 300);

That would set the video objects size to a 300, 300 video resolution.
The drawImage call I have removed the part which selects a cropping of
the source video to automatically show the full video in the specified

The adaptation of the video will of course not happen instantly so until
that happens the video will be scaled to the specified width and height.
And the actual delivered resolution will depend on the COP request and
what the sender support.

>> I agree that it needs to be clearly specified. But COP appears to be
>> less prone to complete failures by using target values. Then at least it
>> will receive video that can be used, even if not optimal. imageattr can
>> result in failures in the negotiation requiring at minimal a second
>> round potentially no agreed on acceptable resolution then that is
>> clearly sub-optimal. That is even without considering involving JS
>> mainipulating this.
> I certainly hope that we can avoid the need for SDP manipulation of JS 
> in all of these proposals.

Looking forward to a bit more detailed explanation of how that is
accomplished in relation to the API.


Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVM
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: