Re: [rtcweb] Alternative Proposal for Dynamic Codec Parameter Change (Was: Re: Resolution negotiation - a contribution)

Magnus Westerlund <magnus.westerlund@ericsson.com> Wed, 02 May 2012 13:13 UTC

Return-Path: <magnus.westerlund@ericsson.com>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3829E21F849C for <rtcweb@ietfa.amsl.com>; Wed, 2 May 2012 06:13:10 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -105.231
X-Spam-Level:
X-Spam-Status: No, score=-105.231 tagged_above=-999 required=5 tests=[AWL=-0.782, BAYES_00=-2.599, HELO_EQ_SE=0.35, J_CHICKENPOX_53=0.6, J_CHICKENPOX_55=0.6, J_CHICKENPOX_56=0.6, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0CI6q5cAN2B4 for <rtcweb@ietfa.amsl.com>; Wed, 2 May 2012 06:13:09 -0700 (PDT)
Received: from mailgw2.ericsson.se (mailgw2.ericsson.se [193.180.251.37]) by ietfa.amsl.com (Postfix) with ESMTP id 25A8321F8496 for <rtcweb@ietf.org>; Wed, 2 May 2012 06:13:08 -0700 (PDT)
X-AuditID: c1b4fb25-b7b18ae000000dce-26-4fa132e3b793
Received: from esessmw0247.eemea.ericsson.se (Unknown_Domain [153.88.253.125]) (using TLS with cipher AES128-SHA (AES128-SHA/128 bits)) (Client did not present a certificate) by mailgw2.ericsson.se (Symantec Mail Security) with SMTP id 19.F3.03534.3E231AF4; Wed, 2 May 2012 15:13:08 +0200 (CEST)
Received: from [127.0.0.1] (153.88.115.8) by esessmw0247.eemea.ericsson.se (153.88.115.94) with Microsoft SMTP Server id 8.3.213.0; Wed, 2 May 2012 15:13:07 +0200
Message-ID: <4FA132E3.4020703@ericsson.com>
Date: Wed, 2 May 2012 15:13:07 +0200
From: Magnus Westerlund <magnus.westerlund@ericsson.com>
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:12.0) Gecko/20120420 Thunderbird/12.0
MIME-Version: 1.0
To: Justin Uberti <juberti@google.com>
References: <4F869648.2020605@alvestrand.no> <4F96B7C9.1030609@ericsson.com> <CAOJ7v-0ZPxxPEpr6-r8tUyiWJ0MJz47s59kD2tiUPo7Tkj9qcA@mail.gmail.com> <4F980BA6.7070405@ericsson.com> <4F9822C2.7090602@alvestrand.no> <4F9E57BF.5000902@ericsson.com> <CAOJ7v-3NAgcRtpmUwqqcHDWjQaPtk6XYKrhbD_7Ja19QFCmq1Q@mail.gmail.com>
In-Reply-To: <CAOJ7v-3NAgcRtpmUwqqcHDWjQaPtk6XYKrhbD_7Ja19QFCmq1Q@mail.gmail.com>
X-Enigmail-Version: 1.4.1
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-Brightmail-Tracker: AAAAAA==
Cc: "rtcweb@ietf.org" <rtcweb@ietf.org>
Subject: Re: [rtcweb] Alternative Proposal for Dynamic Codec Parameter Change (Was: Re: Resolution negotiation - a contribution)
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 02 May 2012 13:13:10 -0000

On 2012-05-01 05:02, Justin Uberti wrote:

> 
>     According to my Colleague Per-Erik you can use video.width and
>     video.height to set the resolution the video element should have.
> 
>     see
>     http://dev.w3.org/html5/spec/the-video-element.html#the-video-element
> 
>     Thus the code you presented should be something like.
> 
>     video.src = webkitURL.createObjectURL(s);
>     video.width = 300;
>     video.height = 300;
>     canvas.getContext("2d").drawImage(video, 0, 0, 300, 300);
> 
>     That would set the video objects size to a 300, 300 video resolution.
>     The drawImage call I have removed the part which selects a cropping of
>     the source video to automatically show the full video in the specified
>     area.
> 
>     The adaptation of the video will of course not happen instantly so until
>     that happens the video will be scaled to the specified width and height.
>     And the actual delivered resolution will depend on the COP request and
>     what the sender support.
> 
> 
> I think this is a reasonable way to approach the problem as long as a
> <video/> element stays in the path. But as Harald mentions we have
> already considered several usages where a <video/> would not be in the
> path, i.e. connecting a MediaStream to a file, or perhaps as input to
> another PeerConnection.

I think in both these cases there is a need to take the information from
these sinks about what they request/desire in resolution and apply that
towards upstream. When it comes to relaying, this is no different what a
conference mixer will need to do in a multi-party session. Take in the
request from the multiple consumers of the media. Merge them into the
most suitable request and then send that to the media source.

For recording, which I don't think there exist a standardized interface
yet, it will need to indicate at what quality level the recording should
happen. I know that Per-Erik commented on this back in 2010, see:
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-November/029069.html

To my understanding this discussion was part of what resulted in the
removal of the proposed recorder interface.


> 
> Perhaps we need to provide hooks directly on the MediaStream to allow
> the resolution to be specified, and these hooks are invoked
> automatically when the MediaStream is connected to a <video/>.
> 

When it comes to video resolution, I still haven't seen an case where
the sink for the video still aren't able to specify a preferred resolution.

As I see it this isn't only about resolution for video. There is a bunch
of other parameters that also needs to be considered and they are
different depending on media type, audio, video, real-time text etc.

We are working on a Internet draft that goes into more details as a
response to Harald's draft-alvestrand-rtcweb-resolution-00. However, it
will take some more working days to complete and go through Ericsson's
internal process.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: magnus.westerlund@ericsson.com
----------------------------------------------------------------------