Re: [rtcweb] Changing video size (Re: use case:)

Randell Jesup <randell-ietf@jesup.org> Wed, 06 July 2011 02:54 UTC

Return-Path: <randell-ietf@jesup.org>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 99BA421F87ED for <rtcweb@ietfa.amsl.com>; Tue, 5 Jul 2011 19:54:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.699
X-Spam-Level:
X-Spam-Status: No, score=-1.699 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, J_CHICKENPOX_15=0.6, J_CHICKENPOX_16=0.6]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mdxRtrYU7P6x for <rtcweb@ietfa.amsl.com>; Tue, 5 Jul 2011 19:54:41 -0700 (PDT)
Received: from arthur.webserversystems.com (arthur.webserversystems.com [174.132.191.98]) by ietfa.amsl.com (Postfix) with ESMTP id D1EFB21F8807 for <rtcweb@ietf.org>; Tue, 5 Jul 2011 19:54:41 -0700 (PDT)
Received: from pool-98-111-140-38.phlapa.fios.verizon.net ([98.111.140.38] helo=[192.168.1.12]) by arthur.webserversystems.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69) (envelope-from <randell-ietf@jesup.org>) id 1QeIG8-0001PM-KF for rtcweb@ietf.org; Tue, 05 Jul 2011 21:54:40 -0500
Message-ID: <4E13CDAC.40401@jesup.org>
Date: Tue, 05 Jul 2011 22:51:24 -0400
From: Randell Jesup <randell-ietf@jesup.org>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.18) Gecko/20110616 Thunderbird/3.1.11
MIME-Version: 1.0
To: rtcweb@ietf.org
References: <5DA67EA1-77DC-4F55-850C-E76E0F133A81@cisco.com> <BANLkTi=g13whA3PCXKPW5Q7a2PzEDY3ESg@mail.gmail.com> <4DFF052A.8020202@alvestrand.no> <01A17A75-0700-416E-A4D4-C6EB97265F8B@cisco.com> <46A1DF3F04371240B504290A071B4DB601301A2E@SZXEML501-MBS.china.huawei.com> <BANLkTik=N6qtPRUesk3mi62rV1Lvixu54w@mail.gmail.com> <46A1DF3F04371240B504290A071B4DB601301A8D@SZXEML501-MBS.china.huawei.com> <BBF498F2D030E84AB1179E24D1AC41D6147C4DA4CE@ESESSCMS0362.eemea.ericsson.se> <4D8B4033-8A5E-464E-82E7-983386498064@cisco.com>
In-Reply-To: <4D8B4033-8A5E-464E-82E7-983386498064@cisco.com>
Content-Type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-Transfer-Encoding: 8bit
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - arthur.webserversystems.com
X-AntiAbuse: Original Domain - ietf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - jesup.org
X-Source:
X-Source-Args:
X-Source-Dir:
X-Mailman-Approved-At: Wed, 06 Jul 2011 08:26:25 -0700
Subject: Re: [rtcweb] Changing video size (Re: use case:)
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Jul 2011 02:54:42 -0000

On 7/5/2011 6:30 PM, Cullen Jennings wrote:
> Agree with folks on this thread about this being a way for the receiver to tell the send it's preferences - Perhaps the receiver might indicate that it would like 1080P video but the best the sender can do is 720P because that is the size of camera they have.  In this case the sender does what the sender wants ignoring the receiver. There are other cases where say the receiver can not decode a bitstream above 768 kbps but the sender would typically send a stream at a much higher rate than this. In this case the sender needs to do what the receiver wants.

These sorts of issues are at least partly addressed in the O/A section 
of the H.264 spec (RFC 6184, update to RFC 3984).  Obviously that only 
applies to H.264.  There are also issues about limitations due to the 
other direction; in particular look at level-asymmetry-allowed and what 
happens if it's not used.

As mentioned, there are issues both of initial negotiation (as implied 
here) and in-call changes.

I think a) the two sides should indicate what they support receiving in 
negotiation, and b) indicate their initial preferred resolution/etc in 
negotiation.  In-call, there should be a mechanism for changing 
preference without a full call renegotiation (I suggest an RTCP-FB 
message), though one could fall back on a full new SDP exchange.

> You might have an old browser that support vp8 and newer version of the browser that does both vp8 and vp9 and you use vp8 when that is all you have but prefer vp9 if both sides support it.

That's just classic SDP and Offer/Answer:

New:
       m=video 1234 RTP/AVP 98 99 100
       a=rtpmap 98 VP9/90000
       a=rtpmap 99 VP8/90000
       a=rtpmap 100 H264/90000

Old:
       m=video 1234 RTP/AVP 99 100
       a=rtpmap 99 VP8/90000
       a=rtpmap 100 H264/90000

> I'm not sure exactly how to phrase all this but roughly speaking we need a way for the sender and receiver to agree on a format that meets the constraints of both and somewhat optimizes the desiree format within these constraints.

RFC 6236 certainly looks like a reasonable option to consider; in fact 
I'd consider it to be the
defacto choice barring some problematic limitation.  It allows asymmetry 
within the limits of the codec
negotiation; if RFC 6184 is used for H.264 then level asymmetry is allowed.

There *might* be an argument for also describing the portions of the 
decoded window that aren't visible
(per the previous conversation).

My biggest issue is that they assume it can only be changed via UPDATE 
or re-INVITE.  Maybe that's
reasonable, but I have to say I'd lean towards solutions that are direct 
end-to-end in-call to cut overhead
and lower reaction time.

   Randell

>
> On Jun 22, 2011, at 12:06 AM, Stefan Håkansson LK wrote:
>
>> All,
>>
>> this RFC could perhaps be useful for this purpose of signaling display size: http://www.rfc-editor.org/rfc/rfc6236.txt
>>
>> Stefan
>>
>> From: rtcweb-bounces@ietf.org [mailto:rtcweb-bounces@ietf.org] On Behalf Of Bert Greevenbosch
>> Sent: den 22 juni 2011 07:56
>> To: Justin Uberti
>> Cc: rtcweb@ietf.org
>> Subject: Re: [rtcweb] Changing video size (Re: use case:)
>>
>> Hi Justin, all,
>>
>> Yes, that is a second issue. Even if the receiving end can signal its current display size, the streaming source might not be able to provide it. Therefore flexibility would be needed for the source on how to handle the display size information.
>>
>> Best regards,
>> Bert
>>
>>
>> From: Justin Uberti [mailto:juberti@google.com]
>> Sent: 22 June 2011 11:59
>> To: Bert Greevenbosch
>> Cc: Cullen Jennings; Aron Rosenberg; rtcweb@ietf.org
>> Subject: Re: [rtcweb] Changing video size (Re: use case:)
>>
>> Given the fact that this request for a video size change is an optimization (either of resources or user experience), I think that SHOULD is probably the strongest label we can put on it. Consider the case of a middlebox; if it has no capacity to scale the stream, due to compute or codec restrictions, there's not much it can do with this request.
>>
>>
>> However, whether we can cater for this also depends on the actual technical solution for negotiation and which codec to use. If adding video size negotiation will not increase complexity too much, I see no reason not to do it. However, if it adds much complexity to the solution, we might need to consider whether it is worth it. Therefore, maybe it should not be a requirement, but something weaker; i.e. it could be phrased as a SHOULD rather than a MUST.
>>
>> Best regards,
>> Bert
>>
>>
>> -----Original Message-----
>> From: rtcweb-bounces@ietf.org [mailto:rtcweb-bounces@ietf.org] On Behalf Of Cullen Jennings
>> Sent: 22 June 2011 00:11
>> To: Aron Rosenberg
>> Cc: rtcweb@ietf.org
>> Subject: Re: [rtcweb] Changing video size (Re: use case:)
>>
>>
>> On Jun 20, 2011, at 1:30 , Harald Alvestrand wrote:
>>
>>> On 06/17/11 21:38, Aron Rosenberg wrote:
>>>> On Fri, Jun 17, 2011 at 11:09 AM, Cullen Jennings<fluffy@cisco.com>  wrote:
>>>>
>>>> Video Size Change.
>>>>
>>>> Alice and Bob are in a video call in their browsers and have negotiate a high resolution video. Bob decides to change the size of the windows his browser is displaying video to a small size.  Bob's browser should be able to regenerate the video codec parameters with Alice's browser to change the resolution of video Alice sends to match the smaller size.
>>>>
>>>>
>>>> Changing compression due to a user change in window size is usually not done since it has a bad long term effect on video quality. In almost all the major video calling systems, video resolution is a function of current bandwidth (which changes as the rate control system detects differences) and/or constrained by the physical device, but not a function of a user dragging the window size. At an equivalent bitrate, it is better to compress a larger resolution and display it smaller (higher PSNR), then compress a smaller resolution if you are within the codec's effective bitrate for that larger resolution.
>> Consider two different video flows using the same codec ...
>>
>> Stream A is 640 x 480 at 1mbps which is this scaled to 320x240 and displayed
>>
>> Stream B is 320 x 240 at 1mbps which is displayed at 320x240.
>>
>> My experience has been with modern video codecs that stream B will look better than stream A. As well as looking better, it will typically also have a better PSNR. There's a bunch of reasons why which are probably not worth going into here but give it a try and you will see what I mean. The key point is both streams where 1mbps. If stream B was sent at 256 kbps, then A would look better.
>>
>>
>>
>>
>> _______________________________________________
>> rtcweb mailing list
>> rtcweb@ietf.org
>> https://www.ietf.org/mailman/listinfo/rtcweb
>> _______________________________________________
>> rtcweb mailing list
>> rtcweb@ietf.org
>> https://www.ietf.org/mailman/listinfo/rtcweb
>>
>> _______________________________________________
>> rtcweb mailing list
>> rtcweb@ietf.org
>> https://www.ietf.org/mailman/listinfo/rtcweb
> _______________________________________________
> rtcweb mailing list
> rtcweb@ietf.org
> https://www.ietf.org/mailman/listinfo/rtcweb
>
>


-- 
Randell Jesup
randell-ietf@jesup.org