Re: [rtcweb] Changing video size (Re: use case:)

[Stefan Håkansson LK <[e-mail]>]

I just want to highlight that changing video display window size is 
captured already in the very first use case ("Simple Video Communication 
Service") in 
http://datatracker.ietf.org/doc/draft-holmberg-rtcweb-ucreqs/?include_text=1. 


In the preparation of the document we also discussed putting a 
requirement on that the receiving end should explicitly signal the size 
or the required codec parameters to the sender but concluded at that 
time that more discussion is needed.

My personal view is that this kind of functionality should be there. 
After all the suitable codec settings depend a lot on e.g. the display 
size. But I'm not sure that a requirement is needed. There must be a 
signaling channel of some kind available to set up streams (to negotiate 
codecs and codec settings). I think it can be re-used for this purpose.

Setting it up this way there could initially be simpler but standard 
compliant implementations that does not support this, with support added 
later - with the web application being unaffected.

//Stefan
On 2011-06-20 16:11, Randell Jesup wrote:
> On 6/20/2011 4:30 AM, Harald Alvestrand wrote:
>> On 06/17/11 21:38, Aron Rosenberg wrote:
>>> On Fri, Jun 17, 2011 at 11:09 AM, Cullen Jennings<[e-mail]
>>> <mailto:[e-mail]>>  wrote:
>>>
>>>
>>>      Video Size Change.
>>>
>>>      Alice and Bob are in a video call in their browsers and have
>>>      negotiate a high resolution video. Bob decides to change the size
>>>      of the windows his browser is displaying video to a small size.
>>>       Bob's browser should be able to regenerate the video codec
>>>      parameters with Alice's browser to change the resolution of video
>>>      Alice sends to match the smaller size.
>>>
>>>
>>> Changing compression due to a user change in window size is usually
>>> not done since it has a bad long term effect on video quality. In
>>> almost all the major video calling systems, video resolution is a
>>> function of current bandwidth (which changes as the rate control
>>> system detects differences) and/or constrained by the physical
>>> device, but not a function of a user dragging the window size. At
>>> an equivalent bitrate, it is better to compress a larger resolution
>>> and display it smaller (higher PSNR), then compress a smaller
>>> resolution if you are within the codec's effective bitrate for that
>>> larger resolution.
>
> Don't use "most current systems" as a guide.  Just because they don't do
> it doesn't mean it's a bad idea (it might be, but more proof than their
> not doing it is needed).
>
> I would assert that in general, it's better to scale before encoding
> than to scale after decoding; and one scale operation is generally
> better than two (at camera->encoder, and decoder->display), given a
> constant bitrate.
>
>> Interesting info, but I don't know if it's true in all cases.
>>
>> If the overall constraint on a conferencing system is bandwidth, the
>> system might find the saving in average bandwidth saving of encoding
>> from a base image of 140x200 for "small" windows rather than "HD"
>> (720x1024?) worth considering, even though the sending of a complete
>> I-frame to reinitialize the video stream might take some effort.
>>
>> I like having the use case of "change recipient video window size, and
>> as a result change sender bandwidth usage"; we might argue about the
>> exact phrasing of the use case.
>>
>
> The more general issue here is one of communication: events such as
> window resize are of interest to the sender, and may cause the sender to
> change what they're sending.    Events could include bandwidth changes,
> decode resolution changes, decode framerate changes  (probably not
> common, but perhaps for managing battery use), exposure (decode partly-
> or fully-obscured, or perhaps the user has zoomed in on one portion of
> the video), etc.  How the encoder reacts to that is largely up to the
> encoder/app (and input from the decoder app on what it wants);
> transmission of the information to allow decision-making is what I'm
> concerned with.
>
> Some of these can/should be spoken to in use-cases.
>