Re: [rtcweb] Resolution negotiation - mandatory / advisory resolutions

"Roni Even" <ron.even.tlv@gmail.com> Thu, 19 April 2012 13:27 UTC

Return-Path: <ron.even.tlv@gmail.com>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0B50321F8615 for <rtcweb@ietfa.amsl.com>; Thu, 19 Apr 2012 06:27:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.49
X-Spam-Level:
X-Spam-Status: No, score=-3.49 tagged_above=-999 required=5 tests=[AWL=0.109, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sUrlLgjGIEKr for <rtcweb@ietfa.amsl.com>; Thu, 19 Apr 2012 06:26:56 -0700 (PDT)
Received: from mail-wg0-f44.google.com (mail-wg0-f44.google.com [74.125.82.44]) by ietfa.amsl.com (Postfix) with ESMTP id E827A21F861A for <rtcweb@ietf.org>; Thu, 19 Apr 2012 06:26:54 -0700 (PDT)
Received: by wgbdr13 with SMTP id dr13so5814115wgb.13 for <rtcweb@ietf.org>; Thu, 19 Apr 2012 06:26:54 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:content-transfer-encoding:x-mailer :thread-index:content-language; bh=WNPH0XrqWCXepJqqZtR+5LKZuhvVGiHr2XSnyxM288w=; b=ACrqipcWKfSgEXMtU683i81QEs2OKZiA5st6SUSKQ2NEsBp2dDpZOKI5BkQGA8yfCk xLp0ZneWkSnFP1L9+s8fBUuYmg1fQhwINH2tYfgZ0F12nb6S8jYhsnh1YFTHGoWp/rkY 3OQVfxD9YvpDig2SnbF+hPM2fh8T7JN8v4txn4NeM83VrhEw1Op9kZnkoflhAPqaXjBJ d5GdmjD6ndJvDTrXEw5iYCQ7sfNTTJcMbDdnG7GRY2zJQT8UfdLpJ1WUVcOQzoOTZ3Lx bRrYYfDhjldYRgmEomnofUv4PZvvuB/aCb0IL6XcxGoZ5VpodMhcQ/J/P355AN8QusGR w9Rg==
Received: by 10.180.97.41 with SMTP id dx9mr26379915wib.9.1334842014024; Thu, 19 Apr 2012 06:26:54 -0700 (PDT)
Received: from windows8d787f9 ([109.65.204.117]) by mx.google.com with ESMTPS id o2sm8330962wiv.11.2012.04.19.06.26.48 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 19 Apr 2012 06:26:51 -0700 (PDT)
From: "Roni Even" <ron.even.tlv@gmail.com>
To: "'Harald Alvestrand'" <harald@alvestrand.no>
References: <4F869648.2020605@alvestrand.no> <4F8FC2E0.60701@alvestrand.no> <4f8fcab8.c268b40a.7fe1.ffffb433@mx.google.com> <4F8FD12C.9070703@alvestrand.no> <4f8fe1a7.2a0db50a.23db.136b@mx.google.com> <4F8FF45C.7090003@alvestrand.no>
In-Reply-To: <4F8FF45C.7090003@alvestrand.no>
Date: Thu, 19 Apr 2012 16:25:14 +0300
Message-ID: <4f90129b.e249b40a.4436.3455@mx.google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Mailer: Microsoft Office Outlook 12.0
Thread-Index: Ac0eHhEyBEjSd7dYTmGMLj3hSAkUWwADgkhA
Content-Language: en-us
Cc: rtcweb@ietf.org
Subject: Re: [rtcweb] Resolution negotiation - mandatory / advisory resolutions
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 19 Apr 2012 13:27:01 -0000

Hi Harald,
This is a good question, my view will be that since Alice gave a preference
like profile and level (and maybe bandwidth) in the original offer than Bob
can send any video that will be within this level. The image attribute says
that Alice prefers to get something specific to help with processing (note
that Alice can limit the received video by using max-mbps and max-fs for
H.264). It may be good to discuss it with others from MMUSIC.

The usage of the image attribute in my view is more relevant to allow the
receiver to ask for a specific image size since this is not defined in H.264
(in H.263 you can specify the common specific resolutions and frame rate
using the optional parameters in RFC 4629). This was a requirement to allow
a receiver to ask for a specific size.
The usage of "wish" was because H.264 encoder is not required to support
scale to a specific resolution and a compliant decoder must support
everything within the negotiated profile and level. So when a receiver asks
for a size it can happen only if the encoder can do it but if the encoder
cannot it is still compliant.

I think that one reason for no scaling requirements on encoder is to support
simple video cameras that support one resolution and send an H.264 stream
(no scaling/cropping capabilities) and the decoder must be able to display
it on any output device (monitor).

Roni


> -----Original Message-----
> From: Harald Alvestrand [mailto:harald@alvestrand.no]
> Sent: Thursday, April 19, 2012 2:18 PM
> To: Roni Even
> Cc: rtcweb@ietf.org
> Subject: Re: [rtcweb] Resolution negotiation - mandatory / advisory
> resolutions
> 
> On 04/19/2012 11:56 AM, Roni Even wrote:
> > Hi Harald,
> > Look at example 4.2.1. The sender of the media will reject the recv
> > value in the offer and may offer other send options. Typically they
> > should be larger than the rejected value in order to allow cropping
> > and/or scaling down to the right size. (Cropping may provide better
> quality than scaling up).
> 
> Thank you - that helps a bit.
> 
> I interpret the example in 4.2.1 (second alternative) as consisting of
> 4 messages; in an offer/answer exchange, this would look like:
> 
> 1) Alice -> Bob: OFFER
>      a=imageattr:97 send [x=800,y=640,sar=1.1,q=0.6] [x=480,y=320] \
>                     recv [x=330,y=250]
> 2) Bob -> Alice: ANSWER
>      a=imageattr:97 recv [x=800,y=640,sar=1.1] \
>                     send [x=[320:16:640],y=[240:16:480],par=[1.2-1.3]]
> 3) Alice -> Bob: OFFER (re-invite, probably)
>      a=imageattr:97 send [x=800,y=640,sar=1.1] \
>                     recv [x=336,y=256]
> 4) Bob -> Alice: ANSWER
>      a=imageattr:97 recv [x=800,y=640,sar=1.1] \
>                     send [x=336,y=256]
> 
> So after message 2, Bob cannot send anything, since he can't reproduce
> what Alice asked for, right?
> 
> And after message 2, Alice can only send 800x640 with a sar of 1.1,
> right? It's forbidden for Alice to send a noncompensated image, or a
> 640x480 image?
> 
> Again, it's the use of the words "wishes" and "desires" that I want to
> be sure I understand correctly - as used in RFC 6236, they both mean
> "demand" / "require" / "MUST". Right?
> 
> > Roni
> >
> >> -----Original Message-----
> >> From: Harald Alvestrand [mailto:harald@alvestrand.no]
> >> Sent: Thursday, April 19, 2012 11:48 AM
> >> To: Roni Even
> >> Cc: rtcweb@ietf.org
> >> Subject: Re: [rtcweb] Resolution negotiation - mandatory / advisory
> >> resolutions
> >>
> >> On 04/19/2012 10:18 AM, Roni Even wrote:
> >>> Hi Harald,
> >>> The reason for the language was that this is really what the
> >>> receiver wished to receive.
> >> The strength of his wish may be the issue here....
> >>>    The issue is that at least for H.264 there is no mandatory
> >>> requirement for the encoder to support all resolutions within the
> >>> supported level. The receiver must be able to crop and scale.
> >> I couldn't parse that.
> >>
> >> Does the sender have to (MUST) send a resolution within the
> >> boundaries of the "recv" spec, or can he send something outside it?
> >> (ie if the receiver sends recv [x=320,y=200], can the sender send
> >> something with x=320,y=240, or is he completely constrained?)
> >>
> >> What does the receiver have to be able to do?
> >>
> >>                      Harald
> >>> Roni Even
> >>>
> >>>> -----Original Message-----
> >>>> From: rtcweb-bounces@ietf.org [mailto:rtcweb-bounces@ietf.org] On
> >>>> Behalf Of Harald Alvestrand
> >>>> Sent: Thursday, April 19, 2012 10:47 AM
> >>>> To: rtcweb@ietf.org
> >>>> Subject: Re: [rtcweb] Resolution negotiation - mandatory /
> advisory
> >>>> resolutions
> >>>>
> >>>> When discussing the issue of resolution negotiation locally, we
> >> found
> >>>> a
> >>>> quandary:
> >>>>
> >>>> We have two different negotiation needs:
> >>>> - Maximum resolution that's possible to use in a given context
> >>>> - The preferred resolution for a given context
> >>>>
> >>>> The current imageattr= spec (RFC 6236) doesn't specify clearly
> >>>> whether the result of the negotiation is to be considered
> mandatory
> >>>> or advisory (it uses the term "wishes" a lot), but the description
> >>>> reads most consistently if one assumes that it is a "MUST" type
> >> request.
> >>>> One interpretation is that the highest functionality available in
> >> the
> >>>> spec is the "limit", while the highest q value is the
> "preference",
> >>>> thus:
> >>>>
> >>>> recv [x=320,y=200,q=1.0] [x=200:1024,y=160:768,q=0.1]
> >>>>
> >>>> would indicate that the receiver is capable of 200x160 to
> 1024x768,
> >>>> but wants 320x200.
> >>>>
> >>>> This maps reasonably to the capabilities spec's talk about
> >> "mandatory"
> >>>> vs "optional" capabilities.
> >>>>
> >>>> Does this interpretation make sense?
> >>>>
> >>>>                          Harald
> >>>>
> >>>>
> >>>> On 04/12/2012 10:46 AM, Harald Alvestrand wrote:
> >>>>> Hello folks,
> >>>>>
> >>>>> after the IETF, I've attempted to gather together information
> >>>>> about resolution negotiation and what we might need to do there.
> >>>>>
> >>>>> I pulled together the paragraphs below - this may want to be
> >>>>> incorporated in an I-D on "SDP usage in RTCWEB", or it may want
> to
> >>>>> go somewhere else. Chairs' guidance is appreciated.
> >>>>>
> >>>>> At the moment, I have it in xml2rfc format, so it should be easy
> >>>>> to
> >>>> do
> >>>>> cut-and-paste on it, but don't want to push it as a separate I-D
> >>>>> unless that's what the chairs desire.
> >>>>> Note - there are some QUERY sections in there - I'm unsure what
> >>>>> those should be resolved to.
> >>>>>
> >>>>> Comments?
> >>>>>
> >>>>>                            Harald
> >>>>>
> >>>>> 2.  Requirements
> >>>>>
> >>>>>      The relevant requirement for video resolution negotiation
> >>>>> from
> >> the
> >>>>>      RTCWEB use cases document
> >>>>>      [I-D.ietf-rtcweb-use-cases-and-requirements] is:
> >>>>>
> >>>>>      o  F25 The browser SHOULD use encoding of streams suitable
> >>>>> for
> >> the
> >>>>>         current rendering (e.g. video display size) and SHOULD
> >> change
> >>>>>         parameters if the rendering changes during the session.
> >>>>>
> >>>>>      The resulting need is:
> >>>>>
> >>>>>      o  To negotiate a maximum spatial resolution for all videos
> >>>>> at
> >>>> call
> >>>>>         setup time
> >>>>>
> >>>>>      o  To negotiate a maximum temporal resolution ("frame rate")
> >>>> across
> >>>>>         all videos at call setup time
> >>>>>
> >>>>>      o  To indicate the desire of the recipient for a particular
> >>>> spatial
> >>>>>         or temporal resolution on a particular video source
> >>>>>
> >>>>>      o  To indicate the intent of the sender to send a video
> >>>>> source in
> >>>> a
> >>>>>         particular spatial or temporal resolution
> >>>>>
> >>>>>      This document does not mention other requirements.
> >>>> _______________________________________________
> >>>> rtcweb mailing list
> >>>> rtcweb@ietf.org
> >>>> https://www.ietf.org/mailman/listinfo/rtcweb
> >