Re: [rtcweb] Resolution negotiation - a contribution

Marshall Eubanks <marshall.eubanks@gmail.com> Thu, 12 April 2012 16:19 UTC

Return-Path: <marshall.eubanks@gmail.com>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CBD5821F869E for <rtcweb@ietfa.amsl.com>; Thu, 12 Apr 2012 09:19:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -103.297
X-Spam-Level:
X-Spam-Status: No, score=-103.297 tagged_above=-999 required=5 tests=[AWL=-0.298, BAYES_00=-2.599, J_CHICKENPOX_19=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uTRPahXMg8hZ for <rtcweb@ietfa.amsl.com>; Thu, 12 Apr 2012 09:19:18 -0700 (PDT)
Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) by ietfa.amsl.com (Postfix) with ESMTP id 76D3A21F864D for <rtcweb@ietf.org>; Thu, 12 Apr 2012 09:19:15 -0700 (PDT)
Received: by lbok13 with SMTP id k13so1881823lbo.31 for <rtcweb@ietf.org>; Thu, 12 Apr 2012 09:19:14 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=8y9F5qPMixS4pmagmgNV6fmI/oIGNl5aNY2klCX7mY8=; b=sISBaLIUEhuG5IghDit72a/uO4ZLFgrwJbmBqhEmjDevPxyHGflE1qBwl44dcRxK0o meHj3a8WL3S+u3NjlCPoHqcSK3Gy0kmfIMqyna9vYJcB/JH9T64+FxoKp8xa4kJPAkOP zNySdoJlHL8kxXa0hOU3g7KqVesgj1IR0pynpwmrt4JurVLPcIQ/2UC0rTy2ST9dUGKv WIqV7R0VDvzCenZAHK2UpvIRBDbpblt9rom8rkag08dfk556BfJQjO+eTpCfaTzI6ppE j2RSQBcdg0npNtuebWJGbF4Fjrx935VJ7QJ5px7oPxlfqs8FnTHarnO27Y1m1fng9NrV G7vw==
MIME-Version: 1.0
Received: by 10.152.132.166 with SMTP id ov6mr2964479lab.35.1334247554381; Thu, 12 Apr 2012 09:19:14 -0700 (PDT)
Received: by 10.112.46.4 with HTTP; Thu, 12 Apr 2012 09:19:14 -0700 (PDT)
In-Reply-To: <4F869648.2020605@alvestrand.no>
References: <4F869648.2020605@alvestrand.no>
Date: Thu, 12 Apr 2012 12:19:14 -0400
Message-ID: <CAJNg7VLAn8_53mYu3Y0CoaYryrJec92zZrv9srfMCc0rZr6VVg@mail.gmail.com>
From: Marshall Eubanks <marshall.eubanks@gmail.com>
To: Harald Alvestrand <harald@alvestrand.no>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: "rtcweb@ietf.org" <rtcweb@ietf.org>
Subject: Re: [rtcweb] Resolution negotiation - a contribution
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Apr 2012 16:19:18 -0000

On Thu, Apr 12, 2012 at 4:46 AM, Harald Alvestrand <harald@alvestrand.no> wrote:
> Hello folks,
>
> after the IETF, I've attempted to gather together information about
> resolution negotiation and what we might need to do there.
>
> I pulled together the paragraphs below - this may want to be incorporated in
> an I-D on "SDP usage in RTCWEB", or it may want to go somewhere else.
> Chairs' guidance is appreciated.
>
> At the moment, I have it in xml2rfc format, so it should be easy to do
> cut-and-paste on it, but don't want to push it as a separate I-D unless
> that's what the chairs desire.
> Note - there are some QUERY sections in there - I'm unsure what those should
> be resolved to.
>
> Comments?
>
>                         Harald
>
> 2.  Requirements
>
>   The relevant requirement for video resolution negotiation from the
>   RTCWEB use cases document
>   [I-D.ietf-rtcweb-use-cases-and-requirements] is:
>
>   o  F25 The browser SHOULD use encoding of streams suitable for the
>      current rendering (e.g. video display size) and SHOULD change
>      parameters if the rendering changes during the session.
>
>   The resulting need is:
>
>   o  To negotiate a maximum spatial resolution for all videos at call
>      setup time
>
>   o  To negotiate a maximum temporal resolution ("frame rate") across
>      all videos at call setup time
>

It is not clear if this is "per session" or "per browser" and that
should be made clear. Here is why it makes a difference:

Per session maximum video limits can give rise to a "race to the
bottom" in the case of a mixed multiparty conference.

Suppose, for example, you have 4 parties. One is on a low powered
phone with a low bandwidth connection, and
all they can handle is SIF at 384 kbps and 15 fps. The other three
have the power to handle 1048p at > 1 Mbps and 30 fps.

Restriction of all uses to the capability of the least-capable
participant is, for example, not
likely to go over well in mixed RTCWEB / CLUE Telepresence sessions.

This is a very common situation, and the solution is to do this negotiation
"per browser." Of course, in practice, this generally requires
middleware (4.3.3. in
draft-ietf-rtcweb-use-cases-and-requirements ).

Regards
Marshall

>   o  To indicate the desire of the recipient for a particular spatial
>      or temporal resolution on a particular video source
>
>   o  To indicate the intent of the sender to send a video source in a
>      particular spatial or temporal resolution
>
>   This document does not mention other requirements.
>
>
> 3.  SDP negotiation of video resolution
>
>   An RTCWEB client MUST support negotiation of resolution using the
>   "imageattr" attribute, as documented in [RFC6236].
>
>   An RTCWEB client MUST support a SAR value of 1.0 (square pixels), and
>   MAY choose to support only the 1.0 value of the "sar" attribute.
>
>   The interpretation of the negotiation is that any video stream in the
>   m= line containing the a=imageattr attribute will have a resolution
>   within the bounds established by the negotiation.
>
>   An RTCWEB client MUST support negotiation of the "a=framerate"
>   attribute, as specified in [RFC4566] section 6.  Note that this is an
>   upper bound on framerate; there is no lower bound negotiated.
>
>   These requirements are necessary and sufficient for establishing the
>   maximum spatial and temporal resolution for all videos at call setup
>   time.  These bounds MAY be renegotiated over the course of the call,
>   but MUST NOT be renegotiated to render any currently transmitted
>   video stream out of bounds; the sending video stream MUST first be
>   changed to be of a resolution that fits within both the old and the
>   new bounds, or halted. <<QUERY: Is this necessary or sufficient
>   restriction???>>
>
>   An RTCWEB client MUST support per-SSRC requests for video
>   resolutions, as described in
>   [I-D.lennox-mmusic-sdp-source-selection].  This satisfies the
>   requirement to indicate the desire of the recipient for a particular
>   spatial or temporal resolution.
>
>   We assume that the media sent from a sender to a receiver contains
>   enough information inside the media format to tell what the
>   resolution and framerate is. <<QUERY: Do we need to extend RFC 5576
>   with an "imageattr" or "framerate" attribute in the forward direction
>   instead?>>
>
>
> 4.  Non-SDP negotiation of video resolution
>
>   An RTCWEB client MAY support the COP mechanism
>   [I-D.westerlund-avtext-codec-operation-point] to negotiate the
>   resolution of video within the limits established by the SDP
>   negotiation without the need for additional SDP exchanges.
>
>
> 5.  Relation to WebRTC API constraint
>
>   It is intended that the resolution negotiation be influenced by the
>   constraints set by the application of either mandatory or optional
>   constraints at the WebRTC API, as registered in the registry
>   established by [I-D.burnett-rtcweb-constraints-registry].
>
>   The following relationships hold for all attributes that the
>   implementation intends to satisfy (note that the constraints listed
>   here have NOT been registered yet):
>
>   video-min-height >= value of imageattr y= xyrange lower bound
>
>   video-max-height <= value of imageattr y= xyrange upper bound
>
>   video-min-framerate is not represented in SDP
>
>   video-max-framerate <= value of a=framerate attribute
>
>   video-min-aspect-ratio <= value of imageattr "par=" prange lower
>   bound
>
>   video-max-aspect-ratio >= value of imageattr "par=" prange upper
>   bound
>
>   The implementation is free to increase "min" values or decrease "max"
>   values (make requirements more restrictive) and add "step" in order
>   to fit with its implementation restrictions. <<NOTE - check the value
>   of the direction of those assumptions>>.
>
>   Constraints specified at PeerConnection creation time are reflected
>   as SDP-wide values.  Constraints specified when creating a
>   MediaStream or attaching a MediaStream to a PeerConnection are
>   reflected as ssrc-specific values.
>
>
> _______________________________________________
> rtcweb mailing list
> rtcweb@ietf.org
> https://www.ietf.org/mailman/listinfo/rtcweb