Re: [rtcweb] Resolution negotiation - a contribution

[Marshall Eubanks <[e-mail]>]

On Thu, Apr 12, 2012 at 4:46 AM, Harald Alvestrand <[e-mail]> wrote:
> Hello folks,
>
> after the IETF, I've attempted to gather together information about
> resolution negotiation and what we might need to do there.
>
> I pulled together the paragraphs below - this may want to be incorporated in
> an I-D on "SDP usage in RTCWEB", or it may want to go somewhere else.
> Chairs' guidance is appreciated.
>
> At the moment, I have it in xml2rfc format, so it should be easy to do
> cut-and-paste on it, but don't want to push it as a separate I-D unless
> that's what the chairs desire.
> Note - there are some QUERY sections in there - I'm unsure what those should
> be resolved to.
>
> Comments?
>
>                         Harald
>
> 2.  Requirements
>
>   The relevant requirement for video resolution negotiation from the
>   RTCWEB use cases document
>   [I-D.ietf-rtcweb-use-cases-and-requirements] is:
>
>   o  F25 The browser SHOULD use encoding of streams suitable for the
>      current rendering (e.g. video display size) and SHOULD change
>      parameters if the rendering changes during the session.
>
>   The resulting need is:
>
>   o  To negotiate a maximum spatial resolution for all videos at call
>      setup time
>
>   o  To negotiate a maximum temporal resolution ("frame rate") across
>      all videos at call setup time
>

It is not clear if this is "per session" or "per browser" and that
should be made clear. Here is why it makes a difference:

Per session maximum video limits can give rise to a "race to the
bottom" in the case of a mixed multiparty conference.

Suppose, for example, you have 4 parties. One is on a low powered
phone with a low bandwidth connection, and
all they can handle is SIF at 384 kbps and 15 fps. The other three
have the power to handle 1048p at > 1 Mbps and 30 fps.

Restriction of all uses to the capability of the least-capable
participant is, for example, not
likely to go over well in mixed RTCWEB / CLUE Telepresence sessions.

This is a very common situation, and the solution is to do this negotiation
"per browser." Of course, in practice, this generally requires
middleware (4.3.3. in
draft-ietf-rtcweb-use-cases-and-requirements ).

Regards
Marshall

>   o  To indicate the desire of the recipient for a particular spatial
>      or temporal resolution on a particular video source
>
>   o  To indicate the intent of the sender to send a video source in a
>      particular spatial or temporal resolution
>
>   This document does not mention other requirements.
>
>
> 3.  SDP negotiation of video resolution
>
>   An RTCWEB client MUST support negotiation of resolution using the
>   "imageattr" attribute, as documented in [RFC6236].
>
>   An RTCWEB client MUST support a SAR value of 1.0 (square pixels), and
>   MAY choose to support only the 1.0 value of the "sar" attribute.
>
>   The interpretation of the negotiation is that any video stream in the
>   m= line containing the a=imageattr attribute will have a resolution
>   within the bounds established by the negotiation.
>
>   An RTCWEB client MUST support negotiation of the "a=framerate"
>   attribute, as specified in [RFC4566] section 6.  Note that this is an
>   upper bound on framerate; there is no lower bound negotiated.
>
>   These requirements are necessary and sufficient for establishing the
>   maximum spatial and temporal resolution for all videos at call setup
>   time.  These bounds MAY be renegotiated over the course of the call,
>   but MUST NOT be renegotiated to render any currently transmitted
>   video stream out of bounds; the sending video stream MUST first be
>   changed to be of a resolution that fits within both the old and the
>   new bounds, or halted. <<QUERY: Is this necessary or sufficient
>   restriction???>>
>
>   An RTCWEB client MUST support per-SSRC requests for video
>   resolutions, as described in
>   [I-D.lennox-mmusic-sdp-source-selection].  This satisfies the
>   requirement to indicate the desire of the recipient for a particular
>   spatial or temporal resolution.
>
>   We assume that the media sent from a sender to a receiver contains
>   enough information inside the media format to tell what the
>   resolution and framerate is. <<QUERY: Do we need to extend RFC 5576
>   with an "imageattr" or "framerate" attribute in the forward direction
>   instead?>>
>
>
> 4.  Non-SDP negotiation of video resolution
>
>   An RTCWEB client MAY support the COP mechanism
>   [I-D.westerlund-avtext-codec-operation-point] to negotiate the
>   resolution of video within the limits established by the SDP
>   negotiation without the need for additional SDP exchanges.
>
>
> 5.  Relation to WebRTC API constraint
>
>   It is intended that the resolution negotiation be influenced by the
>   constraints set by the application of either mandatory or optional
>   constraints at the WebRTC API, as registered in the registry
>   established by [I-D.burnett-rtcweb-constraints-registry].
>
>   The following relationships hold for all attributes that the
>   implementation intends to satisfy (note that the constraints listed
>   here have NOT been registered yet):
>
>   video-min-height >= value of imageattr y= xyrange lower bound
>
>   video-max-height <= value of imageattr y= xyrange upper bound
>
>   video-min-framerate is not represented in SDP
>
>   video-max-framerate <= value of a=framerate attribute
>
>   video-min-aspect-ratio <= value of imageattr "par=" prange lower
>   bound
>
>   video-max-aspect-ratio >= value of imageattr "par=" prange upper
>   bound
>
>   The implementation is free to increase "min" values or decrease "max"
>   values (make requirements more restrictive) and add "step" in order
>   to fit with its implementation restrictions. <<NOTE - check the value
>   of the direction of those assumptions>>.
>
>   Constraints specified at PeerConnection creation time are reflected
>   as SDP-wide values.  Constraints specified when creating a
>   MediaStream or attaching a MediaStream to a PeerConnection are
>   reflected as ssrc-specific values.
>
>
> _______________________________________________
> rtcweb mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/rtcweb