Re: [rtcweb] Resolution negotiation - a contribution

[Harald Alvestrand <[e-mail]>]

On 04/12/2012 06:19 PM, Marshall Eubanks wrote:
> On Thu, Apr 12, 2012 at 4:46 AM, Harald Alvestrand<[e-mail]>  wrote:
>> Hello folks,
>>
>> after the IETF, I've attempted to gather together information about
>> resolution negotiation and what we might need to do there.
>>
>> I pulled together the paragraphs below - this may want to be incorporated in
>> an I-D on "SDP usage in RTCWEB", or it may want to go somewhere else.
>> Chairs' guidance is appreciated.
>>
>> At the moment, I have it in xml2rfc format, so it should be easy to do
>> cut-and-paste on it, but don't want to push it as a separate I-D unless
>> that's what the chairs desire.
>> Note - there are some QUERY sections in there - I'm unsure what those should
>> be resolved to.
>>
>> Comments?
>>
>>                          Harald
>>
>> 2.  Requirements
>>
>>    The relevant requirement for video resolution negotiation from the
>>    RTCWEB use cases document
>>    [I-D.ietf-rtcweb-use-cases-and-requirements] is:
>>
>>    o  F25 The browser SHOULD use encoding of streams suitable for the
>>       current rendering (e.g. video display size) and SHOULD change
>>       parameters if the rendering changes during the session.
>>
>>    The resulting need is:
>>
>>    o  To negotiate a maximum spatial resolution for all videos at call
>>       setup time
>>
>>    o  To negotiate a maximum temporal resolution ("frame rate") across
>>       all videos at call setup time
>>
> It is not clear if this is "per session" or "per browser" and that
> should be made clear.
Since this is negotiation, it has to be observable across the wire, 
which means that it's negotiated per SDP session - that is, per 
PeerConnection pair, which has an SDP session between them. I don't see 
how it could be otherwise.
>   Here is why it makes a difference:
>
> Per session maximum video limits can give rise to a "race to the
> bottom" in the case of a mixed multiparty conference.
>
> Suppose, for example, you have 4 parties. One is on a low powered
> phone with a low bandwidth connection, and
> all they can handle is SIF at 384 kbps and 15 fps. The other three
> have the power to handle 1048p at>  1 Mbps and 30 fps.
>
> Restriction of all uses to the capability of the least-capable
> participant is, for example, not
> likely to go over well in mixed RTCWEB / CLUE Telepresence sessions.
I break this up differently than  you do..... we might be talking about 
the same thing still.

for the mesh conferencing situation, each participant has a direct 
connection to the "weakling".
That session will need to negotiate down to what's available. It's up to 
the app to decide whether it produces multiple streams at different 
resolutions or not (and up to the browser to figure out if it can 
support that).

For the centralized scenario, either the central unit provides 
downsampling (or stripping, if the end nodes are doing some kind of AVC 
or simulcast), or the "race to the bottom" you worry about will happen.

The word "session" has become one of the words I try to avoid using 
without a qualifier....

> This is a very common situation, and the solution is to do this negotiation
> "per browser." Of course, in practice, this generally requires
> middleware (4.3.3. in
> draft-ietf-rtcweb-use-cases-and-requirements ).
>
> Regards
> Marshall
>
>>    o  To indicate the desire of the recipient for a particular spatial
>>       or temporal resolution on a particular video source
>>
>>    o  To indicate the intent of the sender to send a video source in a
>>       particular spatial or temporal resolution
>>
>>    This document does not mention other requirements.
>>
>>
>> 3.  SDP negotiation of video resolution
>>
>>    An RTCWEB client MUST support negotiation of resolution using the
>>    "imageattr" attribute, as documented in [RFC6236].
>>
>>    An RTCWEB client MUST support a SAR value of 1.0 (square pixels), and
>>    MAY choose to support only the 1.0 value of the "sar" attribute.
>>
>>    The interpretation of the negotiation is that any video stream in the
>>    m= line containing the a=imageattr attribute will have a resolution
>>    within the bounds established by the negotiation.
>>
>>    An RTCWEB client MUST support negotiation of the "a=framerate"
>>    attribute, as specified in [RFC4566] section 6.  Note that this is an
>>    upper bound on framerate; there is no lower bound negotiated.
>>
>>    These requirements are necessary and sufficient for establishing the
>>    maximum spatial and temporal resolution for all videos at call setup
>>    time.  These bounds MAY be renegotiated over the course of the call,
>>    but MUST NOT be renegotiated to render any currently transmitted
>>    video stream out of bounds; the sending video stream MUST first be
>>    changed to be of a resolution that fits within both the old and the
>>    new bounds, or halted.<<QUERY: Is this necessary or sufficient
>>    restriction???>>
>>
>>    An RTCWEB client MUST support per-SSRC requests for video
>>    resolutions, as described in
>>    [I-D.lennox-mmusic-sdp-source-selection].  This satisfies the
>>    requirement to indicate the desire of the recipient for a particular
>>    spatial or temporal resolution.
>>
>>    We assume that the media sent from a sender to a receiver contains
>>    enough information inside the media format to tell what the
>>    resolution and framerate is.<<QUERY: Do we need to extend RFC 5576
>>    with an "imageattr" or "framerate" attribute in the forward direction
>>    instead?>>
>>
>>
>> 4.  Non-SDP negotiation of video resolution
>>
>>    An RTCWEB client MAY support the COP mechanism
>>    [I-D.westerlund-avtext-codec-operation-point] to negotiate the
>>    resolution of video within the limits established by the SDP
>>    negotiation without the need for additional SDP exchanges.
>>
>>
>> 5.  Relation to WebRTC API constraint
>>
>>    It is intended that the resolution negotiation be influenced by the
>>    constraints set by the application of either mandatory or optional
>>    constraints at the WebRTC API, as registered in the registry
>>    established by [I-D.burnett-rtcweb-constraints-registry].
>>
>>    The following relationships hold for all attributes that the
>>    implementation intends to satisfy (note that the constraints listed
>>    here have NOT been registered yet):
>>
>>    video-min-height>= value of imageattr y= xyrange lower bound
>>
>>    video-max-height<= value of imageattr y= xyrange upper bound
>>
>>    video-min-framerate is not represented in SDP
>>
>>    video-max-framerate<= value of a=framerate attribute
>>
>>    video-min-aspect-ratio<= value of imageattr "par=" prange lower
>>    bound
>>
>>    video-max-aspect-ratio>= value of imageattr "par=" prange upper
>>    bound
>>
>>    The implementation is free to increase "min" values or decrease "max"
>>    values (make requirements more restrictive) and add "step" in order
>>    to fit with its implementation restrictions.<<NOTE - check the value
>>    of the direction of those assumptions>>.
>>
>>    Constraints specified at PeerConnection creation time are reflected
>>    as SDP-wide values.  Constraints specified when creating a
>>    MediaStream or attaching a MediaStream to a PeerConnection are
>>    reflected as ssrc-specific values.
>>
>>
>> _______________________________________________
>> rtcweb mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/rtcweb