Re: [rtcweb] Issue with "priority" defined in rtcweb-transports

[Taylor Brandstetter <[e-mail]>]

>
> For instance, screenshare-type video varies enormously in
> bandwidth (very low at static images, enormous at slide change) - if it
> competes against audio, a "relative" bitrate ratio will not be useful at
> all.


I think that's where it's important to get the terminology right; for
example, rtcweb-transports uses the term "capacity", which implies that
it's possible to use fewer bits than the capacity. Maybe that's the best
way to think of this. We also can allow some implementation flexibility.

Is there a way we can punt this?


If we do punt this, I still think we should remove the part of
rtcweb-transports that talks about the priority causing codecs to be
configured with send rates at these 1:2:4:8 ratios:

   o  When the available bandwidth is known from the congestion control
>       algorithm, configure each codec and each data channel with a
>       target send rate that is appropriate to its share of the available
>       bandwidth.


We should make it clear that the priority only affects the delivery of
packets (pacing decisions, QoS, SCTP priority), and not how the packets are
generated upstream. Assuming others agree.


On Tue, Oct 17, 2017 at 7:45 AM, Harald Alvestrand <[e-mail]>
wrote:

> Den 16. okt. 2017 20:28, skrev Taylor Brandstetter:
> > This is something that came up in the WebRTC virtual interim as well
> > (but for the simulcast use case, rather than audio vs. video). When
> > bandwidth is constrained, you don't necessarily want to send multiple
> > simulcast layers, each with a low bitrate. You want to start by sending
> > only the lowest quality encoding, until you have enough bits to send the
> > next encoding, etc.
> >
> > I think we could meet most everyone's use cases with a combination of:
> >
> >   * Priority
> >   * Min bitrate
> >   * Max bitrate
> >   * "Relative" bitrate ratio
>
> My thinking is that this grows very complex.
>
> I don't think we have much chance of getting this right for all use
> cases. For instance, screenshare-type video varies enormously in
> bandwidth (very low at static images, enormous at slide change) - if it
> competes against audio, a "relative" bitrate ratio will not be useful at
> all.
>
> >
> > For example, suppose I set up simulcast layers like so:
> >
> > encodings = [
> > {
> >   rid: "f",
> >   priority: "low",
> >   minBitrate:  600000,
> >   maxBitrate: 2500000,
> >   relativeBitrate: 4.0
> > },
> > {
> >   rid: "h",
> >   priority: "medium",
> >   scaleResolutionDownBy: 2.0,
> >   minBitrate: 150000,
> >   maxBitrate: 700000,
> >   relativeBitrate: 2.0
> > },
> > {
> >   rid: "q",
> >   priority: "high",
> >   scaleResolutionDownBy: 4.0,
> >   maxBitrate: 200000,
> >   relativeBitrate: 1.0
> > }
> > ]
> >
> > The algorithm would be "if you can't meet the minimums of every
> > encoding, prioritize the higher priority encodings." So if only 100kbps
> > is available, only the "q" (quarter-resolution) encoding would be used.
> > If the available bandwidth reaches, say, 300kbps, then it would be split
> > between "h" and "q", with a split of 200kbps/100kbps. And so on.
> >
> > I believe this is pretty similar to how our hard-coded implementation
> > already works (I took some of these values directly from the code
> I worry about the complexity of getting this specified. Having code
> helps, of course.
>
> I like extension specs. Especially for trying out things where our
> solutions may be perfect for some scenarios, but might not work in other
> scenarios, and the base specs attempt to cover all cases.
>
> Is there a way we can punt this?
>
> >
> >
> > On Mon, Oct 16, 2017 at 5:10 AM, Magnus Westerlund
> > <[e-mail] <mailto:[e-mail]>>
> > wrote:
> >
> >     Hi Taylor,
> >
> >     I think the text you quoted is directly related to your transmission
> >     queuing. And as no one was willing to write up a specific queuing
> >     algorithm that would be more flexible, the discussion in the WG at
> >     the time this was written ended up in a per prioritization level
> >     round robin queue basically, where each lower priority is given one
> >     transmission slot in the upper queue. Thus giving roughly halving,
> >     but it could be less actually depending on how many processes are
> >     part of a particular level.
> >
> >     I also think you scheme for relative capacity is likely good, but
> >     appear to be missing two important parameters. One for minimal
> >     bit-rate, and one for maximum. At least for audio sources, there is
> >     a limit to scaling these both upwards and downwards. The upper limit
> >     when reached should be redistributed, or otherwise it might be
> >     wasted. The minimal is an interesting one. Because one have to
> >     decided if this means taking more from the lower prioritise and how
> >     they relate to other. The classical example of one audio plus one
> >     video comes to mind, where one would like to maintain the audio in
> >     user interactive applications despite taking more from the video. To
> >     the point where one have to turn off video to maintain audio.
> >
> >     So, I think you have to work on the proposal a bit more.
> >
> >     Cheers
> >
> >     Magnus
> >
> >
> >     Den 2017-10-11 kl. 19:29, skrev Taylor Brandstetter:
> >>
> >>     (Mainly copied from https://github.com/w3c/webrtc-pc/issues/1625
> >>     <https://github.com/w3c/webrtc-pc/issues/1625>)
> >>
> >>     When defining priority, rtcweb-transports says:
> >>
> >>         The WebRTC prioritization model is that the application tells
> the
> >>         WebRTC endpoint about the priority of media and data that is
> >>         controlled from the API.
> >>         ...
> >>         The priority settings affect two pieces of behavior: Packet send
> >>         sequence decisions and packet markings. Each is described in
> >>         its own
> >>         section below.
> >>
> >>     The sections below are "Local prioritization" and "Usage of
> >>     Quality of Service - DSCP and Multiplexing". The "local
> >>     prioritization" section gives this guidance:
> >>
> >>         When an WebRTC endpoint has packets to send on multiple
> >>         streams that
> >>         are congestion-controlled under the same congestion control
> >>         regime,
> >>         the WebRTC endpoint SHOULD cause data to be emitted in such a
> way
> >>         that each stream at each level of priority is being given
> >>         approximately twice the transmission capacity (measured in
> payload
> >>         bytes) of the level below.
> >>
> >>         Thus, when congestion occurs, a "high" priority flow will have
> the
> >>         ability to send 8 times as much data as a "very-low" priority
> >>         flow if
> >>         both have data to send. This prioritization is independent of
> the
> >>         media type. The details of which packet to send first are
> >>         implementation defined.
> >>
> >>         For example: If there is a high priority audio flow sending
> >>         100 byte
> >>         packets, and a low priority video flow sending 1000 byte
> >>         packets, and
> >>         outgoing capacity exists for sending >5000 payload bytes, it
> >>         would be
> >>         appropriate to send 4000 bytes (40 packets) of audio and 1000
> >>         bytes
> >>         (one packet) of video as the result of a single pass of sending
> >>         decisions.
> >>
> >>     This has two significant issues I can see:
> >>
> >>       * It only allows ratios of 1:2:4:8. This is not granular enough
> >>         to be very useful. Especially when balancing transmission
> >>         capacity between audio tracks and video tracks; in practice
> >>         it's common for video tracks to use much more than 8 times the
> >>         bitrate of audio tracks.
> >>       * A single enum is used for both relative bitrate and relative
> >>         QoS priority. But in my experience it's common for an
> >>         application to want a flow that uses /less/ bitrate to have
> >>         a /higher/ QoS priority. This may even be the /more/ common
> >>         situation.
> >>
> >>     So, why do we have one enum for both things? I'd propose defining
> >>     the priority as something that only impacts the priority in
> >>     network queues (QoS-level priority and SCTP priority, as
> >>     previously discussed), and use something else to control the
> >>     relative bitrate allocation.
> >>
> >>     For example, a floating point value attached to
> >>     each |RTCEncodingParameters| (say, |relativeCapacity|), where an
> >>     encoding with twice the |relativeCapacity| of another encoding
> >>     will be given twice the transmission capacity. If implementations
> >>     can handle ratios of 1:2:4:8, I don't see any reason why they
> >>     shouldn't be able to handle arbitrary ratios.
> >>
> >>
> >>
> >>     _______________________________________________
> >>     rtcweb mailing list
> >>     [e-mail] <mailto:[e-mail]>
> >>     https://www.ietf.org/mailman/listinfo/rtcweb
> >>     <https://www.ietf.org/mailman/listinfo/rtcweb>
> >
> >     --
> >
> >     Magnus Westerlund
> >
> >     ------------------------------------------------------------
> ----------
> >     Media Technologies, Ericsson Research
> >     ------------------------------------------------------------
> ----------
> >     Ericsson AB                 | Phone  [phone]
> <tel:+46%2010%20714%2082%2087>
> >     Torshamnsgatan 23
> >     <https://maps.google.com/?q=Torshamnsgatan+23&entry=gmail&source=g>
>          | Mobile [phone] <tel:+46%2073%20094%2090%2079>
> >     SE-164 80 Stockholm, Sweden | mailto: [e-mail]
> <mailto:[e-mail]>
> >     ------------------------------------------------------------
> ----------
> >
> >
> >
> >
> > _______________________________________________
> > rtcweb mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/rtcweb
> >
>
>