Re: Multi-path QUIC Extension Experiments

[Christian Huitema <[e-mail]>]

On 7/19/2021 11:22 PM, Lucas Pardue wrote:
> Hi Christian,
>
>
>
> On Tue, 20 Jul 2021, 02:10 Christian Huitema, <[e-mail]> wrote:
>
>> In first order, you can split the multipath issue in two. On one hand,
>> the mechanics of setting up paths, acknowledging path specific data,
>> etc. On the other hand, the fine art of setting path at the right time,
>> scheduling the right amount of packets on these paths, sensing the
>> properties of the paths, etc.
>>
>> I think that the mechanics of multipath are well covered in
>> https://datatracker.ietf.org/doc/draft-liu-multipath-quic/, and I think
>> we know enough about the problem to adopt that in the WG and drive it to
>> publication.
>>
> As an individual: could you qualify a bit more about what you mean by "know
> enough"? For instance, the draft contains things like path priority and QoE
> control signals. I'm unsure if those are the solution to put on a standards
> track document, it seems like we are still figuring things out. Placing
> those aside, the proposal in the I-D does seem like it would allow
> interoperable experimentation of multipath QUIC, which is a good way to
> flush out the things we don't know. In the past, we've seen other
> proposals, like draft-deconinck,  that seem like they allow similar
> experimentation too. Has there been any progress on aligning/combining
> proposals?

Good question. I much prefer draft-liu, because it is defines fewer new 
frames and concepts and is easier to implement -- I did implement it in 
picoquic. There is possibly an argument that some of the extensions in 
draft-deconinck should be integrated in draft-liu, or maybe that some of 
the content of draft-liu should be moved out. I think adopting draft-liu 
would be force these discussions.


>> I also think that we have ample opportunities to learn more about the
>> fine arts of multipath scheduling. I love the idea of sharing as much of
>> that experience as possible between implementers, and I appreciate that
>> the Ali Baba team has been doing just that. But we probably are still
>> going to learn more about all that for several years.
>>
> Agree. Thanks for sharing the results. As above, do the findings help
> towards defining what interoperable QoE signals might be needed?

The QoE signals carry preferences from one application end to the other, 
such as "my video buffer is nearly empty, please do something to prevent 
a stall." Such signals are largely application dependent, and there will 
always be a discussion between making that a common transport frame that 
may carry application specific data versus just make it part of the 
application. I like having the QoE frame, because it is a good place to 
hook a QoE API, as in "dear application, transport just received this 
QoE frame; should it change some of what it is doing?"

Draft-liu also includes a PATH-STATUS frame, which provides generic 
controls with two parameters, a "status" and a "priority". In my 
implementation, the status field is only used to distinguish between 
path that should be dropped (status=abandon) and other paths. The 
priority field is used in the scheduling algorithm. Packets are only 
sent over the paths with high priority, unless those paths are marked as 
"suspect", for example because high packet loss has been observed.

The priority field can be used in the common scenario in which a client 
has established both a Wi-Fi and a cellular path, but would rather not 
use the cellular path if that can be avoided.


>
>
>> If we really need to publish something, we might be able to publish some
>> kind of basic scheduling algorithm, with known limitations and known
>> applicability. A bit  like we published a Reno specification for QUIC
>> and pretended that was good enough.
>
> One important thing there was to provide a default congestion controller
> that was known to be safe on the Internet. That way, QUIC implementations
> could bootstrap everything else they needed. Are there such safety concerns
> with multipath scheduling?

Define "safety"? I think there is common agreement that each path should 
be subject to its own congestion control, as specified in the base 
transport spec. Using congestion control addresses the main network 
safety concern, i.e., not causing undue congestion to other network 
users. There have been discussions in MP-TCP of "overall fairness", in 
which multipath transports would be subject to stricter congestion 
control than single path transports. For example, transports that have 
access to a good path should try not compete for capacity with 
single-path users of the other paths. Something like using Cubic or BBR 
on the main path, LEDBAT on the other paths. That could be done, with 
some extra work when the status of paths changes. But then, in QUIC, 
such restrictions would be almost impossible to enforce.

The basic algorithm for multipath scheduling is simple: consider all the 
available paths; send the next packet on a given path if congestion 
control allows it on that path. After that, transport need to take into 
account path priority and also some "health" indicator, for example 
whether high losses or high delays were observed recently. Sending data 
on a path with dubious health is likely to cause losses and delays, so 
it should either not be done at all, or only for redundant 
transmissions. Then of course there is a need for probing traffic to 
assess when paths are healthy, which can be either redundant traffic, or 
just made up traffic (picoquic just uses ping+pad). I think the baseline 
specification should stop there. After that, there are per-application 
issues, for example dealing with applications in which some traffic is 
higher priority than other -- which is what the QoE discussion is about.

-- Christian Huitema