Re: [Mops] Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC

["Holland, Jake" <[e-mail]>]

GREASE-reordering is a brilliant idea, +1 to that.  (Particularly for writes that need to be broken up into packets anyway, and probably also for anything still buffered when more writes come in.)

If the end result is that UDP 443 on server side is excluded from reordering buffers, that’s a clear win regardless of any other network decisions that may or may not happen.

Best regards,
Jake

From: Ian Swett <ianswett=[e-mail]>
Date: Thu,2021-07-29 at 7:43 PM
To: Zaheduzzaman Sarker <[e-mail]>
Cc: Sergio Garcia Murillo <[e-mail]>, "[e-mail]" <[e-mail]>, Roberto Peon <[e-mail]>, Matt Joras <[e-mail]>, "Das, Dibakar" <[e-mail]>, Alan Frindell <[e-mail]>, "[e-mail]" <[e-mail]>, Anna Brunström <[e-mail]>, David Schinazi <[e-mail]>, "[e-mail]" <[e-mail]>, "[e-mail]" <[e-mail]>, Kirill Pugin <[e-mail]>
Subject: Re: [Mops] Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC

TLDR: Ideally, handling reordering on the receiving host will reduce user latency and be cheaper for everyone.

Changing the TCP ecosystem is slow, so I can't advise immediately doing this for TCP traffic, but doing this now for QUIC(and maybe documenting it in Manageability?) seems viable.  If we wait longer, the ecosystem will be ossified.  I'm not sure anyone is willing to introduce intentional reordering to GREASE the loss detection system and ensure implementations adapt, but I'll discuss if that's something we would do for some traffic when sending multi-packet bursts.

I believe the vast majority of QUIC applications would benefit from increased reordering if it reduced latency and buffering.  In TCP, you can detect reordering from the sequence number.  But for QUIC, the metadata is encrypted, so a given network can only attempt to reduce reordering in a much narrower context, which is much less useful.  Add into that the fact you don't know the application, and the benefits of delaying packets within the network/path become even more difficult to reason about.

To frame this in statistical terms, the sum of multiple normal distributions is a normal distribution with the sum of the means and a sum of the variances(https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables<https://urldefense.com/v3/__https:/en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables__;!!GjvTz_vk!CYEeHbhK7C7RyDCja4rhiCVtdTRzsDAvDFa_mKJVustvWtuOr6WXbuj53eCLHh4$>).  Since the standard deviation is the square root of the variance, the standard deviation is dominated by the largest source of variation, and if each link introduces some variance, it's really only the one that introduces the most variation that matters.  Given no link knows whether it's the dominant source of variation on the path, I'd suggest every link pass along packets as soon as possible.  Obviously, link-layer error correction and other approaches which increase reliability without introducing delay are still welcome and valuable.

Thanks, Ian



On Thu, Jul 29, 2021 at 2:10 PM Zaheduzzaman Sarker <[e-mail]<mailto:[e-mail]>> wrote:
I would say this is not specific to media over QUIC. However, a potential solution to this could boost the deployment of potential radio technologies ( for example - Unacknowledgement Mode (UM) of RLC in cellular networks) which might actually help low latency media usecases.

BR
Zahed

On 2021-07-29, 22:35, "QUIC on behalf of Sergio Garcia Murillo" <[e-mail]<mailto:[e-mail]> on behalf of [e-mail]<mailto:[e-mail]>> wrote:

Just for clarification, if this needs to be handled, this should be handled at QUIC level and it is not an specific problem for the Media over QUIC protocol, right?

El jue., 29 jul. 2021 18:15, Holland, Jake <jholland=[e-mail]<mailto:[e-mail]>> escribió:
Hi Anna,

(And replacing Nathalie’s email with the correct one, hopefully)

That was not quite my understanding, no.  Sorry if I opened this discussion poorly.  In Nathalie’s initial response, she clarified that it’s configurable, including the option to disable the reordering buffer.

However, I understood this as an option available to the service provider operating the MP-DCCP split path proxies that tunnel the carried traffic, not necessarily to the endpoints or the applications having their traffic split.

Depending on how it’s deployed, this may or may not be a problem for particular endpoints, but I couldn’t tell if there was an obvious or easy way for an application to select the use case (particularly one that is just trying to use QUIC and transparently getting traffic-splitting delivery) in a way that the operator will understand, so I thought it best to raise the point for discussion.

Best,
Jake

From: Anna Brunström <[e-mail]<mailto:[e-mail]>>
Date: Wed,2021-07-28 at 5:31 PM
To: Matt Joras <[e-mail]<mailto:[e-mail]>>, "Holland, Jake" <[e-mail]<mailto:[e-mail]>>
Cc: David Schinazi <[e-mail]<mailto:[e-mail]>>, Roberto Peon <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, Ian Swett <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "Das, Dibakar" <[e-mail]<mailto:[e-mail]>>, Alan Frindell <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, Kirill Pugin <[e-mail]<mailto:[e-mail]>>
Subject: RE: [Mops] Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC

I think multipath and L2 are a bit different as multipath may lead to large amounts of reordering and delay variations. Not all applications/implementations may handle this well. But you should of course also have the possibility to just pass the packets on, and I agree that in many cases this will be the best.

In the context of MP-DCCP, both scheduling and reordering are modular components so you can certainly choose to pass the packets on without delay. The need for supporting this is also stated in the draft. But for the multipath case, I think it is useful to also have the ability to reduce reordering if needed. For ATSSS I guess weather that is desirable or not may be controlled by the selected mode.

@Jake: if you understood our conversation in ANRW as suggesting that MP-DCCP should always reorder, that was a misunderstanding.

Best,
Anna


From: Matt Joras <[e-mail]<mailto:[e-mail]>>
Sent: den 29 juli 2021 01:58
To: Holland, Jake <jholland=[e-mail]<mailto:[e-mail]>>
Cc: David Schinazi <[e-mail]<mailto:[e-mail]>>; Roberto Peon <[e-mail]<mailto:[e-mail]>>; Anna Brunström <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; Ian Swett <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>; Das, Dibakar <[e-mail]<mailto:[e-mail]>>; Alan Frindell <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>; Kirill Pugin <[e-mail]<mailto:[e-mail]>>
Subject: Re: [Mops] Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC

Speaking as an individual, a couple comments inline adding to what Jake is saying.

On Wed, Jul 28, 2021 at 3:58 PM Holland, Jake <jholland=[e-mail]<mailto:[e-mail]>> wrote:
And yet, we still can see network buffering to maintain packet ordering in new work today, including work specifically targeted at QUIC.

In the “CCID5” ANRW talk from Monday’s 2nd session, a reordering buffer was mentioned in the transparent proxy for the MP-DCCP tunnel they’re wrapping QUIC packets in to get multipath traffic-splitting during transport within wireless carriers (a proposal coming soon to tsvwg):
https://irtf.org/anrw/2021/program.html#:~:text=nathalie%20romo%20moreno<https://urldefense.com/v3/__https:/irtf.org/anrw/2021/program.html*:*:text=nathalie*20romo*20moreno__;I34lJQ!!GjvTz_vk!AVpyqSAj3HYNjnpUTryB_6DdElZmzNy4D78JApMaTuSanBUHBhhvFtmh1HJdILQ$>

I asked specifically about the reordering buffer at the mic, since this kind of thing has made me sad before. (There was also a side discussion in the chat.)  I got the impression they believe they’re seeing some benefits to apps by including this reordering buffer in the network, citing the wider range of reordering that you get for split-path transport (as compared to the L2 forwarding case).

I would question whether any application-level benefits could be measured by the network intermediaries. We (i.e., endpoint owners) have a hard enough time surmising application-level improvements from _endpoint_ transport metrics, let alone trying to intuit those as a network intermediary without any application context whatsoever. Making network-level decisions based on limited information is a recipe for "optimizations" that seem good on paper and for a limited set of metrics but are worse in reality. This is exactly the kind of thing that has led to the proliferation of TCP session terminating PEPs, which have overall been a real impedance to making improvements with TCP on the Internet.


If there’s endpoint implementations that aren’t doing a good enough job here and therefore we’re seeing new network deployments that introduce buffering because their measurements indicate it’s helpful on common use cases, it could be worthwhile to get this straightened out before it gets too normalized and we start having networks reintroduce hol-blocking in a way that hurts some QUIC use cases in the name of helping others.

Although I expect this can and should be solved at the endpoints, if there is data showing that the ordering solves a real problem with current implementations, reasonable people can reasonably conclude that network buffering to maintain packet order is a good idea.

(Note: UDP client-side port might be an option for a network-visible signal that might “just work” for many (hopefully most?) ordering schemes to avoid buffering for ordering, but it might need some API support and it might lead to some other kinds of ugly nat problems if there’s too many flows doing it...)

+Anna, Nathalie and Markus.  Hopefully they can comment on this also.

Best,
Jake

From: David Schinazi <[e-mail]<mailto:[e-mail]>>
Date: Wed,2021-07-28 at 2:16 PM
To: Roberto Peon <[e-mail]<mailto:[e-mail]>>
Cc: Ian Swett <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "Das, Dibakar" <[e-mail]<mailto:[e-mail]>>, Alan Frindell <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, Kirill Pugin <[e-mail]<mailto:[e-mail]>>
Subject: Re: [Mops] Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC

Why would we need a signal here? This applies to all traffic, be it TCP QUIC or anything else. Firmwares introducing latency to reorder packets was a reaction to bad implementations of TCP from a long time ago that have been fixed in systems that care about performance. In today's world, L2 is better off delivering any and all packets in the order they arrive instead of introducing buffer bloat.

David

On Wed, Jul 28, 2021 at 1:24 PM Roberto Peon <fenix=[e-mail]<mailto:[e-mail]>> wrote:
The ideal would be to have public bits that were intended to be interpreted by (as you say, visible to) those layers so any L2 could adapted appropriately without reinventing the wheel.
It isn’t just the local radio firmware that one needs to worry about—it is also the basestation(s) that may be “helping”.

Separately, but also important, is being able to get signals from the application about what tradeoffs should be at the network. I believe that this dovetails into many of the multipath issues, btw.

A couple potentially interesting params are:
  A bit to say please don’t HoL block
  Some kind of mechanism(s) to bound retries (e.g. “don’t retry bit”, but that is obviously not as expressive as throw out packet older than X microseconds)

-=R

From: QUIC <[e-mail]<mailto:[e-mail]>> on behalf of Ian Swett <ianswett=[e-mail]<mailto:[e-mail]>>
Date: Wednesday, July 28, 2021 at 12:42 PM
To: "Das, Dibakar" <[e-mail]<mailto:[e-mail]>>
Cc: Alan Frindell <afrind=[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, Kirill Pugin <[e-mail]<mailto:[e-mail]>>
Subject: Re: Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC

Hi,

I can't answer for Alan, but my belief is yes.  Client wifi stacks sometimes also do some reordering(and introduce the corresponding latency), so if we could design an indication that in-order delivery has no value, it could be fairly widely applicable.

That being said, I don't know what the right mechanism is?  Would we need something visible to a network or can we get away with a socket option that propagates to the local 5G network or Wifi firmware when possible?

Ian

On Wed, Jul 28, 2021 at 3:15 PM Das, Dibakar <[e-mail]<mailto:[e-mail]>> wrote:
Hi Kirill, Alan,

I could not attend the call this week and wont be able to attend this side meeting either.

But I had a general question about the performance of all such QUIC based protocols over wireless. Typically, the 5G and WiFI MAC layers deliver frames in-order which sort of recreates the HOL blocking problem at lower layers. I would expect this to in turn prevent the QUIC protocol to achieve its full performance gains at least in some congested network scenarios. Considering that in-order delivery is made optional in 5G PDCP, I was wondering if there could be a value to have some signaling defined in the QUIC (or RUSH ?) protocol that would allow lower layers to make better decision about whether to enable/disable in-order delivery for certain streams.

I apologize in advance if this is not the right venue to ask questions.

Regards,
Dibakar



From: QUIC <[e-mail]<mailto:[e-mail]>> On Behalf Of Alan Frindell
Sent: Wednesday, July 28, 2021 8:42 AM
To: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; QUIC WG <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Cc: Kirill Pugin <[e-mail]<mailto:[e-mail]>>
Subject: Reminder: Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC

Video Ingest over QUIC Side Meeting Friday 7/30 18:00 UTC / 11 Pacific

Link to draft agenda and video conference details: https://github.com/afrind/draft-rush/blob/main/meeting-materials/agenda.2021.07.03.md<https://urldefense.com/v3/__https:/github.com/afrind/draft-rush/blob/main/meeting-materials/agenda.2021.07.03.md__;!!GjvTz_vk!E0SzSsjcIQqc-TDdIf5-y7XjoWfnEA-7r9fdRAjEKZXc1GYhGomlKIXMwmDZ0Ls$>

-Alan

När du skickar e-post till Karlstads universitet behandlar vi dina personuppgifter<https://urldefense.com/v3/__https:/www.kau.se/gdpr__;!!GjvTz_vk!AVpyqSAj3HYNjnpUTryB_6DdElZmzNy4D78JApMaTuSanBUHBhhvFtmhpcriyO0$>.
When you send an e-mail to Karlstad University, we will process your personal data<https://urldefense.com/v3/__https:/www.kau.se/en/gdpr__;!!GjvTz_vk!AVpyqSAj3HYNjnpUTryB_6DdElZmzNy4D78JApMaTuSanBUHBhhvFtmhrH9ZMhY$>.
--
Mops mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/mops<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/mops__;!!GjvTz_vk!CYEeHbhK7C7RyDCja4rhiCVtdTRzsDAvDFa_mKJVustvWtuOr6WXbuj51KXgZqY$>