[tsvwg] Re: Robustness to packet reordering

[Neal Cardwell <[e-mail]>]

> Given how modern implementations work, reordering at the link-layer is
almost always harmful.

For modern implementations of the RACK style or newer, that sounds true.
But I would have guessed there is a large installed base of TCP senders out
there with pre-RACK TCP stacks. And for those, presumably, as Greg noted,
"older TCP implementations that don't support RACK would have problems with
reordering." So AFAICT there may be some tricky trade-offs. If most 5G
link-layer retransmissions can be completed in X milliseconds (X less than,
say, 10ms), then it may be worth the user's 5G cell phone modem buffering
received packets for X milliseconds to try to deliver in-order data to the
receiver if this can be done in a low-latency manner. AFAIK various layers
of networking software (and humans using it) already need to tolerate ~10ms
of jitter due to various link layer effects for the most common last mile
link-layer technologies (wifi, cellular, DOCSIS) and CPU scheduling effects
for networking stacks (especially in user space). But I'm sure there are
other factors I'm not taking into account... :-)

neal


On Thu, Feb 6, 2025 at 5:15 PM David Schinazi <[e-mail]>
wrote:

> Having the link-layer delay packets to provide them in-order to the
> transport layer was helpful multiple decades ago when TCP implementations
> had more naive algorithms and were very resource-constrained. Given how
> modern implementations work, reordering at the link-layer is almost always
> harmful. TCP and QUIC stacks can handle reordering well, thanks to both
> protocol features and implementations having more memory to work with. The
> delay induced by this reordering will generally cause more harm than good.
> Furthermore, one of the biggest motivators for QUIC was to break
> head-of-line blocking and allow out-of-order delivery to the application
> layer. We have large bodies of data showing that this improves performance.
> Please disable reordering at the 5G layer.
>
> David
>
> On Thu, Feb 6, 2025 at 2:15 PM Christian Huitema <[e-mail]>
> wrote:
>
>>
>> On 2/6/2025 12:55 PM, Martin Thomson wrote:
>> >
>> > On Fri, Feb 7, 2025, at 04:59, Greg White wrote:
>> >> This is an important topic relating to the expectations and
>> >> requirements that transport protocols place on layer 2 protocols.  In
>> >> layer 2 standards bodies that I've been involved in, it has been
>> >> understood that "the upper layers" expect in-order delivery,
>> > As far as QUIC goes, it is sensitive to reordering in the network.
>> Some reordering will be interpreted as damage (Christian cited the relevant
>> parts) and performance suffers in a few minor way when things arrive out of
>> order (ACKs are less efficient, data needs to be held, memory accesses are
>> less likely to be contiguous, etc...).
>> >
>> > However, the idea that the network might seek to "fix" these problems,
>> when doing so necessarily involves extra work and delays, is not a good
>> trade.  Stuff that is delayed to "fix" a reordering that happened might
>> delay signals that the QUIC stack could use, even if some data needs to be
>> held at the endpoint.  QUIC packets contain many things, some of which
>> don't need to be strictly ordered to be useful.
>>
>> Applications that are sensitive to delays will break their traffic into
>> multiple QUIC streams. In case of packet loss, only one of those streams
>> will be blocked, the others will be delivered without "head of queue
>> blocking". Implementing L2 correction will make the response worse, not
>> better, because all the streams will be delayed for the duration of the
>> L2 correction instead of just one.
>>
>> -- Christian Huitema
>>
>>