Re: Robustness to packet reordering

Christian Huitema <huitema@huitema.net> Thu, 06 February 2025 16:23 UTC

Return-Path: <huitema@huitema.net>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4E6B7C180B5A; Thu, 6 Feb 2025 08:23:52 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.703
X-Spam-Level:
X-Spam-Status: No, score=-1.703 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=neutral reason="invalid (public key: not available)" header.d=mfg.outbound
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ifygi7jBkOFX; Thu, 6 Feb 2025 08:23:51 -0800 (PST)
Received: from semf08.mfg.siteprotect.com (semf08.mfg.siteprotect.com [64.26.60.171]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CFAA1C180B4E; Thu, 6 Feb 2025 08:23:50 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mfg.outbound; s=default; h=Content-Transfer-Encoding:Content-Type: In-Reply-To:From:References:To:Subject:MIME-Version:Date:Message-ID:reply-to: sender:cc:bcc; bh=PEduRjPIqgMtrKs7ptd1q/wVN9V4YafjMz/SrF9TTAE=; b=BUzrNFGuabM 5f1X/uQFVsZh8s9HBtSFYwPbcnFGr+NKUCZxOLjV7HFrd6Oax93ILtJNF8hschSGVNELxAJPM2KnJ z97lFMTcA9qzp9RrWyQiIfT65xh2V+NEd6GnvhPYS4M07k9/qLfosm7oW7vpVpgtsp2Uh/qMxIW8D DpK6AjFwsc02S+EuEPlqlRAVo97+Qxt/GGB3mH22f915iTke9VuEwZQqDqFgrnZZIwte6WU/Wm6g3 iA+Jx2iOMcdNC9rKUOllYpECETekPsNvZEQ6y0xI5BAH3cn+eptQ6dn+92shVXlEI+T6fnmuD/M8W DWG6Ii/ipZdej7kpcWh1VQg==;
Received: from smtpauth02.mfg.siteprotect.com ([64.26.60.151]) by se02.mfg.siteprotect.com with esmtp (Exim 4.92) (envelope-from <huitema@huitema.net>) id 1tg4fH-003fkN-QZ; Thu, 06 Feb 2025 11:23:49 -0500
Received: from [192.168.1.112] (unknown [172.56.203.90]) (Authenticated sender: huitema@huitema.net) by smtpauth02.mfg.siteprotect.com (Postfix) with ESMTPSA id 4Ypj8X6dFnz2YQR60; Thu, 6 Feb 2025 11:23:40 -0500 (EST)
Message-ID: <fa524b1a-0083-44fc-90d0-1c8f256258c9@huitema.net>
Date: Thu, 06 Feb 2025 08:23:38 -0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Robustness to packet reordering
To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>, Ingemar Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>, "quic@ietf.org" <quic@ietf.org>
References: <AM8PR07MB81375E2D3CA840AEDA0F7E63C2F72@AM8PR07MB8137.eurprd07.prod.outlook.com> <38891fa7-b188-4767-8364-ae0a10c318b2@huitema.net> <AM8PR07MB81370760A8293580DDDB386BC2F62@AM8PR07MB8137.eurprd07.prod.outlook.com> <537f0095-68a3-43d5-a2f0-f91b4499017d@huitema.net> <AM8PR07MB81377C0A967260302A687113C2F62@AM8PR07MB8137.eurprd07.prod.outlook.com>
Content-Language: en-US
From: Christian Huitema <huitema@huitema.net>
In-Reply-To: <AM8PR07MB81377C0A967260302A687113C2F62@AM8PR07MB8137.eurprd07.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"; format="flowed"
Content-Transfer-Encoding: 8bit
Authentication-Results: mfg.siteprotect.com; auth=pass smtp.auth=huitema@huitema.net
X-Originating-IP: 64.26.60.151
X-SpamExperts-Domain: mfg.outbound
X-SpamExperts-Username: 64.26.60.150/31
Authentication-Results: mfg.siteprotect.com; auth=pass smtp.auth=64.26.60.150/31@mfg.outbound
X-SpamExperts-Outgoing-Class: ham
X-SpamExperts-Outgoing-Evidence: Combined (0.07)
X-Recommended-Action: accept
X-Filter-ID: Pt3MvcO5N4iKaDQ5O6lkdGlMVN6RH8bjRMzItlySaT9jI9Ds0+TJTcNTg4DMzS8/PUtbdvnXkggZ 3YnVId/Y5jcf0yeVQAvfjHznO7+bT5y/DScqHYFMZLwmx6yLgYxTSN7JZ8x51pXe8980C10/EKVi 4+SrPobmzYgA8ug2rA1/v+5gw9IJ1n6pxipzJwwiWN7TzoaZQV7Ck4fKlO+XnEZQb1M3nP/aVkZP /RjRdQBvxT1UqsRkA9qFNyupWgN858fMnqZUTt7CyKlJUh+zhqgYqABgs2/YAX/yBgm9d2vloW5O 8nKi+aEgzCvYcGcNn17NKh9p/w7qzG8kYg5zcSBdmcsqSvM8aD2DlRGiCU8/wJVyGDMOl6Z2bJkW 2fbn3+uVPy4qJ4XNUiQOREK3xvBpiSBKfrEnmrDcp7BzCUsY4L7u6QJ/jXXAPItovk04WCvmn1IS DvxBA68tSDRfJFvEkjU2IZs0qHcxreE84qWzFtXV8maz/1DxiEQsBT9b8p8NyvSkRHwTwTx9G1M5 EMvdGlnEQJMn5KuObFLxmLretxRp8RUXQ0zG3NTd7kg13WOGcz8xu9rZctX8sgTNonUlO58lzbID FnBI582KCIFDFxQeM5hKKDqHvwRqP46o31/E3ahF5MMcDI7KdpjQKQq00y8ef45tB3j6HRlrmdMi /N8uQ6NBdVvWgbLvK6Z3IO3fJ9DGWEJopzxygkVT7yk4xAm9D8KTeKJT7gNACPfxVynFIF3Xrs2K JHQMxL9PL44/hrSNUBLj7yl7ZdLauIw2HNAQHBPEUF9IEWosJVEImfO3L4Yreq31xl9Xnj7ZDrsE paGv+y2uezlyzp4j8DzThqzFcMy7pHH9jcXXfxEj2d/xmizmqI+HnJWmDMLdQKcof8wqZrg1Xj+B VtcbX6nwr91lZN8JVcydBRphcAJK0fY/3gADQdrr7U0COoeGQd0kE+MbxRJ6df8DL1v6U20yyYb0 NObBE1uU7g2ibnFrVbRuhjiW0LHI4igXvLceUV8ShebT8U8Xw9HTDfreWeBFm9pNPUFDsiQcrCGS 6D1M8jeeJSytG/Wf88r2kBubK6391sxAqxbtq1CC1/LZwU9EUDwTCfbSvjFHgHqV660RNHZCFIG/ 9wYQQJyxC1I28DgKIfeVhSVdRinC5q1VkADk5nWIiIcthdTaotkIarQOjBgXlf3YbVfG6+cIUAg4 nWJ8xr0PBByl/sG5xgFyG2Z8WHz3Dshc1OOGUXnRskw=
X-Report-Abuse-To: spam@se02.mfg.siteprotect.com
X-Complaints-To: abuse@se01.mfg.siteprotect.com
Message-ID-Hash: EEVN427DVMF56HXHKEF4OTZKMWVFVO6A
X-Message-ID-Hash: EEVN427DVMF56HXHKEF4OTZKMWVFVO6A
X-MailFrom: huitema@huitema.net
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-quic.ietf.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header
X-Mailman-Version: 3.3.9rc6
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/u1Q6HE1ok5iru1Qqd1oFIq2Q8lQ>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Owner: <mailto:quic-owner@ietf.org>
List-Post: <mailto:quic@ietf.org>
List-Subscribe: <mailto:quic-join@ietf.org>
List-Unsubscribe: <mailto:quic-leave@ietf.org>

The two documents you want to read or cite are RFC 9002 and the QUIC 
Acknowledgment Frequency draft 
(https://datatracker.ietf.org/doc/html/draft-ietf-quic-ack-frequency) 
Specifically, section 6 of RFC 9002 defining "Loss detection", defining 
kPacketThreshold and recommending at least 3 and then kTimeThreshold, 
with a suggested value of 9/8 RTT.

-- Christian Huitema

On 2/6/2025 1:34 AM, Ingemar Johansson S wrote:
> OK, so that means that time based loss detection is not yet implemented in QUIC stacks, or ?
>
> The reason I ask is that we poll the interest in the support for out of order delivery of packets in 5G. The outline is that we ensure in order delivery for up to some given milliseconds, to handle possible HARQ retransmissions on the MAC layer. Beyond that we forward packets as they are processed by the radio stack.
>
> The rationale behind this is to avoid that packets for latency sensitive flows (streams) are delayed more than necessary if they share the same data radio bearer as other streams.
>
> We see that TCP Linux is robust to packet reordering (up to 1 RTT reordering depth) and fast retransmits are avoided (except in the initial phase when the reordering window grows). Thus I wonder if there are plans to make also QUIC stacks support time based RACK?
>
> /Ingemar
>
>
>
>> -----Original Message-----
>> From: Christian Huitema <huitema@huitema.net>
>> Sent: Thursday, 6 February 2025 08:57
>> To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>; Ingemar
>> Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>;
>> quic@ietf.org
>> Subject: Re: Robustness to packet reordering
>>
>> RACK is the default for QUIC. The detection gap is set to 3 by default,
>> that is, a packet considered lost if a packet N+3 or later has been
>> received. Implementations are free to implement dynamic RACK gaps and
>> the ACK Frequency extension can ask the peer to send packets
>> accordingly, but AFAIK most implementations go for robustness and keep
>> the detection gap fixed.
>>
>> Note that QUIC does not "reorder" packets -- packets are processed as
>> soon as they arrive.
>>
>> -- Christian Huitema
>>
>> On 2/5/2025 10:50 PM, Ingemar Johansson S wrote:
>>> Hi Christian
>>>
>>> I was perhaps a bit unclear, sorry.
>>> I refer to "in the stack".
>>>
>>> The RACK function in Linux TCP increases the reordering window when
>> packet reordering is detected and thus avoids fast retransmits. In
>> current Linux TCP the reordering window can increase to accept up to one
>> RTT reordering.
>>> Do any current QUIC stack support this RACK functionality and/or is it
>> planned.
>>> /Ingemar
>>>
>>>> -----Original Message-----
>>>> From: Christian Huitema <huitema@huitema.net>
>>>> Sent: Thursday, 6 February 2025 02:00
>>>> To: Ingemar Johansson S
>>>> <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>; quic@ietf.org
>>>> Cc: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
>>>> Subject: Re: Robustness to packet reordering
>>>>
>>>>
>>>> On 2/5/2025 6:21 AM, Ingemar Johansson S wrote:
>>>>> Hi
>>>>>
>>>>> A very short question, with possibly many answers:
>>>>> Do current QUIC stacks support out of order delivery up to one RTT,
>>>>> as
>>>> is the case with TCP Linux with RACK enabled.
>>>>
>>>> Define support. Do you mean "at the API" or "in the stack"?
>>>>
>>>> The programming APi for QUIC exposes a set of "streams", as well as
>>>> possibly a "datagram" capability. Data sent on one stream is
>>>> delivered in sequence on that stream. Multiple streams can be
>>>> processed in parallel. There is no guarantee of ordering between
>>>> stream. If a packet containing data for stream number N is lost,
>>>> delivery on that stream will stop until the loss recovery -- but
>>>> delivery will continue for other streams.
>>>>
>>>> Data sent as datagram is delivered whenever the datagram arrives.
>>>> There is no guarantee of ordering between datagrams, or between
>>>> datagrams and streams. There is also no expectation of loss recovery:
>>>> if a packet containing a datagram is lost, the datagram is lost.
>>>>
>>>> In the stack, QUIC stacks are expected to check whether a packet was
>>>> "already received", and not process duplicates. Note that if packets
>>>> are lost, the frames contained in the packet may be processed
>>>> differently -- some may not be resent, either because they are now
>>>> obsolete, or because the frame has already been received -- for
>>>> example, in case of spurious retransmission. The packets carrying the
>>>> repeated data will have their own packet number, different from the
>> initial packet.
>>>> In order to not process duplicate, implementations have to maintain
>>>> knowledge of already received packets. That knowledge typically has
>>>> some kind of horizon, such as "the last N packets". Packets that are
>>>> older than that will be ignored, because there is no way to tell
>>>> whether they are duplicate. The value of that "horizon" is
>> implementation dependent.
>>>> About 1 RTT worth of packet makes sense, but implementations could
>>>> use something else, like for example 3*PTO.
>>>>
>>>> -- Christian Huitema
>>>>
>>>>