Re: [tsvwg] L4S issue #22: CE Ambiguity and Reordering

Jonathan Morton <> Mon, 24 February 2020 17:11 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 5933E3A0ED3 for <>; Mon, 24 Feb 2020 09:11:50 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.848
X-Spam-Status: No, score=-1.848 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 2ZkrPIPxraIp for <>; Mon, 24 Feb 2020 09:11:48 -0800 (PST)
Received: from ( [IPv6:2a00:1450:4864:20::12b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 4E7BA3A0EF0 for <>; Mon, 24 Feb 2020 09:11:48 -0800 (PST)
Received: by with SMTP id s23so7319610lfs.10 for <>; Mon, 24 Feb 2020 09:11:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=+58MOMEit1yrxfgTwca1jdZ/bgDLjetanK+pou9fB/U=; b=Yrry3/dE7FSPSPz4D7syiVXeLNXUp766HzQzDYQPt4254BiQ77vXH/LTKf/XdJCBCn WelalNLohmHg40MJ0Pp0ZXhMsda+tCsKdiE5Gc4Ik5T4CN91B9t4h67aU2cBZb0/6Fud 1BWf8T76RGVuA4HwFId+72ayTX34JfS1HaI+Em07ZLUwkg80X8FfdsLoxl4qqCkY869l sPFBoYVXHvJZRwlv/LkqshojAf1JLoA3RWWnmfaJ7Ax26jivAUvunLsjIIJ4cyzuWAhz EFY2NqaKTrHiGTABJDM+8WjlAEy6PbExuo3isu9XpeqVHhzHAyYyI+xvtPH5jZOFr63v qEwA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=+58MOMEit1yrxfgTwca1jdZ/bgDLjetanK+pou9fB/U=; b=UfYiqM4bYRgc+0xhoDxjt2WVWpWYpMxV30XdfxXK/e0ClVtzflxu7oEY256xU8lD8x aJo1i5cuPDrGSM/51/gzakdwjKqGwN4vo3qZrXGWfid+KFrt4ir2JOQQvUc4KD8odjUy 3xc6669fk8CCEl5HHJwilbxaMTbhDYM6vRtaZHwqLQbeZMCYlHTj/PUK1C7kAgnBDJz9 K1KH9aKK+MT2KauhhSXeVJq9/KRETc59h8bx9fMPZohGgN2StP96uvxn21/IH0zQFrj8 bbga5C1hK4Dm7t5vkFo+oquzQdaSDog703Xd4SX34EL9VeUT2BObP40FRj7kzejErB5m E+dQ==
X-Gm-Message-State: APjAAAU04o1tfsCsTC66E/UmB8qsmpehAF1TWjTKS1WVulGwRW7ZkfxD M4hc3NqcFRmkD/LrLn7j+1A=
X-Google-Smtp-Source: APXvYqx9qhBUxcfFXaBDQxG4TilafOhxMedDRvS18tJ4JnOrzAp72/7ZE8m/M39dy0cUvi7pXf5rwg==
X-Received: by 2002:a05:6512:45c:: with SMTP id y28mr2957014lfk.58.1582564306405; Mon, 24 Feb 2020 09:11:46 -0800 (PST)
Received: from jonathartonsmbp.lan ( []) by with ESMTPSA id k4sm5953718lfo.48.2020. (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Feb 2020 09:11:45 -0800 (PST)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
From: Jonathan Morton <>
In-Reply-To: <>
Date: Mon, 24 Feb 2020 18:11:31 +0100
Cc: Bob Briscoe <>, Sebastian Moeller <>, tsvwg IETF list <>
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <>
To: "Holland, Jake" <>
X-Mailer: Apple Mail (2.3445.9.1)
Archived-At: <>
Subject: Re: [tsvwg] L4S issue #22: CE Ambiguity and Reordering
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 24 Feb 2020 17:11:56 -0000

> On 24 Feb, 2020, at 5:15 pm, Holland, Jake <> wrote:
> What I was aiming toward when asking the question originally was that if there’s something that already exists and could be cited that demonstrates the scope of the problem justifying a normative SHOULD for RACK-like behavior, it would be beneficial to include it.  If there is no such source to cite, I didn’t mean to say it’s necessary to create one before moving forward.
> To me the reordering issue itself seems so minor that I’m confused why there’s so much text and meeting time discussion about it.  I was worried I was missing something.  If the whole issue is addressed by the appendix text, it seems like even the “SHOULD” for RACK-like behavior from the transport is not justified, since there’s basically no harm even without it?

My understanding here is that the root of this issue is that CE-marked packets originating upstream of the DualQ may be classified into a different queue than the rest of their flow, and may therefore be delivered out of order.  DualQ treats CE packets as always belonging to the L4S queue, so this issue affects Classic flows only.

The maximum expected quantity of reordering is easiest to define in time units; the CE packet would be about 0.5ms late if the L4S queue was at its target depth while the Classic queue was empty, while it would be 15ms early if the reverse were true.  The behaviour of a RACK-type transport can be directly predicted based on those values, while predicting the behaviour of a traditional stack may require those times to be converted into packet serialisation times by way of the flow's throughput.

The "RACK requirement" applies only to L4S flows, so I believe it is not relevant to resolving this.  Classic flows will continue to use a mixture of RACK (eg. Linux) and traditional (eg. default FreeBSD at this time) stacks.  It is the latter that are most likely to be affected, so testing (and thought experiments) should focus there.

 - Jonathan Morton