Re: [tsvwg] L4S issue #22: CE Ambiguity and Reordering

Bob Briscoe <> Wed, 26 February 2020 00:11 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 50EE73A0926 for <>; Tue, 25 Feb 2020 16:11:19 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.099
X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id z6aJqoau4Aqw for <>; Tue, 25 Feb 2020 16:11:17 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 761753A0927 for <>; Tue, 25 Feb 2020 16:11:17 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;; s=default; h=Content-Transfer-Encoding:Content-Type: In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=2VTq3nD79GLCY1OyxlPH91H/QDMEaAib1ojAIs8zEFA=; b=dLSbm4zIpCRGc4/ku5bGyCdUT6 Z4xeuMbTaTJ1WdakOfC/pgvMmk32FEj1ajMvEDXXog4hQdocTt5cJjwBqwe/m0cT/30tlRbj8z37X POU4WHtW28PbYxFtDqFyWgB0LAxoYOxIE5SD13cguECZEwLXXllB3WCdrjEsvGffPW5CSUyWkJ/oJ xlN9Z4rwcPd2TwerDy2LO0569XKqw4WOQ+l0tjpdh8e+wIaEOsyI86UDdfWoSKpd7wMF8A24+1jxw hr8E2TPi3eJAwgahh3t3dC5x8nQhwoWOsuUk6M9HPTNAiApGvss3tR/q8XYhmYqJ4v6b4aDzGA812 WCD9ewjQ==;
Received: from [] (port=41094 helo=[]) by with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.92) (envelope-from <>) id 1j6kI8-00044A-3R; Wed, 26 Feb 2020 00:11:08 +0000
To: Jonathan Morton <>, "Holland, Jake" <>
Cc: Sebastian Moeller <>, tsvwg IETF list <>
References: <> <> <> <>
From: Bob Briscoe <>
Message-ID: <>
Date: Wed, 26 Feb 2020 00:11:06 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname -
X-AntiAbuse: Original Domain -
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain -
X-Get-Message-Sender-Via: authenticated_id:
Archived-At: <>
Subject: Re: [tsvwg] L4S issue #22: CE Ambiguity and Reordering
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 26 Feb 2020 00:11:19 -0000


On 24/02/2020 17:11, Jonathan Morton wrote:
>> On 24 Feb, 2020, at 5:15 pm, Holland, Jake <> wrote:
>> What I was aiming toward when asking the question originally was that if there’s something that already exists and could be cited that demonstrates the scope of the problem justifying a normative SHOULD for RACK-like behavior, it would be beneficial to include it.  If there is no such source to cite, I didn’t mean to say it’s necessary to create one before moving forward.
>> To me the reordering issue itself seems so minor that I’m confused why there’s so much text and meeting time discussion about it.  I was worried I was missing something.  If the whole issue is addressed by the appendix text, it seems like even the “SHOULD” for RACK-like behavior from the transport is not justified, since there’s basically no harm even without it?
> My understanding here is that the root of this issue is that CE-marked packets originating upstream of the DualQ may be classified into a different queue than the rest of their flow, and may therefore be delivered out of order.  DualQ treats CE packets as always belonging to the L4S queue, so this issue affects Classic flows only.
> The maximum expected quantity of reordering is easiest to define in time units; the CE packet would be about 0.5ms late if the L4S queue was at its target depth while the Classic queue was empty,
A CE packet from a Classic flow will never be delivered late.
I think you have not understood how the Coupled DualQ works.
> while it would be 15ms early if the reverse were true.
This part is correct.

This is why the phenomenon can only occur if there are (or have very 
recently been) two bottlenecks on the path:
1) a Classic AQM with a queue sufficient for for it to mark the Classic 
flow with CE
2) a Coupled DualQ with a queue built up in its Classic queue

> The behaviour of a RACK-type transport can be directly predicted based on those values, while predicting the behaviour of a traditional stack may require those times to be converted into packet serialisation times by way of the flow's throughput.
> The "RACK requirement" applies only to L4S flows, so I believe it is not relevant to resolving this.  Classic flows will continue to use a mixture of RACK (eg. Linux) and traditional (eg. default FreeBSD at this time) stacks.  It is the latter that are most likely to be affected, so testing (and thought experiments) should focus there.

>   - Jonathan Morton

Bob Briscoe