[tcpm] ECN CE that was ECT(0) incorrectly classified as L4S

Bob Briscoe <ietf@bobbriscoe.net> Thu, 13 June 2019 16:48 UTC

Return-Path: <ietf@bobbriscoe.net>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id A58701200E9; Thu, 13 Jun 2019 09:48:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bobbriscoe.net
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id 9LP-nbU2C40i; Thu, 13 Jun 2019 09:48:18 -0700 (PDT)
Received: from server.dnsblock1.com (server.dnsblock1.com []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1D659120047; Thu, 13 Jun 2019 09:48:17 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Type:MIME-Version:Date:Message-ID:Cc: Subject:From:To:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Vnr3zcv8JQ+9Pht8FIa3d5h3AGxFIVJgCn/TDCFkFz0=; b=0EmJejWLrFImbWqxL5ET8rN74N QOkksxbBgYJU4urWtvgI8XU1RnCoDJipZNoPPDPPWHXs79uFBdmQ0DUpwgVSFUw4ztDGLgK0K+5MY BLRZf7IGdD8x71FDQIEf0X7esUdBT2NIhW86aUMB8Ecs0M2btzcHGyEYqwMo7bDV7hm9M+OYNdquV BAsbhyVfawvsN67zPt3qWhyFtrcekd96f+JVapWnyH0YyW2KmGCY9Uifg1c/iewmHiJu0+4k2rhRN q/SJvjjFIOuF5604w2vYnFLcbLW84YzfppmBg/efdSMTnIo22BFo1hoassfb5AGdQgFesNA+BSR4Z VwwuDH1w==;
Received: from [] (port=36364 helo=[]) by server.dnsblock1.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.92) (envelope-from <ietf@bobbriscoe.net>) id 1hbSta-0003B8-Vn; Thu, 13 Jun 2019 17:48:15 +0100
To: "ecn-sane@lists.bufferbloat.net" <ecn-sane@lists.bufferbloat.net>, tcpm IETF list <tcpm@ietf.org>
From: Bob Briscoe <ietf@bobbriscoe.net>
Cc: tsvwg IETF list <tsvwg@ietf.org>
Message-ID: <24f7b15a-129f-ca44-60e0-32c7d23eadf4@bobbriscoe.net>
Date: Thu, 13 Jun 2019 17:48:14 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="------------CADEF1FB433DB76B60AD3057"
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - server.dnsblock1.com
X-AntiAbuse: Original Domain - ietf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - bobbriscoe.net
X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net
X-Authenticated-Sender: server.dnsblock1.com: in@bobbriscoe.net
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/3ro6q76NFp5UKkYEdRIfmDlYwnY>
Subject: [tcpm] ECN CE that was ECT(0) incorrectly classified as L4S
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Jun 2019 16:48:22 -0000

[I'm sending this to ecn-sane 'cos that's where I detect that this 
concern is still rumbling.
I'm also sending to tcpm@ietf 'cos there's a question for TCP experts 
just before the quoted text below.
And tsvwg@ietf is where it ought to be discussed.]

Now that the IPR issue with L4S has been put to bed, one by one I am 
going through the other concerns that have been raised about L4S.

In the IETF draft that records all the pros and cons of different 
identifiers to use for L4S, under the "ECT(1) and CE" choice (which is 
currently the one adopted at the IETF) there was already an explanation 
of why there would be vanishingly low risk of any harmful consequences 
from CE that was originally ECT(0) being classified into the L4S queue:

Re-reading that, I have found some things unstated that I had thought 
were obvious. So I've spelled it all out long-hand in the text below, 
which is now in my local copy of the draft and will be in the next 
revision unless people suggest improvements/corrections here.

*Q#1:* If this glosses over any concerns you have, please explain.
Otherwise I will continue to consider that this is effectively a 
non-issue, which is the conclusion everyone in the TCP community came to 
at the time the L4S identifier was chosen back in 2015.

*Q#2: *The last couple of lines are the only part I am not sure of. Do 
most of today's TCP implementations recover the reduction in congestion 
window when they discover later that a fast retransmit was spurious? 
There's a note at the end of the intro to rfc4015 saying there was 
insufficient consensus to standardize this behaviour, but that most 
likely means it's done in different ways, rather than it isn't done at all.



    Risk of reordering classic CE packets:  Classifying all CE packets
       into the L4S queue risks any CE packets that were originally
       ECT(0) being incorrectly classified as L4S.  If there were delay
       in the Classic queue, these incorrectly classified CE packets
       would arrive early, which is a form of reordering.  Reordering can
       cause TCP senders (and senders of similar transports) to
       retransmit spuriously.  However, the risk of spurious
       retransmissions would be extremely low for the following reasons:

       1.  It is quite unusual to experience queuing at more than one
           bottleneck on the same path (the available capacities have to
           be identical).

       2.  In only a subset of these unusual cases would the first
           bottleneck support classic ECN marking while the second
           supported L4S ECN marking, which would be the only scenario
           where some ECT(0) packets could be CE marked by a non-L4S AQM
           then the remainder experienced further delay through the
           Classic side of a subsequent L4S DualQ AQM.

       3.  Even then, when a few packets are delivered early, it takes
           very unusual conditions to cause a spurious retransmission, in
           contrast to when some packets are delivered late.  The first
           bottleneck has to apply CE-marks to at least N contiguous
           packets and the second bottleneck has to inject an
           uninterrupted sequence of at least N of these packets between
           two packets earlier in the stream (where N is the reordering
           window that the transport protocol allows before it considers
           a packet is lost).

              For example consider N=3, and consider the sequence of
              packets 100, 101, 102, 103,... and imagine that packets
              150,151,152 from later in the flow are injected as follows:
              100, 150, 151, 101, 152, 102, 103...  If this were late
              reordering, even one packet arriving 50 out of sequence
              would trigger a spurious retransmission, but there is no
              spurious retransmission here, because packet 101 moves the
              cumulative ACK counter forward before 3 packets have
              arrived out of order.  Later, when packets 148, 149, 153...
              arrive, even though there is a 3-packet hole, there will be
              no problem, because the packets to fill the hole are
              already in the receive buffer.

       4.  Even with the current recommended TCP (N=3) spurious
           retransmissions will be unlikely for all the above reasons.
           As RACK [I-D.ietf-tcpm-rack] is becoming widely deployed, it
           tends to adapt its reordering window to a larger value of N,
           which will make the chance of a contiguous sequence of N early
           arrivals vanishingly small.

       5.  Even a run of 2 CE marks within a classic ECN flow is
           unlikely, given FQ-CoDel is the only known widely deployed AQM
           that supports classic ECN marking and it takes great care to
           separate out flows and to space any markings evenly along each

       It is extremely unlikely that the above set of 5 eventualities
       that are each unusual in themselves would all happen
       simultaneously.  But, even if they did, the consequences would
       hardly be dire: the odd spurious fast retransmission.  Admittedly
       TCP reduces its congestion window when it deems there has been a
       loss, but even this can be recovered once the sender detects that
       the retransmission was spurious.

Bob Briscoe                               http://bobbriscoe.net/