Re: [tsvwg] Fwd: New Version Notification for draft-heist-tsvwg-ecn-deployment-observations-02.txt

Bob Briscoe <> Tue, 09 March 2021 00:25 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 38C1C3A1AB9 for <>; Mon, 8 Mar 2021 16:25:45 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -6.434
X-Spam-Status: No, score=-6.434 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_HI=-5, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id arzBltuTdNlm for <>; Mon, 8 Mar 2021 16:25:43 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 081083A19BD for <>; Mon, 8 Mar 2021 16:25:42 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;; s=default; h=Content-Transfer-Encoding:Content-Type: In-Reply-To:MIME-Version:Date:Message-ID:From:References:To:Subject:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=lB0HkUkoqZQQ+u8t0pdirTKCKXFmGgggDqV0CQSlghU=; b=Hg99wn9gNVRNAuwsLHmyHGrMYT Y09owTblT2JbxUyoDw7nC14wS8WJvkSTuYijuA+YRwgw/wOVWndwjGf1OaqQSMrz487z+RTNg/k9z Kk3a4GAn3MP6rxQP4eh95Sq6ofuM6j9A3aE+Wf9dSjrfcXhxoroiNlSjx6xHuwHtlDaGUowUcCqAb CBZFv6Hw6EFm6l4Ex1J6hIHQO9UdWmaWXSIQ7cOT+MD2qAxO1JStBf1xNrgzGQLNWdmknNS3GvR7H Hz4/BZwivrQ5nGvNtCIwKOQiND8mzUhaNw5FJZn/Dz3bwrJitRb0bZ/lTiQUC/FrpUj051pnxX4nc hDMWzgdg==;
Received: from ([]:37946 helo=[]) by with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from <>) id 1lJQBw-0004O6-Se; Tue, 09 Mar 2021 00:25:40 +0000
To: Pete Heist <>, tsvwg IETF list <>,
References: <> <> <>
From: Bob Briscoe <>
Message-ID: <>
Date: Tue, 9 Mar 2021 00:25:40 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname -
X-AntiAbuse: Original Domain -
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain -
X-Get-Message-Sender-Via: authenticated_id:
Archived-At: <>
Subject: Re: [tsvwg] Fwd: New Version Notification for draft-heist-tsvwg-ecn-deployment-observations-02.txt
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 09 Mar 2021 00:25:45 -0000


Thanks for the updated draft a fortnight ago, with the extra columns I 
asked for.

I've been excluding the rows with obvious pathologies.

Of the 90 IPs in your TCP results, 74 have at least one obvious 
pathology, leaving only 16 clean.
And actually, there was a pathology suggested in your slides today that 
isn't in the above,
     If I check for that too, 83 IPs have at least one pathology, 
leaving only 7 clean.

Any idea what is going on?


In the TCP results, unlike with QUIC, the reverse feedback is visible. 
So I can check for more pathologies.

I checked for this obvious pathology, that wasn't directly mentioned in 
the slides today:
* Congestion experienced (CE) in one direction but no echo congestion 
experienced (ECE) feedback in the other.
* Or conversely, echo congestion experienced (ECE) feedback in one 
direction but no congestion experienced (CE) in the other.

Of the 90 IPs in the TCP results, 40 from the WAN have this pathology 
and 38 from the LAN
(taking the direction as that of the data)
Only 16 IPs, don't have this pathology in either direction.

I would have thought you guys would have picked up these when checking 
for the ECE/CE >= 2
     'cos otherwise you would have got loads of divide by zero errors 
(and/or zeros).

If I include the pathology you mention where ECE/CE < 2
In all, of the 90 IPs in the TCP results, after I exclude any IP with a 
pathology, I only have 7 IPs left.


PS. I agree it's v strange for an ISP to share capacity between its 
members by how many flows they are using.

On 08/03/2021 22:19, Pete Heist wrote:
> I'm following up in this thread to one question that's come up- why is
> an ISP deploying fq_codel in its backhaul?
> In short, this ISP encourages research and experiments by its members,
> and this was a way to both see how much congestion exists, and attempt
> to manage it with AQM when needed. Experiments with fq_codel were
> started a couple of years ago, and I helped add support for it to their
> custom router interface early last year. However, it's only actually
> been enabled on a couple of routers, because:
> - it takes a while to update routers in this environment
> - some of the routers run kernels prior to fq_codel support
> - congestion doesn't usually last long enough to give it much urgency,
> and sometimes it's fixed by just adding capacity
> IMO, fq_codel isn't ideal in this situation. When there actually is
> congestion to manage, flow-fairness means that one user can dominate
> the queue with many flows, which is exactly one of the problems we'd
> like to avoid. Host fairness could improve on this, or better yet,
> member fairness using the member database, as members can have more
> than one IP/MAC. But, fq_codel is what's available now, and generally
> usable enough in this case to get started with AQM.
> On Mon, 2021-03-08 at 11:09 +0100, Pete Heist wrote:
>> -------- Forwarded Message --------
>> Subject: New Version Notification for draft-heist-tsvwg-ecn-
>> deployment-
>> observations-02.txt
>> Date: Mon, 08 Mar 2021 01:57:07 -0800
>>> A new version of I-D, draft-heist-tsvwg-ecn-deployment-
>>> observations-
>>> 02.txt
>>> has been successfully submitted by Peter G. Heist and posted to the
>>> IETF repository.
>>> Name:           draft-heist-tsvwg-ecn-deployment-observations
>>> Revision:       02
>>> Title:          Explicit Congestion Notification (ECN) Deployment
>>> Observations
>>> Document date:  2021-03-08
>>> Group:          Individual Submission
>>> Pages:          30
>>> URL:
>>> Status:
>>> Html:
>>> Htmlized:
>>> Diff:
>>> Abstract:
>>>     This note presents data gathered at an Internet Service
>>> Provider's
>>>     gateway on the observed deployment and usage of ECN.  Relevant
>>> IP
>>>     counter and flow tracking data was collected and analyzed for
>>> TCP
>>> and other protocols.
>> A few updates in this version:
>> * Improved statistics for the separated "known" and "unknown" AQMs.
>> * Clarified the detection method for possible AQMs.
>> * Added possibility of QUIC-ECN traffic in non-TCP data.
>> Pete

Bob Briscoe