Re: [tsvwg] Status of ECN encapsulation drafts (i.e., stuck)

Bob Briscoe <> Sat, 14 March 2020 18:07 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id ED5D23A090B for <>; Sat, 14 Mar 2020 11:07:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.099
X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id dAQAyIdtwfx5 for <>; Sat, 14 Mar 2020 11:07:45 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 835C83A090A for <>; Sat, 14 Mar 2020 11:07:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;; s=default; h=Content-Transfer-Encoding:Content-Type: In-Reply-To:MIME-Version:Date:Message-ID:References:Cc:To:Subject:From:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=aWTXPXhIX+Cnsq0OVJDkMQbKTySBixxxE+e9az4w5kg=; b=Fhz0ue4yjW6vYnojn4HmJDA8s0 P0Cj2vTWbIXN39W8Y+LTPEIrAYBJE+ZjLVIZ+jOIeQWuiMPD4Ye72FMqKIA+cbqXR4QTRKRM4t6tF G6GPYC8PE90HadPpKicCh6Oaw7Lwc0L2eN2fgrKwHWvqM+StTFmmXaVM6TxIoAAlGdLNcXbShrO5n D1yX3kh33YtuhsbAwhuI/CTC/K5KKREUUEARpERxzuVaWGHM1riMB8itWpN7ZJV9r/FTBhXja0qAf jS3EyHvKbdJN1yP1G2arJ09i7d8Vfkgg2IRKM7ZLSFRfqlKvSP+wxZyjsd0jwxPg22Q7B4ijfs1BF 78Pj6TOw==;
Received: from [] (port=34972 helo=[]) by with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from <>) id 1jDBCJ-000GCz-0d; Sat, 14 Mar 2020 18:07:43 +0000
From: Bob Briscoe <>
To: Sebastian Moeller <>
Cc: Jonathan Morton <>, "" <>
References: <> <> <> <> <> <> <> <> <> <> <> <> <>
Message-ID: <>
Date: Sat, 14 Mar 2020 18:07:33 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname -
X-AntiAbuse: Original Domain -
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain -
X-Get-Message-Sender-Via: authenticated_id:
Archived-At: <>
Subject: Re: [tsvwg] Status of ECN encapsulation drafts (i.e., stuck)
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sat, 14 Mar 2020 18:07:48 -0000


On 13/03/2020 22:54, Sebastian Moeller wrote:
> Dear All,
> so in this example we need an operator that operates a tunnel (encapsulation and decapsulation) and an AQM on the same tunnel and is considered to both (mis-)configure the tunnel to allow fragmentation and operate an SCE-AQM on hops along the tunnel's path.
Tunnels are often constructed over the top of the network operator that 
runs the physical equipment (incl. any AQM). Overlays are one of the 
common motivations for tunnelling.

> Is that really considered to be a sensible deployment that is worth catering to? I would guess having a tunnel that fragments/defragments alone would be painful/sub-optimal. Why would an operator that consciously accepted such a tunnel design (as there probably real world constraints that can justify such an arrangement) would then go and add a non-tunnel compatible AQM? Are fragmenting tunnels really such a big part of the commercial ISP world?t
Short answer: I don't know.
Where an operator who sets up a tunnel can control the MTU of the links, 
I would imagine fragmentation is highly unlikely. But for overlays using 
tunnelling, most tunnel endpoints use PMTUD between themselves to find a 
good tunnel MTU, and I would imagine they use fragmentation for IPv4 
(because they have to, if their "packet too big" ICMPs are being 
blocked) for any incoming packets that don't fit once they add tunnel 
headers. But I don't know how often they actually have to fragment 
(depends on end-systems strategies are to keep the MTU below this point).

The only concrete data I can find is from the year 2000.
In this case all the packets on the link were fragments, and "A 
significant portion of the fragmented traffic ... is tunneled traffic." 
I don't know whether this link was chosen for this study because all the 
packets were fragments, or whether it would have been easy to find other 
links with a high proportion of fragments.

Finding the best max packet size is an area where neither IPv4 nor IPv6 
has ever found a good solution. Getting a tunnel to fragment and 
reassemble is indeed painfully sub-optimal, but all the other solutions 
have their own problems. It is possible the sub-optimality is often 
going on under-the-covers, just because it works. I do know that, for 
IPv4, the Don't Fragment (DF) flag is often ignored by tunnels, as a 
preferable alternative to just ditching the traffic.

In the absence of the IP layer solving this problem, Gorry's group found 
that about 20% of end-systems are just clamping the IPv4 TCP max segment 
size lower than necessary. And for IPv6 most are just using the min 
segment size. See:
Again only for TCP, middleboxes can edit the MSS advertised in each 
packet to fool the hosts into using a smaller max segment size. You can 
configure a Cisco box (and I'm sure others) to do this when you're using 
it for tunnelling. The above study from Gorry's group didn't find much 
evidence of this though.

PLPMTUD solves this problem end-to-end (see [RFC4821] for TCP etc. and 
[draft-ietf-tsvwg-datagram-plpmtud] not yet published for UDP). However, 
where a tunnel already solves the problem (sub-optimally) for IPv4 using 
fragmentation, I can't imagine that anyone would disable it, because it 
is still necessary and it still works.

Further reading.
I just compiled a list, then realized everything is already cited here, 
in the context of minimizing latency:

You could also try 

> Best Regards
> 	Sebastian
>> On Mar 13, 2020, at 19:50, Bob Briscoe<>  wrote:
>> Jonathan,
>> On 13/03/2020 18:15, Jonathan Morton wrote:
>>>> On 13 Mar, 2020, at 7:58 pm, Bob Briscoe<>  wrote:
>>>> Are you reading my recent emails, or the main email?
>>> The recent ones, because I'm trying to establish this fundamental point first.  Until I can figure out what you *do* mean by CE marks being doubled, there's no point in discussing anything more subtle.
>>> So again, I ask: what context am I missing?
>> I said "No." I.e. the AQM is not marking before the tunnel ingress. It is marking between tunnel ingress and egress.
>> How can I know what context you're missing if you haven't read the email? Presumably that means you're missing all the context given in the email.
>> Pls can you take these sort of emails off the IETF lists - I'm sure no-one cares for this sort of content-free conversation.
>> Bob
>>>   - Jonathan Morton
>> -- 
>> ________________________________________________________________
>> Bob Briscoe

Bob Briscoe