Re: [tsvwg] [Int-area] 2nd TSVWG WGLC on ecn-encap-guidelines and rfc6040-update-shim drafts, closes 6 May 2019

Bob Briscoe <> Sat, 27 July 2019 10:01 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 978C3120289; Sat, 27 Jul 2019 03:01:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: 1.335
X-Spam-Level: *
X-Spam-Status: No, score=1.335 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id M4BNV48B7PTT; Sat, 27 Jul 2019 03:01:11 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 2190512027E; Sat, 27 Jul 2019 03:01:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;; s=default; h=Content-Transfer-Encoding:Content-Type: In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=VFiEWV4uyTOZELOzsRiamQdVz6aUW4wZ3tzORojQZQw=; b=ChLaQPj267uJLtH8whR4PVykLR QJAPsVzeuIw4ijy9gxhyJGS+1qpGB5PPBfCBMmBN+Mq7HX53lrAF+kbS0QpAM3+hYqbqZICXbf8FW D79sY1WYHTZ+e3O1zt2MggEf2qEtcJ2UreZ3Rts6uWFlvELH15EuMwsyC+WLLECxsK1sioW7SX7Ve c14OTc4zKCgyZCBvwY88jx1tBaBqstMaD2mJimYMOjY1xGEvVTuuEGgwcfieaA98IeCxmowQA1faZ dQmDZXqulpl7Mx/ukhml0E++UQuGSNU03KHWDjtkiDo0SUMAqgvNwUeyKg7bNG94P8IaBlImaIYF+ 2tADKsJQ==;
Received: from [] (port=41640 helo=[]) by with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.92) (envelope-from <>) id 1hrJVg-0002i8-8z; Sat, 27 Jul 2019 11:01:05 +0100
To: Jonathan Morton <>
Cc: Joe Touch <>, "Black, David" <>, "" <>, tsvwg <>
References: <> <> <> <> <>
From: Bob Briscoe <>
Message-ID: <>
Date: Sat, 27 Jul 2019 06:01:01 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname -
X-AntiAbuse: Original Domain -
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain -
X-Get-Message-Sender-Via: authenticated_id:
Archived-At: <>
Subject: Re: [tsvwg] [Int-area] 2nd TSVWG WGLC on ecn-encap-guidelines and rfc6040-update-shim drafts, closes 6 May 2019
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sat, 27 Jul 2019 10:01:14 -0000


On 10/07/2019 11:38, Jonathan Morton wrote:
>> On 8 Jul, 2019, at 3:27 pm, Bob Briscoe <> wrote:
>> These are quite significant updates to outer fragment processing at the tunnel egress. But, given something has to be said, I can't think of a better way (see the original quoted email about why the logical OR of the ECN codepoints as defined in RFC3168 is no longer sufficient - and it's no simpler anyway).
> If I may offer such an alternative approach, which avoids the need to keep persistent state at the reassembly point whilst still properly handling RFC3168, L4S and SCE expected semantics:
> - If all incoming fragments are Not-ECT marked, the outgoing packet must also be so marked.
> - If any fragment has CE set, the reassembled packet must have CE set.
> (This guarantees correct RFC3168 and SCE behaviour for each conventional AQM marking action.  Your proposal doesn't, as it will generally result in fewer CE marks downstream especially if the smaller fragments end up being marked; subsequent upstream CE marks have to relieve a counter deficit before they will be honoured.  The tradeoff is that L4S may see some technical "over marking" but this should be tolerable.)
Yes, with byte-preserving, as packets are re-assembled the number of 
marked packets reduces. Counter-intuitively, that's correct, even for 
compatibility with TCP's single congestion response per RTT.

I originally suggested the requirement in RFC3168 to preserve the number 
of marked packets, but it's incorrect. It's not compatible with TCP's 
single response per RTT (or the response to the proportion of marks of 
other TCP-Friendly real-time congestion controls).

This is not a matter of compatibility with just one of SCE or L4S. The 
logical OR approach is wrong for both, and the byte-preserving approach 
is correct for both - see previous response to Markku.

Reasoning: the paramount requirement when reassembling fragments is to 
reconstruct the marking probability that would have occurred had the 
packets not been fragmented when the AQM in the tunnel marked them. The 
logical OR approach increases the marking probability as if congestion 
was higher, while byte-preserving keeps it constant.

If it helps, consider the reductio ad absurdam proof that, as fragments 
get smaller, the logical OR approach would ultimately result in every 
packet being marked.

Regarding state, the problem isn't amount of state because reassembly 
requires per-packet state and the the approach I suggested adds only 2 
int's per tunnel decap.

The problem seems to be common read-write access to these variables. 
However, it's not important that they are updated before forwarding 
continues. So updates can be queued in parallel to forwarding.

Also the state can include occasional errors, so it doesn't have to be 
strictly persistent - meaning it's unnecessary to preserve during a 
re-boot or if a tunnel endpoint is moved.

> - Notwithstanding the above rules, the ECT(0) vs ECT(1) choice should be made according to the majority of fragmented payload bytes so marked, on the individual packet being reassembled.  In the case of a tie, break in favour of ECT(0).
My original proposed wording deliberately allows such an algorithm. 
Byte-preserving was stated as a goal. The specific mechanism was only an 

(I'm not convinced majority voting would be simpler than byte counting, 
which never leads to an exception case. But that's irrelevant anyway).

> (We may expect that L4S packets will be entirely ECT(1) marked except for fragments or whole packets carrying CE; this also applies to obsolete Nonce Sum semantics.  SCE and RFC-3168 flows will be ECT(0) marked by default, with perhaps some ECT(1) marking applied by SCE middleboxes.  SCE is reasonably tolerant of disruption to its markings, because the control loop is fundamentally stable.)
As above, my proposed wording is nothing to do with support for one 
semantic or another. It is for all semantics.
> - A mixture of Not-ECT and other ECN codepoints is \unexpected and may imply upstream shenanigans.
Yup, the wording in 3168, and my wording both agree with you here.


>   - Jonathan Morton

Bob Briscoe