Re: [tsvwg] I-D Action: draft-ietf-tsvwg-ecn-l4s-id-13.txt

Bob Briscoe <> Thu, 04 March 2021 18:21 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 77F113A13BF for <>; Thu, 4 Mar 2021 10:21:28 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.434
X-Spam-Status: No, score=-1.434 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 1Dp-ZDyut2lr for <>; Thu, 4 Mar 2021 10:21:25 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 5E68A3A13BD for <>; Thu, 4 Mar 2021 10:21:25 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;; s=default; h=Content-Transfer-Encoding:Content-Type: In-Reply-To:MIME-Version:Date:Message-ID:References:To:From:Subject:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=wD+7LekmT986MWhns9LkNUhoAmUzwIu8OMYnhUFuTcE=; b=NPZI4qg6snDVRTAomuwTTeoYOZ hv2ieTOcFI7uKAW0vIdDFHC91nMe5beiGz156jAjhLeLgm7y97vVoE9Ik18dbiZC12wo0rygIDACq cKL/xqLA4t0A6GSFHA/A6N41yYaRRFw1AH+53Du/Dp6Ks9k7HIFvKRCpu/3JVhCWVcetTdqHc0rXp dG3UtkX6rKpr4vu4q5Gkw2DqhLTpt6sLGpNYSQJD7atMzrkX2buQXmgdyEHixQGMFL1+Cerj2nIZY QDMXyL/JkiCPqivTozy9oto8NGANtLyr1kGNW/LQXjn/3t7ybMKYwyFJ1dPtfaUP09/9rlcSaS3G0 QZ/r/PjQ==;
Received: from ([]:52314 helo=[]) by with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from <>) id 1lHsbE-0000ux-3a; Thu, 04 Mar 2021 18:21:24 +0000
From: Bob Briscoe <>
To: Ingemar Johansson S <>, "" <>
References: <> <20210225200602.lix6lyq2w6c6bclz@debian.debian> <> <> <> <> <>
Message-ID: <>
Date: Thu, 4 Mar 2021 18:21:22 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname -
X-AntiAbuse: Original Domain -
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain -
X-Get-Message-Sender-Via: authenticated_id:
Archived-At: <>
Subject: Re: [tsvwg] I-D Action: draft-ietf-tsvwg-ecn-l4s-id-13.txt
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 04 Mar 2021 18:21:29 -0000


Sorry, you probably thought this thread had ended this morning, but...

I've had someone suggest (off list), that the conclusion that RTCP 
datagrams can be ECN-capable ought to be written (informatively) into 
ecn-l4s-id. It would go in Appendix A.2.1, which would have to be 
retitled: "Setting ECT in -TCP- Control Packets and Retransmissions".

Before doing that, I checked RFC6679. It says
"RTCP reports MUST NOT be ECT marked, since ECT-marked traffic may be 
dropped if the path is not ECN compliant."

This seems rather paranoid, given the path is first checked for ECN 
support before using it on RTP media. It justifies this by saying the 
path might change to one that black-holes ECN-capable traffic. Whatever, 
if RFC6679 (proposed standard) says this, I don't want to contradict it. 
Do you think it would be OK to say:

"RFC6679 precludes the use of ECT on RTCP datagrams, in case the path 
changes after it has been checked for ECN traversal. L4S experiments 
will need to observe this rule. Nonetheless, as ECN usage becomes more 
widespread, it would be useful to gather data on whether such caution is 

Does that seem reasonable?


On 04/03/2021 07:33, Bob Briscoe wrote:
> Ingemar,
> OK. I think we've converged.
> Thank you
> Bob
> On 04/03/2021 07:15, Ingemar Johansson S wrote:
>> 😊, quite likely that the misunderstanding was on my side
>> Please see inline [IJ2]
>> /Ingemar
>>> -----Original Message-----
>>> From: Bob Briscoe <>
>>> Sent: den 3 mars 2021 18:18
>>> To: Ingemar Johansson S <>om>; Asad Sajjad
>>> Ahmed <>no>;
>>> Subject: Re: [tsvwg] I-D Action: draft-ietf-tsvwg-ecn-l4s-id-13.txt
>>> Ingemar,
>>> I obviously didn't phrase my question clearly. Pls see inline [BB]...
>>> On 03/03/2021 14:31, Ingemar Johansson S wrote:
>>>> Hi Bob, Asad + others
>>>> Answer to the @Ingemar question inline below [IJ1] :
>>>> /Ingemar
>>>>> -----Original Message-----
>>>>> From: Bob Briscoe <>
>>>>> Sent: den 2 mars 2021 14:54
>>>>> To: Asad Sajjad Ahmed <>no>;; Ingemar
>>>>> Johansson S <>
>>>>> Subject: Re: [tsvwg] I-D Action: draft-ietf-tsvwg-ecn-l4s-id-13.txt
>>>>> Asad and Ingemar,
>>>>> @Asad. Thank you v much for reminding me of that section in RFC3168.
>>>>> The text in ecn-l4s-id wasn't trying to allow a higher min window
>>>>> than existing RFCs. Indeed, it was trying to encourage implementers
>>>>> the other way - lower. But without forcing anyone, because
>>>>> implementing a lower minimum than 1  requires a step-change in
>>>>> complexity (as your thesis on this subject taught me).
>>>>> I don't think the first sentence with the SHOULD contradicts RFC3168,
>>>>> but it seems the second sentence makes it look that way to some
>>> people's eyes.
>>>>> So let's reword it. Also, I notice that it was only applicable to
>>>>> TCP, not all transports. How about...
>>>>>          A scalable congestion control SHOULD remain responsive to
>>>>>          congestion when typical RTTs over the public Internet are
>>>>>          significantly smaller because they are no longer inflated by
>>>>>          queuing delay.  It would be preferable for the minimum 
>>>>> window of a
>>>>>          scalable congestion control to be lower than the 1 
>>>>> segment minimum
>>>>>          for TCP mentioned in S.6.1.2 of [RFC3168] (or an 
>>>>> equivalent for other
>>>>>          transports). However, a lower minimum is not set as a formal
>>>>> requirement
>>>>>          for L4S experiments (see Appendix A.1.6 for rationale)."
>>>>> @Ingemar, you said your min in SCReAM is 3 datagrams, but you could
>>>>> configure it lower. Are SCReAM's datagrams smaller than the MTU? Is
>>>>> the text above clear enough for a non-TCP transport? Whatever, would
>>>>> you set a different limit?
>>>> [IJ1] Yes, you can probably set it lower. In practice however I 
>>>> have not seen
>>> much need to try that extreme as SCReAM is mostly to be used with 
>>> services
>>> that require edge user access, where RTTs still are in the order of
>>> milliseconds at best. This means that the BDP will be a few 
>>> kilobytes at a
>>> minimum. I probably need to look closer into this if necessary.. The 
>>> RTP
>>> packets are delivered by the video encoder pipeline where the MTU is 
>>> set in
>>> the RTP payload packetize. One can perhaps change the MTU on the fly to
>>> lower values of the BDP of the links become very low, not sure 
>>> however if
>>> that is possible for instance in gstreamer.
>>> [BB] I was trying to ask what you meant the typical min window would 
>>> be in
>>> bytes.
>>> That's the only reason I was trying to ask what MTU you were 
>>> thinking of, not
>>> because I was thinking you could reduce the MTU to force more 
>>> packets per
>>> window (that's a bad idea IMO).
>> [IJ2] OK, understood. SCReAMs window is in bytes, but the RTP packets 
>> of course need to be transmitted as whole units. The background is 
>> that SCReAM is desiged to be able to congestion control VoIP which 
>> typically has much smaller packets than video (e.g. 200bytes istf 
>> 1400bytes). A window size in bytes, rather than packets was therefore 
>> easier to implement. SCReAM will have problem to go below 1 RTP 
>> packet per window. I picked 3 packets as some kind of minimum but it 
>> can perhaps be reduced to one packet but below one packet per window 
>> that can be problematic. I believe that it is doable but then one 
>> need to redesign a bit and I have never really tried out SCReAM over 
>> links with very short RTT.
>>> Also, what about this question: Is the text above clear enough for a 
>>> non-TCP
>>> transport?
>> [IJ2] Yes, I don't see any difference for e.g. RTP/UDP in this respect.
>>> Bob
>>> PS. See Fig 1 here for 
>>> back-of-envelope
>>> calculations to work out if you could have a min-window problem once 
>>> you
>>> reduce Qdelay. When we first started testing L4S we found we were
>>> hitting TCP's min window when we ran more than half a dozen flows in
>>> parallel.
>>>>> Bob
>>>>> On 25/02/2021 20:06, Asad Sajjad Ahmed wrote:
>>>>>> Bob,
>>>>>> Thanks for updating and for have put in a lot of effort into this 
>>>>>> draft.
>>>>>> I have a minor comment regarding the cwnd reduction over ECN when
>>>>> cwnd
>>>>>> is already very low:
>>>>>> Current draft gives a formal requirement:
>>>>>> §4.3:
>>>>>>         "A scalable congestion control SHOULD remain responsive to
>>>>>>          congestion when typical RTTs over the public Internet are
>>>>>>          significantly smaller because they are no longer 
>>>>>> inflated by
>>>>>>          queuing delay.  It would be preferable for the minimum 
>>>>>> window of a
>>>>>>          scalable congestion control to be lower than the 2 segment
>>> minimum
>>>>>>          of TCP Reno [RFC5681] but this is not set as a formal 
>>>>>> requirement
>>>>>>          for L4S experiments (see Appendix A.1.6 for rationale)."
>>>>>> §A.1.6:
>>>>>>        "However, the requirement in Section 4.3 is worded as a 
>>>>>> "SHOULD"
>>>>>>         because the existence of a minimum window is not all 
>>>>>> bad.  When
>>>>>>         competing with an unresponsive flow, a minimum window 
>>>>>> naturally
>>>>>>         protects the flow from starvation by at least keeping 
>>>>>> some data
>>>>>>         flowing."
>>>>>> But, this contradict with RFC3168 (PS):
>>>>>> §6.1.2:
>>>>>>        "[...] Therefore, the sending TCP MUST reset the
>>>>>>         retransmit timer on receiving the ECN-Echo packet when the
>>> congestion
>>>>>>         window is one.  The sending TCP will then be able to send 
>>>>>> a new
>>>>>>         packet only when the retransmit timer expires."
>>>>>> The contradiction here is "SHOULD" vs. "MUST". Also TCP CC, as given
>>>>>> out by RFC5681, does NOT mandate a minimum transmission rate. TCP's
>>>>>> loss recovery at low cwnd relies on RTO to recover lost segments and
>>>>>> TCP using ECN is supposed to mimic this behavior [floyd98]. 
>>>>>> Otherwise,
>>>>>> you risk starving the Not-ECT set of flow, driving up the queue and
>>>>>> increasing the marking probability by going unresponsive at cwnd of
>>>>>> two segments. This is about harm toward other traffic so it make no
>>>>>> sense to relax a formal "MUST". Of couse, it is desireable that the
>>>>>> AQM take action and evict packets from non-responsive flows, but 
>>>>>> this is
>>>>> not true for all AQM, one example being CoDel.
>>>>>> But, the minimum cwnd of 2 does not originate from L4S, but has been
>>>>>> the current practice for a very long time at least under the 
>>>>>> Linux TCP
>>> stack.
>>>>>> This, obviously, also needs to be addressed.
>>>>>> Asad
>>>>>> [floyd98]
>>>>> 861fc
>>>>>> b972bfc-e0a4204086eb23c8&q=1&e=417868d5-deeb-4c4f-9d1e-
>>>>> debe95d3e33c&u=
>>>>>> (§8)
>>>>>> On 21/02/22 14:01:03, wrote:
>>>>>>> A New Internet-Draft is available from the on-line Internet-Drafts
>>>>> directories.
>>>>>>> This draft is a work item of the Transport Area Working Group WG of
>>> the
>>>>> IETF.
>>>>>>>            Title           : Identifying Modified Explicit 
>>>>>>> Congestion Notification
>>> (ECN)
>>>>> Semantics for Ultra-Low Queuing Delay (L4S)
>>>>>>>            Authors         : Koen De Schepper
>>>>>>>                              Bob Briscoe
>>>>>>>     Filename        : draft-ietf-tsvwg-ecn-l4s-id-13.txt
>>>>>>>     Pages           : 58
>>>>>>>     Date            : 2021-02-22
>>>>>>> Abstract:
>>>>>>>       This specification defines the identifier to be used on IP 
>>>>>>> packets
>>>>>>>       for a new network service called low latency, low loss and 
>>>>>>> scalable
>>>>>>>       throughput (L4S).  L4S uses an Explicit Congestion 
>>>>>>> Notification (ECN)
>>>>>>>       scheme that is similar to the original (or 'Classic') ECN 
>>>>>>> approach.
>>>>>>>       'Classic' ECN marking was required to be equivalent to a 
>>>>>>> drop, both
>>>>>>>       when applied in the network and when responded to by a 
>>>>>>> transport.
>>>>>>>       Unlike 'Classic' ECN marking, for packets carrying the L4S
>>>>>>>       identifier, the network applies marking more immediately 
>>>>>>> and more
>>>>>>>       aggressively than drop, and the transport response to each 
>>>>>>> mark is
>>>>>>>       reduced and smoothed relative to that for drop. The two 
>>>>>>> changes
>>>>>>>       counterbalance each other so that the throughput of an L4S 
>>>>>>> flow will
>>>>>>>       be roughly the same as a non-L4S flow under the same 
>>>>>>> conditions.
>>>>>>>       Nonetheless, the much more frequent control signals and 
>>>>>>> the finer
>>>>>>>       responses to them result in much more fine-grained 
>>>>>>> adjustments, so
>>>>>>>       that ultra-low and consistently low queuing delay 
>>>>>>> (typically sub-
>>>>>>>       millisecond on average) becomes possible for L4S traffic 
>>>>>>> without
>>>>>>>       compromising link utilization.  Thus even capacity-seeking 
>>>>>>> (TCP-like)
>>>>>>>       traffic can have high bandwidth and very low delay at the 
>>>>>>> same time,
>>>>>>>       even during periods of high traffic load.
>>>>>>>       The L4S identifier defined in this document distinguishes 
>>>>>>> L4S from
>>>>>>>       'Classic' (e.g. TCP-Reno-friendly) traffic.  It gives an 
>>>>>>> incremental
>>>>>>>       migration path so that suitably modified network 
>>>>>>> bottlenecks can
>>>>>>>       distinguish and isolate existing traffic that still 
>>>>>>> follows the
>>>>>>>       Classic behaviour, to prevent it degrading the low queuing 
>>>>>>> delay and
>>>>>>>       low loss of L4S traffic.  This specification defines the 
>>>>>>> rules that
>>>>>>>       L4S transports and network elements need to follow to 
>>>>>>> ensure they
>>>>>>>       neither harm each other's performance nor that of Classic 
>>>>>>> traffic.
>>>>>>>       Examples of new active queue management (AQM) marking
>>> algorithms
>>>>> and
>>>>>>>       examples of new transports (whether TCP-like or real-time) 
>>>>>>> are
>>>>>>>       specified separately.
>>>>>>> The IETF datatracker status page for this draft is:
>>>>>>> There are also htmlized versions available at:
>>>>>>> A diff from the previous version is available at:
>>>>>>> Please note that it may take a couple of minutes from the time of
>>>>>>> submission until the htmlized version and diff are available at
>>>>>>> Internet-Drafts are also available by anonymous FTP at:
>>>>> -- 
>>> __________________________________________________________
>>>>> ______
>>>>> Bob Briscoe
>>>>> 75f4171a-2a6f6ec2-861fcb972bfc-e28ad1342845f12e&q=1&e=417868d5-
>>>>> deeb-4c4f-9d1e-debe95d3e33c&
>>> -- 
>>> __________________________________________________________
>>> ______
>>> Bob Briscoe
>>> d73900a3-88a27959-86d8a30ca42b-6abcf6465bd4a19d&q=1&e=e8dae484-
>>> 35ef-4d79-b1ee-6fe94beff7ce&

Bob Briscoe