Re: [tsvwg] Updated review: start of WGLC on L4S drafts: L4S Arch

Bob Briscoe <ietf@bobbriscoe.net> Mon, 25 October 2021 23:38 UTC

Return-Path: <ietf@bobbriscoe.net>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 399B93A0952 for <tsvwg@ietfa.amsl.com>; Mon, 25 Oct 2021 16:38:23 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.43
X-Spam-Level:
X-Spam-Status: No, score=-5.43 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-3.33, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bobbriscoe.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qb9mdldXyHXq for <tsvwg@ietfa.amsl.com>; Mon, 25 Oct 2021 16:38:18 -0700 (PDT)
Received: from mail-ssdrsserver2.hostinginterface.eu (mail-ssdrsserver2.hostinginterface.eu [185.185.85.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1FE2A3A0B0A for <tsvwg@ietf.org>; Mon, 25 Oct 2021 16:38:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Transfer-Encoding:Content-Type: In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=uqLKWiFDkO6IZFwSZoNxWJpJdZc59El/efdznVVzuys=; b=LAy4sUY4LnMqyX4AfPtcMDCKzk XFwh9HQYljcjdtc3nrxzgkkO0VNoqMT7+tob6czDOrHjYFriECSsd1/53NRjvim/FWjKA2N/QuR53 O5FF8Q73wXLHfJKqnAifjVvLbqPiLevXSZ2GGZT1Gb/8POjdGIwuYDxEammRpxVxghg+ZiuRDsGir NAOae3frtzMdHeO8wU90m3kif/jpMVrk/LqPikg1FfEUATtH0MUtITxPHAF+zR26SObelACvDet3w iaInc5VG1LFZ1hxl1V18vhykAOIhZb37Uq/GPB9C9o5/CeN7XrbcnT5F8NqISd+o0C1+/EiXbI3wP ij6AdCCQ==;
Received: from 67.153.238.178.in-addr.arpa ([178.238.153.67]:48774 helo=[192.168.1.13]) by ssdrsserver2.hostinginterface.eu with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from <ietf@bobbriscoe.net>) id 1mf9Xf-00051L-9q; Tue, 26 Oct 2021 00:38:07 +0100
To: Sebastian Moeller <moeller0@gmx.de>
Cc: Gorry Fairhurst <gorry@erg.abdn.ac.uk>, "tsvwg@ietf.org" <tsvwg@ietf.org>
References: <7dd8896c-4cd8-9819-1f2a-e427b453d5f8@mti-systems.com> <B575CC81-4633-471A-991F-8F78F3F2F47F@ericsson.com> <aa968ff5-262c-1fd4-981d-05507ac1e59e@erg.abdn.ac.uk> <6cd9cf3c-cdc0-4274-6c5b-11a4aebf268b@erg.abdn.ac.uk> <d83506cd-48f2-4077-c15c-824b5d8fe39c@bobbriscoe.net> <FF9DC7CD-6D0B-42AD-8B19-626F1F20FBD2@gmx.de>
From: Bob Briscoe <ietf@bobbriscoe.net>
Message-ID: <6288a605-bbfa-5ea4-b20f-22747788c807@bobbriscoe.net>
Date: Tue, 26 Oct 2021 00:38:07 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <FF9DC7CD-6D0B-42AD-8B19-626F1F20FBD2@gmx.de>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - ssdrsserver2.hostinginterface.eu
X-AntiAbuse: Original Domain - ietf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - bobbriscoe.net
X-Get-Message-Sender-Via: ssdrsserver2.hostinginterface.eu: authenticated_id: in@bobbriscoe.net
X-Authenticated-Sender: ssdrsserver2.hostinginterface.eu: in@bobbriscoe.net
X-Source:
X-Source-Args:
X-Source-Dir:
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/a6ziNXggHUxg7-Jz_n-C5O8KLxc>
Subject: Re: [tsvwg] Updated review: start of WGLC on L4S drafts: L4S Arch
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 25 Oct 2021 23:38:23 -0000

Sebastian,

On 25/10/2021 14:46, Sebastian Moeller wrote:
> Hi Gorry, hi Bob,
>
>
> as far as I can tell this now leaks factually wrong "averaged base RTT to a CDN" number into even more places... Even if Bob's recent opinion, that  the average RTT to the _closest_ CDN is a suitable proxy for "internet RTT" is accepted, it at least needs to be described veridically as what it is, calling that number "averaged base RTT to a CDN" does not do justice to the source that number was generated from.
>
> Regards
> 	Sebastian
>
> P.S.: I do not accept that the RTT to the closest CDN is a decent proxy for the internet, as not all content providers/ISPs work with all CDNs so it does really not matter if I sit in Lagos, Nigeria with 4ms RTT to a google CDN if the content I need sits 100ms away at the nearest CDN2 site... and even in  "rich" countries it is not guaranteed that the path to any content is equally close. If Bob would argue for average RTT to the biggest X CDNs he might have a point, but "to the closest" simply is painting too rosy a picture here.

[BB] I know ISPs usually work with one or two CDNs, as opposed to all of 
them. I said the figures produced in that study are a workable estimate. 
They are not perfect. But most CDNs that serve a country are going to 
have pretty similar RTTs.

Whatever, this data is miles better than a guess. That's all I'm trying 
to achieve. I can certainly describe the source of the data with more 
circumspection. But, I can't both write and read emails simultaneously, 
and I'm afraid I've now missed the deadline for submitting the drafts 
again - for a few days. I'll updated my editor's copy.


Bob

>
>
>
>> On Oct 25, 2021, at 14:15, Bob Briscoe <ietf@bobbriscoe.net> wrote:
>>
>> Gorry,
>>
>> Thank you again for yet another careful review.
>> No comment = accepted. For qualifications or push back, pls see [BB] inline...
>>
>>
>> On 13/08/2021 12:00, Gorry Fairhurst wrote:
>>> Adding to my own review, after re-reading L4S ID  ...
>>>
>>> Gorry
>>>
>>> On 12/08/2021 10:44, Gorry Fairhurst wrote:
>>>> I reviewed Low Latency, Low Loss, Scalable Throughput (L4S) Internet Service Architecture during the WGLC. I support publication, but I think there are issues that would need to be addressed relating to the way L4S is presented.
>>>>
>>>> Best wishes,
>>>>
>>>> Gorry
>>>>
>>>>
>>>> This is a review of draft-ietf-tsvwg-l4s-arch-10:
>>>>
>>>> 1a) Issue: “It is increasingly common for _all_ of a user's applications at any
>>>>     one time to require low delay: interactive Web, Web services, voice,
>>>>     conversational video, interactive video, interactive remote presence,
>>>>     instant messaging, online gaming, remote desktop, cloud-based
>>>>     applications and video-assisted remote control of machinery and
>>>>     industrial processes. “
>>>> - I generally agree on the value for a wide range of applications and sect 6.1 appears to contain a helpful list. ~ However, I disagree that the word “all” is necessary or correct in this section. I think there are cases that do not fit. Background software update, retrieval of logging data, some cases where loss protection is more important than for other traffic, etc, So I insist there are examples that do not require this. I expect I could find more examples, but I’d prefer the word “all” to be removed, to leave this as a generalisation.
>> [BB] This no way intends to say "all applications require low delay".
>> The words bracketing around the "all" are not to be overlooked... /Increasingly common/ and  /at any one time/.
>> How about this to make that point clearer:
>>
>> "At any one time, it is increasingly common for /all/ of the traffic in a bottleneck link (e.g. a household's Internet access) to come from applications that prefer low delay:..."
>>
>>
>>>> ---
>>>> 1b) Issue: I’d also quibble with: “so that all of a user's applications can shift to it when their stack is updated.“ - again, is the word “all” really necessary or correct here?
>> [BB] OK. "So that a user's applications can shift..."
>>
>>>> ---
>>>> 2) NiT
>>>> “The L4S
>>>>     approach also requires component mechanisms at the endpoints to
>>>>     fulfill its goal.”
>>>> - True, can you provide examples of what these mechanisms are (or a cross ref)? e.g., are they Protocol Mechanisms (as they are called in section 4.1)? or is this more the  to Host Mechanisms (section 4.3) ?
>> [BB] Yes a bit abstract isn't it. I've actually rewritten parts of the whole para, that were also abstract:
>>
>>     L4S is designed for incremental deployment. It is possible to deploy
>>     the L4S service at a bottleneck link alongside the existing best
>>     efforts service [DualPI2Linux] so that unmodified applications can
>>     start using it as soon as the sender's stack is updated.  Access
>>     networks are typically designed with one link as the bottleneck for
>>     each site (which might be a home, small enterprise or mobile device),
>>     so deployment at either or both ends of this link should give nearly
>>     all the benefit in the respective direction.  With some transport
>>     protocols, namely TCP and SCTP, the sender has to check for suitably
>>     updated receiver feedback, whereas with more recent transport
>>     protocols such as QUIC and DCCP, all receivers have always been
>>     suitable.
>>
>>     This document presents the L4S architecture, by describing and
>>     justifying the component parts and how they interact to provide the
>>     scalable, low latency, low loss Internet service.  It also details
>>     the approach to incremental deployment, as briefly summarized above.
>>
>>
>>
>>>> ---
>>>> 3) Issue or reword.
>>>> “Although DCTCP as-is 'works' well over the public
>>>>        Internet, most implementations lack certain safety features that
>>>>        will be necessary once it is used outside controlled environments
>>>>        like data centres (see Section 6.4.3 and Appendix A).”
>>>> - If this is published by the iETF, then I think this needs to have consensus that DCTP is acceptable. I don’t now which RFC states that.
>>>> Later this ID states: “ DCTCP needs some safety concerns to be fixed for general use
>>>>     over the public Internet (see Section 4.3 of
>>>>     [I-D.ietf-tsvwg-ecn-l4s-id]),”
>>>> which seems much closer to what is needed in this section.
>> [BB] Ah yes, the intent was to reassure that is works well over wide-area RTTs. I can see how "it works over the public Internet" would be misconstrued.
>>        Although DCTCP as-is functions well over wide-area
>>        round trip times, most implementations lack certain safety
>>        features that would be necessary for use outside controlled
>>        environments like data centres (see Section 6.4.3 and Appendix A).
>>
>> ---
>>>> 4) NiT
>>>> “With
>>>>        Classic congestion controls, such as Reno or Cubic, because flow
>>>>        rate has scaled since TCP congestion control was first designed in
>>>>        1988, it now takes hundreds of round trips (and growing) to
>>>>        recover after a congestion signal (whether a loss or an ECN mark)
>>>>        as shown in the examples in Section 5.1 and [RFC3649].
>>>> “
>>>> - This only seems like a correct statement, if the flow is a bulk / high volume flow that is using a high capacity path? Please check and clarify.
>> [BB] We will certainly add
>>      "assuming the flow lasts long enough".
>> But the "hundreds of round trips and growing" statement is accurate. There is no assumption of a high BDP path - the first example with a Cubic recovery time of 4.3s (166 rounds) in Section 5.1 is lower than the the global average for a download from a CDN - at least using the best data I can find.
>>
>> I agree that both x-references only describe the scaling, not where we are on the scale. Rather than putting loads of detail in the terminology definition, we've added the following after the example it refers to in § 5.1:
>> "For a feel of where the global average lone-flow download sits on this scale at the time of writing (2021), according to [BDPdata] globally averaged fixed access capacity was 103Mb/s in 2020 and averaged base RTT to a CDN was 25-34ms in 2019. Averaging of per-country data was weighted by Internet user population. So a lone CUBIC flow would at best take about 200 round trips (5 s) to recover from each of its sawtooth reductions, if the flow even lasted that long. This is described as 'at best' because it assume everyone uses an AQM, whereas In reality most users still have a bloated tail-drop buffer. So likely average recovery time would be at least 4x 5 s, if not more, because RTT under load would be at least double, and recovery time depends on the square of RTT."
>>
>> That's why we said "hundreds". Is that reasonable?
>> Here's the figures:
>>
>>
>> avg fixed downstream [Mb/s]
>> avg CDN RTT under load [ms]
>> avg BDP under load
>> Recovery time [s]
>> Recovery time [#RTTs]
>> Global avg
>> 103
>> 35ms
>> (25-34ms base RTT + avg queue - half sawtooth)
>> 438kB
>> 6.6s
>> 188
>> Example ref'd in Draft
>> 120
>> 25.5ms
>> (30ms max RTT - half sawtooth)
>> 383kB
>> 4.2s
>> 166
>>
>> This assumes Cubic.
>> The top row has the bottom half of the sawtooth in Reno-friendly mode, and the top half in Cubic. There is no formula for a sawtooth in transition, so we've taken the average of the two: 6.8s & 6.4s.
>> The bottom row is fully in Reno-friendly mode still.
>>
>> Reference:
>> [BDPdata] Briscoe, B., "PI2 Parameters", Technical Report TR-BB-2021-001 arXiv:2107.01003 [cs.NI], July 2021, <https://arxiv.org/abs/2107.01003>.
>>
>>
>>>> ---
>>>> 5) NiT
>>>> “such as QUIC.”
>>>> - Reference needed for [RFC9000].
>> [BB] Transport protocol references like this are given when they appear elsewhere, but we've added this one here as well, to support the statement about Reno being the default CC in QUIC.
>>
>>>> ---
>>>> 6) NiT
>>>> “[I-D.ietf-tsvwg-ecn-l4s-id] recommends ECT(1)”
>>>> draft-ietf-tsvwg-ecn-l4s-id appears to “specify” the use of ECT(1) for this purpose. Please check whether ECT(1) is only recommended.
>>>> ---
>>>> 7) Issue: TCP is a core protocol, and L4S support depends on Accurate ECN: Is accurate ECN a normative?  (see comment 22)
>>>> “Work to standardize and implement more
>>>>            accurate ECN feedback for TCP (AccECN) is in
>>>>            progress [I-D.ietf-tcpm-accurate-ecn],”
>> [BB] When I asked about this before (years ago), the WG and chairs agreed on it being informative.
>> Reasoning: AccECN should be a normative ref in a scalable CC spec that requires AccECN (for instance, Acc-ECN is normative in draft-briscoe-iccrg-prague-congestion-control).
>> But there is no normative statement that references AccECN in l4s-arch, it's always referred to informatively, like QUIC or DCCP is.
>> Similarly for ecn-l4s-id, but we'll discuss that in your other review.
>>
>>>> ---
>>>> 8) NiT In section 2, ref useful:
>>>> “high level explanation of how FQ”
>>>> - Please expand and add reference for FQ.
>> [BB] We've expanded FQ. But there is no ref to an L4S-FQ - the only description is in this draft - other than the Linux patch. And where this draft describes it, it refers to the appropriate section of the FQ-CoDel RFC.
>>
>>>> ---
>>>> 9) Typo:
>>>> “ The 'Low Loss” part of the name denotes”
>>>> - This has mixed quote marks. In other places, double quotes are used.
>>>>
>>>> ---
>>>>
>>>> 10) Issues relating to critique of diffserv:
>>>>
>>>> 10a) Making assertions about Diffserv:
>>>> I really don’t like this choice of words, it’s far too general a conclusion:
>>>> “      Also, Diffserv only works for a small subset of the traffic on a
>>>>        link. “
>>>> - Diffserv  works fine when all traffic is marked with a set of DSCPs - I hope you’d agree, but there are cases where it wouldn’t give benefit, e.g. when all traffic on a congested link/router is marked with one code point, whereas there may be benefit if all traffic on a link/router is marked with one code point, when that traffic later shares bandwidth with traffic carrying other marks.
>> [BB] I think this implies you'd be happier if we wrote:
>>       "Also, Diffserv can only provide a latency benefit if a small subset of the traffic on a bottleneck link requests low latency."
>> OK?
>>
>>>> ---
>>>> 10b) Making assertions about Diffserv:
>>>> “As already explained, it is not applicable when all the
>>>>        applications in use at one time at a single site (home, small
>>>>        business or mobile device) require low latency.”
>>>> - See above. I really don’t like saying it is not applicable. It can be applied, and it can give benefit - I’d accept there was no benefit if the traffic was all marked the same and the resources were constrained in that part of the network.
>> [BB] s/is not applicable/has no effect/
>> OK?
>>
>>>> ---
>>>> 10c) In security considerations, making assertions about Diffserv:
>>>> “Diffserv only works if some packets
>>>>     get less favourable treatment than others.”
>>>> - Again not so, the word “works” is problematic. Offers benefit is perhaps more true?
>> [BB] We've used "makes a difference" this time.
>>
>>>> ---
>>>> 10d) In security considerations, making assertions about Diffserv:
>>>> “So Diffserv has to use
>>>>     traffic rate policers to limit how much traffic can be favoured.”
>>>> - I do not agree in this general conclusion and I think this was discussed in previous comments. This is not the place to make sweeping statements about diffserv, or how it might be used. I would strongly urge you to remove this sweeping statement.
>>>> ---
>>>> 10e) In security considerations, making assertions about Diffserv:
>>>> “In
>>>>     turn, traffic policers require traffic contracts between users and
>>>>     networks as well as pairwise between networks. “
>>>> - Does it actually need to be a contract with a user, I don’t thinks so, there are many cases where diffserv marks are applied in the network, or at an enterprise edge. This is not the document to make such statements about diffserv. I’m going to argue that you need to remove this phrase, people can decide to use L4S without having to address this.
>>>> ---
>> [BB] We've reduced all the discussion of Diffserv in the first para of Sec Consid's by referring back to previous discussion:
>>
>> "Because the L4S service reduces delay without increasing the delay of Classic traffic, it should not be necessary to rate-police access to the L4S service. In contrast, Section 5.2 explains how Diffserv only makes a difference if some packets get less favourable treatment than others, which typically requires traffic policing, which can, in turn, lead to further complexity such as traffic contracts at trust boundaries."
>>
>>
>>> I have now completed a review of L4S ID, I’ve now also reviewed “1.1.  Latency, Loss and Scaling Problems” and I saw a much more polished version of the text with respect to DiffServ. To avoid doubt, here is the text I saw in L4S ID::
>>>
>>>     The Diffserv architecture provides Expedited Forwarding [RFC3246], so
>>>     that low latency traffic can jump the queue of other traffic. If
>>>     growth in high-throughput latency-sensitive applications continues,
>>>     periods with solely latency-sensitive traffic will become
>>>     increasingly common on links where traffic aggregation is low. For
>>>     instance, on the access links dedicated to individual sites (homes,
>>>     small enterprises or mobile devices).  These links also tend to
>>>     become the path bottleneck under load.  During these periods, if all
>>>     the traffic were marked for the same treatment at these bottlenecks,
>>>     Diffserv would make no difference.  Instead, it becomes imperative to
>>>     remove the underlying causes of any unnecessary delay.
>>> ....
>>>
>>>     Active queue management (AQM) was originally developed to solve this
>>>     problem (and others).  Unlike Diffserv, which gives low latency to
>>>     some traffic at the expense of others, AQM controls latency for _all_
>>>     traffic in a class.
>>>
>>> The above in the L4S-ID seems to me to be an acceptable presentation of the diffserv analysis and I did not find issues with this text, and other mentions in L4S-ID.
>>>
>>> I wonder why the arch and L4S-ID both present this argument, ... and if we really do need two sets explaining diffserv inb both documents?
>>>
>>> If this is really needed to be in both specs, could we just use the same words and avoid different flavours of discussion arising from the L4S ARCH document?
>> [BB] I hope the wording in l4s-arch is now more precise, thanks to you pointing out all the failings. Discussion of Diffserv in each document is for different reasons, one of which (the top level problem statement) overlaps:
>>
>> l4s-arch:
>> §4.1 Protocol mechanism allows combination with other identifiers, summarising and referring to ecn-l4s-id for details
>> §5.2 What L4S adds to existing approaches (the problem statement, also summarised in a sentence in the Intro)
>> §8.1 Security Considerations, comparing L4S with Diffserv to illustrate the likelihood that L4S will need policing - now mainly refers back to §5.2
>>
>> ecn-l4s-id:
>> §1.1: Problem statement, justifying the need for an L4S identifier (also summarised in a sentence in the Intro)
>> §5.4: Interaction of L4S with other identifiers
>>
>> The main problem statement is certainly in l4s-arch (which started out incorporating the problem statement written for the L4S BoF).
>>
>> The problem statement in ecn-l4s-id focuses in on the need for an L4S identifier. This needs to start with the bigger problem that L4S is trying to solve. I believe you're saying that you'd prefer that initial step of the rationale in ecn-l4s-id to be replaced by a reference to l4s-arch, but it then loses its logical progression. I think this is a question of editorial style, which I'd rather not change that at this stage.
>>
>>>> 11) Issue: Making assertions about Diffserv:
>>>> “   In particular, because networks tend not to trust end systems to
>>>>        identify which packets should be favoured over others, where
>>>>        networks assign packets to Diffserv classes they often use packet
>>>>        inspection of application flow identifiers or deeper inspection of
>>>>        application signatures.  Thus, nowadays, Diffserv doesn't always
>>>>        sit well with encryption of the layers above IP.  So users have to
>>>>        choose between privacy and QoS.
>>>> “
>>>> This might, or might not be true, but there are no RFCs or references sited, and I am unsure why this para is actually needed in an RFC: If it is, then it needs some specific discussio by the WG, but maybe it would be more prudent to not         say this.
>> [BB] We have had discussion of this on the ML.
>> 	• "Networks tend not to trust end systems to identify which packets..." => Diffserv bleaching discussions.
>> 	• Which then raised the question of how operators that use Diffserv actually determine which packets get what.
>> 		• You may remember I used the example of the UK ISP, PlusNet, which uses DPI to put traffic into different Diffserv classes. Also, I've pointed to home gateways that do this - I remember I referred to a specific AL-Lu home gateway that my operator-employer used to use, which communicated back to a regularly updated central database of traffic signatures.
>> 		• And mobile networks use PI or DPI (see RFC8404)
>> 	• The final logical step was discussed on the tsvwg ML during the BoF stages of L4S (you may recall Kevin Smith's involvement), and it's the inevitable conclusion of the previous steps. Also "doesn't always" ensures it is not overstated.
>> 	• Also, this issue was discussed at length in the IETF more widely during the preparation of RFC 8404 (see §2.2.2).
>>
>> We've done this:
>>      s/they often use packet inspection of application flow identifiers/
>>       /they tend to use packet inspection of application flow identifiers/
>> And we've cited RFC8404 at the end of the sentence about encryption.
>>
>>>> ---
>>>> 12) Issue: This seems not quite true:
>>>>
>>>> “     A.  Per-flow forms of L4S like FQ-CoDel are incompatible with full
>>>>            end-to-end encryption of transport layer identifiers for
>>>>            privacy and confidentiality (e.g. IPSec or encrypted VPN
>>>>            tunnels), because they require packet inspection to access the
>>>>            end-to-end transport flow identifiers.
>>>> “
>>>> I am also unsure why this para is actually needed in this ID!
>>>>
>>>> A network flow ID can be used with FQ, and I think should be noted. I also thought this seemed slightly misleading by not mentioning that TLS over UDP still exposes the port info needed to classify in FQ. I’s suggest removing this ... it seems a well known effect of classification/scheduling on traffic aggregates.
>> [BB] A couple of years ago, we removed all the arguments about FQ that were not relevant to this doc. But we left this one because it is one of the main reasons why it's important to have an architecture that is a superset of FQ and DualQ.
>>
>> Adding a network flow ID defeats the whole point of a confidentiality/privacy solution like IPsec ESP that deliberately encapsulates the transport layer flow ID to obfuscate against inferences made by analysing application flow behaviour.  So I'd rather not suggest that, as it will get thrown out again during SECDIR review.
>>
>> At the end of the parenthesis we've added:
>> (e.g. IPSec or encrypted VPN tunnels, as opposed to TLS over UDP)
>>
>>
>>>> ---
>>>> 13) Ref
>>>> In reference to scavenger service, can you add a reference to an RFC as well as [LEDBAT_AQM]?
>> [BB] I've realized we should have said "e2e scavenger behaviours [RFC6817]".
>> We didn't mean "services" like the LE PHB here, which can be identified by FQ.
>>
>>>> ---
>>>> 14) Query:
>>>> “ Indeed BBRv2 [BBRv2] uses L4S ECN...”
>>>> - Is this necessarily true that ECN is used, and the final spec for BBR2 specifies this? Might it be fairer to say:
>>>> “ Indeed BBRv2 [BBRv2] can use L4S ECN...”
>> [BB] "can use L4S ECN where available"
>>
>>>> ---
>>>> 15) NiT: In 6.4.1:
>>>> “   L4S AQMs will not have to be deployed throughout the Internet before
>>>>     L4S will work for anyone.”
>>>> - Is this “work” for anyone? or “can benefit anyone”?
>> [BB] correct
>>
>>>> ---
>>>> 16) NiT:
>>>> “Deployment in mesh topologies depends on how over-booked the core is.”
>>>> - sentence ends in “is”... which seems strange. Is “over-booked” well understood or should this be “over-provisioned” perhaps?
>>>> I’d suggest something more like:
>>>> “Deployment in a mesh topology depends on how much the core is over-provisioned.”
>> [BB] When I talk with network people, they use 'over-booked' quite frequently. I can't think of a better alternative (it's the opposite of over-provisioned, BTW). Searching for synonyms doesn't help, the best is over-subscribed which is less good. Overbooked is also in wide general use, e.g. the flight/hotel was overbooked. We're going to have to leave it as is. Except apparently it doesn't need a hyphen.
>>
>>
>>>> ---
>>>> 17) NiT: In 6.4.2.:
>>>>     “For any one L4S flow to work, it requires 3 parts to have been
>>>>     deployed.”
>>>> - what does “work” mean here? is this offer benefit or is this dysfunctional in some way unless this happens or what?
>> [BB] "provide benefit", particularly in the context of incentives here.
>>
>>>> ---
>>>> 18) NiT:
>>>> This seems wrong from a diffserv viewpoint:
>>>> “they are required not to change the L4S
>>>>     identifier, merely treating the traffic as best efforts traffic, as
>>>>     they already treat traffic with ECT(1) today.”
>>>> - e.g. could we change to something like :
>>>> “they are required not to change the L4S
>>>>     identifier, merely treating the traffic in the same way as not-ECT traffic, as
>>>>     they already treat traffic with ECT(1) today.”
>> [BB] Good catch.
>> But we've changed it to: "as they might already treat ECT(1) traffic today." as well.
>>
>>>> ---
>>>> 19) NiT:
>>>> “to police L4S traffic in order to
>>>>     protect competing Classic ECN traffic.”
>>>> - Is policing in an order - if so explain, otherwise please rephrase to “to police L4S traffic to protect competing Classic ECN traffic”
>> [BB] -> "so as to protect" which avoids "to ... to"
>>
>>>> ---
>>>> 20) NiT:
>>>> “Their packet classifier (item 2 in Figure 1) could identify
>>>>     such customers against some other field (e.g. source address range)
>>>>     as well as ECN.”
>>>> /ECN/as classifying on the ECN Field/.
>>>> ---
>>>> 21) NiT:
>>>> “The TCP authentication option (TCP-AO [RFC5925]) can be used to
>>>>        detect tampering with TCP congestion feedback.”
>>>> -It seems strange to not also mention that QUICs use of TLS also prevent this tampering.
>> [BB] We've generalized as well:
>> Transport layer authentication such as the TCP authentication option (TCP-AO [RFC5925])
>> or QUIC's use of TLS [RFC9001] can detect any tampering with congestion feedback.
>>
>>>> ---
>>>> 22)Query:  I think some references could need to be normative:
>>>> RFC3168
>>>> RFC8311
>>>> I-D.ietf-tcpm-accurate-ecn
>> [BB] We had not included any normative language at all in l4s-arch, being informational. We took a similar approach to the Diffserv architecture [RFC2475]. One could possibly argue that this sentence pulls in the relevant part of RFC3168 by reference:
>>        For L4S, the names used for the four codepoints of the 2-bit IP-
>>        ECN field are unchanged from those defined in [
>> RFC3168
>> ]: Not ECT,
>>        ECT(0), ECT(1) and CE, where ECT stands for ECN-Capable Transport
>>        and CE stands for Congestion Experienced.
>>
>> But looking at that, it needs explaining better anyway, which makes it clearer that the normative inclusion of RFC3168 is being done by ecn-l4s-id, not l4s-arch:
>>
>>       "L4S uses the ECN field as an identifier [I-D.ietf-tsvwg-ecn-l4s-id]
>>        with the names for the four codepoints of the 2-bit IP-ECN field
>>        unchanged from those defined in [RFC3168]: Not ECT, ECT(0),..."
>>
>> The references to RFC8311 are commentary about RFC8311, not normative. Whereas in ecn-l4s-id, it's definitely normative - as the authority for the ID to be assigned. Similarly, l4s-arch references to AccECN are commentary about it, not normative.
>>
>> _____
>> Finally, this was a really useful review, which has hopefully removed some potential trip-wires. Thank you again.
>> Amazingly, I hadn't already included you in the ACKs - we have now.
>>
>> Cheers
>>
>>
>> Bob
>>
>>
>>>> -----
>>>>
>>>> On 29.07.21, 18:18, "tsvwg on behalf of Wesley Eddy"<tsvwg-bounces@ietf.org on behalf of wes@mti-systems.com>  wrote:
>>>>
>>>>      This message is starting a combined working group last call on 3 of the
>>>>      L4S drafts:
>>>>
>>>>      - Architecture:https://datatracker.ietf.org/doc/draft-ietf-tsvwg-l4s-arch/
>>>>
>>>>      - DualQ:
>>>> https://datatracker.ietf.org/doc/draft-ietf-tsvwg-aqm-dualq-coupled/
>>>>
>>>>      - ECN ID:https://datatracker.ietf.org/doc/draft-ietf-tsvwg-ecn-l4s-id/
>>>>
>>>>      The WGLC will last through 4 weeks from today, and then we'll see what
>>>>      to do next.  Please submit any comments you have on these to the TSVWG
>>>>      list in that timeframe.
>>>>
>>>>      The chairs are considering a possible virtual interim following the
>>>>      close in order to work through feedback received.
>>>>
>>>>      The work on the L4S operational guidance draft is continuing in
>>>>      parallel, but that draft is not being last called yet.
>>>>
>>>>
>> -- 
>> ________________________________________________________________
>> Bob Briscoe
>> http://bobbriscoe.net/

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/