From nobody Mon Oct 25 06:48:42 2021 Return-Path: X-Original-To: tsvwg@ietfa.amsl.com Delivered-To: tsvwg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3621B3A0A10 for ; Mon, 25 Oct 2021 06:48:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.668 X-Spam-Level: X-Spam-Status: No, score=-1.668 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K3agznbeVprL for ; Mon, 25 Oct 2021 06:48:36 -0700 (PDT) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 804B33A0CC1 for ; Mon, 25 Oct 2021 06:47:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1635169610; bh=oLpFmIEJpFG0W8RHTOjMF4BkaXmdRRlvY2OGBO/lB1A=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=hGUKX4FYplsLg5md4DR0XtM9khPWEO0oh9P9Jh/UQpvU/UnLsVOjrQkryzbvyosYv bGKhUp50YIZPb9RuIogsvuFen/ysUB1f2gK/N8V0ZWgTgjP5s3jFxETl4F8m/eJTc/ xCpv6uQMdqJ8g1iCsGfVLMx1qRhwaWG/BZ+NZZG0= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [10.11.12.24] ([134.76.241.253]) by mail.gmx.net (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MMGNC-1mLp0B3KbO-00JKKw; Mon, 25 Oct 2021 15:46:49 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.7\)) From: Sebastian Moeller In-Reply-To: Date: Mon, 25 Oct 2021 15:46:48 +0200 Cc: Gorry Fairhurst , "tsvwg@ietf.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <7dd8896c-4cd8-9819-1f2a-e427b453d5f8@mti-systems.com> <6cd9cf3c-cdc0-4274-6c5b-11a4aebf268b@erg.abdn.ac.uk> To: Bob Briscoe X-Mailer: Apple Mail (2.3608.120.23.2.7) X-Provags-ID: V03:K1:pP6BhTgjr/dogtJkBjRsQ8Ja1MdMut3Bhi/uCn/F8SphR83/GWc ov33PU5K3TYe5FXfhPG/aNUJOeI7IdaCC+1dJu72rH42bWPGp1Md2b1c9YhvCFJuZUS6W1u vG3CPSlHfAah88+qORG2DWaQG16dRxBi6CvqawJJeltO6AP77y7gghWtKn+3pS5UPmfD0BT kn3hSc8Uxrw5n0zjZ95Fg== X-UI-Out-Filterresults: notjunk:1;V03:K0:+RrMiQcU7dg=:s7NST4J7Vk4b2gpx4QlWTX T+Yg4ncrUfco3BN2cEFbccYJ1vafAjdov/BqlR0cfNqhVe1IfkPCYrGZNHZm40ynMFRzBqb3J 79jmVhkOQ/++k5S0Jz+IpCpK26stgXvCwWIo9x7Brjs+D/cgHZ9DkQlAHwFhbEtQY+dyU4vI3 SseHVjrbh8sCySRrncOdr8EnlQgNJDTw/+EhS/zPfylST50HWq1Za/0RugYZZZNi85oI7J/Jt etPNxvtYTsBIT7JaIRmLAy27t9ZJSMjiiScMGhjLtolh4XJW4Kf2ob1V9SqK8Bh4F/wXPmuSH o23SeoQitWCFbKK9xKsUOFEhpjOh+iv78YlYcWhzdJ0m2+gOuQwi8gpk6FDRbJdUCH6/P1hNe moOg2MfjDfbfQANAHByGiU20lWKwM7D/nNxqeBygL1f45DYAJoiLjMquavyOggFIKhfT/ldye yHwCFfLPmx6PUG+MMUa84SE6KuglCvQWpy7/AZsHtgJDY8sVK7MDejqKHgvTnSiN/c1AkY40z +6XHsfXpE14fIjv9db2FeANhR8LfYvg/Kgd7Z+YNsEPZiVSp1bROLuuHO2F/eM32fpk3qwgEw ut3A2qc72Fx3K4QQESPlFe6WH6NXNC3z15EM/mSHSIqt4V1SU/HdbUUMfu7qan1JFIrJVotMO eu5jaVdAZ8uD+33VTJvAe0ETDdSn4xoWXrZJErU4c8WoqFc6yVAZqdYzpBvRQpcT06MSvEXQq l7oBPC+3eiy6afPcjpPaPIrpbnsRB8ook2lvtXSWPtP2EhrsbtqV15ZeIgBZrV50ddCkxsv5W ewNXakauWClkv/lyYAphzr20XDXdSHoLJVV4A8BjdbDvvpg6CrS5e9iCst+fkZbYnOVYJTLYQ zpEqaZ4U2DteBpVDBwpfbTS9cym0kNzAZC+4C/KRADpmjknCxNyzGpnkryp6jxS9cSY6LI6jB rZ3X/2spNsM3ACNZxmaxtshc+TcIZkruNiwuk6MBv+v6nlVcF4ZsWiVfxxOmNxuIIMW8w79Pq LRFD7LbBb9QlInWb7tgSZDrazlfWAvM1iblVbgt+UkFPXLT9X/+9xRMMFA/IdwV1LPYBCpp0I aU4ae+alCO0uMc= Archived-At: Subject: Re: [tsvwg] Updated review: start of WGLC on L4S drafts: L4S Arch X-BeenThere: tsvwg@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Transport Area Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Oct 2021 13:48:41 -0000 Hi Gorry, hi Bob, as far as I can tell this now leaks factually wrong "averaged base RTT = to a CDN" number into even more places... Even if Bob's recent opinion, = that the average RTT to the _closest_ CDN is a suitable proxy for = "internet RTT" is accepted, it at least needs to be described = veridically as what it is, calling that number "averaged base RTT to a = CDN" does not do justice to the source that number was generated from. Regards Sebastian P.S.: I do not accept that the RTT to the closest CDN is a decent proxy = for the internet, as not all content providers/ISPs work with all CDNs = so it does really not matter if I sit in Lagos, Nigeria with 4ms RTT to = a google CDN if the content I need sits 100ms away at the nearest CDN2 = site... and even in "rich" countries it is not guaranteed that the path = to any content is equally close. If Bob would argue for average RTT to = the biggest X CDNs he might have a point, but "to the closest" simply is = painting too rosy a picture here. > On Oct 25, 2021, at 14:15, Bob Briscoe wrote: >=20 > Gorry, >=20 > Thank you again for yet another careful review.=20 > No comment =3D accepted. For qualifications or push back, pls see [BB] = inline... >=20 >=20 > On 13/08/2021 12:00, Gorry Fairhurst wrote: >> Adding to my own review, after re-reading L4S ID ...=20 >>=20 >> Gorry=20 >>=20 >> On 12/08/2021 10:44, Gorry Fairhurst wrote:=20 >>> I reviewed Low Latency, Low Loss, Scalable Throughput (L4S) Internet = Service Architecture during the WGLC. I support publication, but I think = there are issues that would need to be addressed relating to the way L4S = is presented.=20 >>>=20 >>> Best wishes,=20 >>>=20 >>> Gorry=20 >>>=20 >>>=20 >>> This is a review of draft-ietf-tsvwg-l4s-arch-10:=20 >>>=20 >>> 1a) Issue: =E2=80=9CIt is increasingly common for _all_ of a user's = applications at any=20 >>> one time to require low delay: interactive Web, Web services, = voice,=20 >>> conversational video, interactive video, interactive remote = presence,=20 >>> instant messaging, online gaming, remote desktop, cloud-based=20 >>> applications and video-assisted remote control of machinery and=20= >>> industrial processes. =E2=80=9C=20 >>> - I generally agree on the value for a wide range of applications = and sect 6.1 appears to contain a helpful list. ~ However, I disagree = that the word =E2=80=9Call=E2=80=9D is necessary or correct in this = section. I think there are cases that do not fit. Background software = update, retrieval of logging data, some cases where loss protection is = more important than for other traffic, etc, So I insist there are = examples that do not require this. I expect I could find more examples, = but I=E2=80=99d prefer the word =E2=80=9Call=E2=80=9D to be removed, to = leave this as a generalisation.=20 >=20 > [BB] This no way intends to say "all applications require low delay".=20= > The words bracketing around the "all" are not to be overlooked... = /Increasingly common/ and /at any one time/. > How about this to make that point clearer: >=20 > "At any one time, it is increasingly common for /all/ of the traffic = in a bottleneck link (e.g. a household's Internet access) to come from = applications that prefer low delay:..." >=20 >=20 >>> ---=20 >>> 1b) Issue: I=E2=80=99d also quibble with: =E2=80=9Cso that all of a = user's applications can shift to it when their stack is updated.=E2=80=9C = - again, is the word =E2=80=9Call=E2=80=9D really necessary or correct = here?=20 >=20 > [BB] OK. "So that a user's applications can shift..." >=20 >>> ---=20 >>> 2) NiT=20 >>> =E2=80=9CThe L4S=20 >>> approach also requires component mechanisms at the endpoints to=20= >>> fulfill its goal.=E2=80=9D=20 >>> - True, can you provide examples of what these mechanisms are (or a = cross ref)? e.g., are they Protocol Mechanisms (as they are called in = section 4.1)? or is this more the to Host Mechanisms (section 4.3) ?=20 >=20 > [BB] Yes a bit abstract isn't it. I've actually rewritten parts of the = whole para, that were also abstract: >=20 > L4S is designed for incremental deployment. It is possible to = deploy > the L4S service at a bottleneck link alongside the existing best > efforts service [DualPI2Linux] so that unmodified applications can > start using it as soon as the sender's stack is updated. Access > networks are typically designed with one link as the bottleneck for > each site (which might be a home, small enterprise or mobile = device), > so deployment at either or both ends of this link should give = nearly > all the benefit in the respective direction. With some transport > protocols, namely TCP and SCTP, the sender has to check for = suitably > updated receiver feedback, whereas with more recent transport > protocols such as QUIC and DCCP, all receivers have always been > suitable. >=20 > This document presents the L4S architecture, by describing and > justifying the component parts and how they interact to provide the > scalable, low latency, low loss Internet service. It also details > the approach to incremental deployment, as briefly summarized = above. >=20 >=20 >=20 >>> ---=20 >>> 3) Issue or reword.=20 >>> =E2=80=9CAlthough DCTCP as-is 'works' well over the public=20 >>> Internet, most implementations lack certain safety features = that=20 >>> will be necessary once it is used outside controlled = environments=20 >>> like data centres (see Section 6.4.3 and Appendix A).=E2=80=9D=20= >>> - If this is published by the iETF, then I think this needs to have = consensus that DCTP is acceptable. I don=E2=80=99t now which RFC states = that.=20 >>> Later this ID states: =E2=80=9C DCTCP needs some safety concerns to = be fixed for general use=20 >>> over the public Internet (see Section 4.3 of=20 >>> [I-D.ietf-tsvwg-ecn-l4s-id]),=E2=80=9D=20 >>> which seems much closer to what is needed in this section.=20 >=20 > [BB] Ah yes, the intent was to reassure that is works well over = wide-area RTTs. I can see how "it works over the public Internet" would = be misconstrued. > Although DCTCP as-is functions well over wide-area > round trip times, most implementations lack certain safety > features that would be necessary for use outside controlled > environments like data centres (see Section 6.4.3 and Appendix = A). >=20 > ---=20 >>> 4) NiT=20 >>> =E2=80=9CWith=20 >>> Classic congestion controls, such as Reno or Cubic, because = flow=20 >>> rate has scaled since TCP congestion control was first = designed in=20 >>> 1988, it now takes hundreds of round trips (and growing) to=20 >>> recover after a congestion signal (whether a loss or an ECN = mark)=20 >>> as shown in the examples in Section 5.1 and [RFC3649].=20 >>> =E2=80=9C=20 >>> - This only seems like a correct statement, if the flow is a bulk / = high volume flow that is using a high capacity path? Please check and = clarify.=20 >=20 > [BB] We will certainly add=20 > "assuming the flow lasts long enough".=20 > But the "hundreds of round trips and growing" statement is accurate. = There is no assumption of a high BDP path - the first example with a = Cubic recovery time of 4.3s (166 rounds) in Section 5.1 is lower than = the the global average for a download from a CDN - at least using the = best data I can find. >=20 > I agree that both x-references only describe the scaling, not where we = are on the scale. Rather than putting loads of detail in the terminology = definition, we've added the following after the example it refers to in = =C2=A7 5.1: > "For a feel of where the global average lone-flow download sits on = this scale at the time of writing (2021), according to [BDPdata] = globally averaged fixed access capacity was 103Mb/s in 2020 and averaged = base RTT to a CDN was 25-34ms in 2019. Averaging of per-country data was = weighted by Internet user population. So a lone CUBIC flow would at best = take about 200 round trips (5 s) to recover from each of its sawtooth = reductions, if the flow even lasted that long. This is described as 'at = best' because it assume everyone uses an AQM, whereas In reality most = users still have a bloated tail-drop buffer. So likely average recovery = time would be at least 4x 5 s, if not more, because RTT under load would = be at least double, and recovery time depends on the square of RTT." >=20 > That's why we said "hundreds". Is that reasonable? > Here's the figures: >=20 >=20 > avg fixed downstream [Mb/s] > avg CDN RTT under load [ms] > avg BDP under load > Recovery time [s] > Recovery time [#RTTs] > Global avg > 103 > 35ms=20 > (25-34ms base RTT + avg queue - half sawtooth) > 438kB > 6.6s > 188 > Example ref'd in Draft > 120 > 25.5ms > (30ms max RTT - half sawtooth) > 383kB > 4.2s > 166 >=20 > This assumes Cubic. > The top row has the bottom half of the sawtooth in Reno-friendly mode, = and the top half in Cubic. There is no formula for a sawtooth in = transition, so we've taken the average of the two: 6.8s & 6.4s. > The bottom row is fully in Reno-friendly mode still. >=20 > Reference:=20 > [BDPdata] Briscoe, B., "PI2 Parameters", Technical Report = TR-BB-2021-001 arXiv:2107.01003 [cs.NI], July 2021, = .=20 >=20 >=20 >>> ---=20 >>> 5) NiT=20 >>> =E2=80=9Csuch as QUIC.=E2=80=9D=20 >>> - Reference needed for [RFC9000].=20 >=20 > [BB] Transport protocol references like this are given when they = appear elsewhere, but we've added this one here as well, to support the = statement about Reno being the default CC in QUIC. >=20 >>> ---=20 >>> 6) NiT=20 >>> =E2=80=9C[I-D.ietf-tsvwg-ecn-l4s-id] recommends ECT(1)=E2=80=9D=20 >>> draft-ietf-tsvwg-ecn-l4s-id appears to =E2=80=9Cspecify=E2=80=9D the = use of ECT(1) for this purpose. Please check whether ECT(1) is only = recommended.=20 >>> ---=20 >>> 7) Issue: TCP is a core protocol, and L4S support depends on = Accurate ECN: Is accurate ECN a normative? (see comment 22)=20 >>> =E2=80=9CWork to standardize and implement more=20 >>> accurate ECN feedback for TCP (AccECN) is in=20 >>> progress [I-D.ietf-tcpm-accurate-ecn],=E2=80=9D=20 >=20 > [BB] When I asked about this before (years ago), the WG and chairs = agreed on it being informative. > Reasoning: AccECN should be a normative ref in a scalable CC spec that = requires AccECN (for instance, Acc-ECN is normative in = draft-briscoe-iccrg-prague-congestion-control). > But there is no normative statement that references AccECN in = l4s-arch, it's always referred to informatively, like QUIC or DCCP is. > Similarly for ecn-l4s-id, but we'll discuss that in your other review. >=20 >>> ---=20 >>> 8) NiT In section 2, ref useful:=20 >>> =E2=80=9Chigh level explanation of how FQ=E2=80=9D=20 >>> - Please expand and add reference for FQ.=20 >=20 > [BB] We've expanded FQ. But there is no ref to an L4S-FQ - the only = description is in this draft - other than the Linux patch. And where = this draft describes it, it refers to the appropriate section of the = FQ-CoDel RFC. >=20 >>> ---=20 >>> 9) Typo:=20 >>> =E2=80=9C The 'Low Loss=E2=80=9D part of the name denotes=E2=80=9D=20= >>> - This has mixed quote marks. In other places, double quotes are = used.=20 >>>=20 >>> ---=20 >>>=20 >>> 10) Issues relating to critique of diffserv:=20 >>>=20 >>> 10a) Making assertions about Diffserv:=20 >>> I really don=E2=80=99t like this choice of words, it=E2=80=99s far = too general a conclusion:=20 >>> =E2=80=9C Also, Diffserv only works for a small subset of the = traffic on a=20 >>> link. =E2=80=9C=20 >>> - Diffserv works fine when all traffic is marked with a set of = DSCPs - I hope you=E2=80=99d agree, but there are cases where it = wouldn=E2=80=99t give benefit, e.g. when all traffic on a congested = link/router is marked with one code point, whereas there may be benefit = if all traffic on a link/router is marked with one code point, when that = traffic later shares bandwidth with traffic carrying other marks.=20 >=20 > [BB] I think this implies you'd be happier if we wrote: > "Also, Diffserv can only provide a latency benefit if a small = subset of the traffic on a bottleneck link requests low latency." > OK? >=20 >>> ---=20 >>> 10b) Making assertions about Diffserv:=20 >>> =E2=80=9CAs already explained, it is not applicable when all the=20 >>> applications in use at one time at a single site (home, small=20= >>> business or mobile device) require low latency.=E2=80=9D=20 >>> - See above. I really don=E2=80=99t like saying it is not = applicable. It can be applied, and it can give benefit - I=E2=80=99d = accept there was no benefit if the traffic was all marked the same and = the resources were constrained in that part of the network.=20 >=20 > [BB] s/is not applicable/has no effect/ > OK? >=20 >>> ---=20 >>> 10c) In security considerations, making assertions about Diffserv:=20= >>> =E2=80=9CDiffserv only works if some packets=20 >>> get less favourable treatment than others.=E2=80=9D=20 >>> - Again not so, the word =E2=80=9Cworks=E2=80=9D is problematic. = Offers benefit is perhaps more true?=20 >=20 > [BB] We've used "makes a difference" this time. >=20 >>> ---=20 >>> 10d) In security considerations, making assertions about Diffserv:=20= >>> =E2=80=9CSo Diffserv has to use=20 >>> traffic rate policers to limit how much traffic can be = favoured.=E2=80=9D=20 >>> - I do not agree in this general conclusion and I think this was = discussed in previous comments. This is not the place to make sweeping = statements about diffserv, or how it might be used. I would strongly = urge you to remove this sweeping statement.=20 >>> ---=20 >>> 10e) In security considerations, making assertions about Diffserv:=20= >>> =E2=80=9CIn=20 >>> turn, traffic policers require traffic contracts between users = and=20 >>> networks as well as pairwise between networks. =E2=80=9C=20 >>> - Does it actually need to be a contract with a user, I don=E2=80=99t = thinks so, there are many cases where diffserv marks are applied in the = network, or at an enterprise edge. This is not the document to make such = statements about diffserv. I=E2=80=99m going to argue that you need to = remove this phrase, people can decide to use L4S without having to = address this.=20 >>> ---=20 >=20 > [BB] We've reduced all the discussion of Diffserv in the first para of = Sec Consid's by referring back to previous discussion: >=20 > "Because the L4S service reduces delay without increasing the delay of = Classic traffic, it should not be necessary to rate-police access to the = L4S service. In contrast, Section 5.2 explains how Diffserv only makes a = difference if some packets get less favourable treatment than others, = which typically requires traffic policing, which can, in turn, lead to = further complexity such as traffic contracts at trust boundaries." >=20 >=20 >> >>>=20 >>=20 >> I have now completed a review of L4S ID, I=E2=80=99ve now also = reviewed =E2=80=9C1.1. Latency, Loss and Scaling Problems=E2=80=9D and = I saw a much more polished version of the text with respect to DiffServ. = To avoid doubt, here is the text I saw in L4S ID::=20 >>=20 >> The Diffserv architecture provides Expedited Forwarding [RFC3246], = so=20 >> that low latency traffic can jump the queue of other traffic. If=20= >> growth in high-throughput latency-sensitive applications = continues,=20 >> periods with solely latency-sensitive traffic will become=20 >> increasingly common on links where traffic aggregation is low. For=20= >> instance, on the access links dedicated to individual sites = (homes,=20 >> small enterprises or mobile devices). These links also tend to=20 >> become the path bottleneck under load. During these periods, if = all=20 >> the traffic were marked for the same treatment at these = bottlenecks,=20 >> Diffserv would make no difference. Instead, it becomes imperative = to=20 >> remove the underlying causes of any unnecessary delay.=20 >> ....=20 >>=20 >> Active queue management (AQM) was originally developed to solve = this=20 >> problem (and others). Unlike Diffserv, which gives low latency to=20= >> some traffic at the expense of others, AQM controls latency for = _all_=20 >> traffic in a class.=20 >>=20 >> The above in the L4S-ID seems to me to be an acceptable presentation = of the diffserv analysis and I did not find issues with this text, and = other mentions in L4S-ID.=20 >>=20 >> I wonder why the arch and L4S-ID both present this argument, ... and = if we really do need two sets explaining diffserv inb both documents?=20 >>=20 >> If this is really needed to be in both specs, could we just use the = same words and avoid different flavours of discussion arising from the = L4S ARCH document?=20 >=20 > [BB] I hope the wording in l4s-arch is now more precise, thanks to you = pointing out all the failings. Discussion of Diffserv in each document = is for different reasons, one of which (the top level problem statement) = overlaps: >=20 > l4s-arch:=20 > =C2=A74.1 Protocol mechanism allows combination with other = identifiers, summarising and referring to ecn-l4s-id for details > =C2=A75.2 What L4S adds to existing approaches (the problem statement, = also summarised in a sentence in the Intro) > =C2=A78.1 Security Considerations, comparing L4S with Diffserv to = illustrate the likelihood that L4S will need policing - now mainly = refers back to =C2=A75.2 >=20 > ecn-l4s-id: > =C2=A71.1: Problem statement, justifying the need for an L4S = identifier (also summarised in a sentence in the Intro) > =C2=A75.4: Interaction of L4S with other identifiers >=20 > The main problem statement is certainly in l4s-arch (which started out = incorporating the problem statement written for the L4S BoF).=20 >=20 > The problem statement in ecn-l4s-id focuses in on the need for an L4S = identifier. This needs to start with the bigger problem that L4S is = trying to solve. I believe you're saying that you'd prefer that initial = step of the rationale in ecn-l4s-id to be replaced by a reference to = l4s-arch, but it then loses its logical progression. I think this is a = question of editorial style, which I'd rather not change that at this = stage. >=20 >>=20 >> >>>=20 >>=20 >>> 11) Issue: Making assertions about Diffserv:=20 >>> =E2=80=9C In particular, because networks tend not to trust end = systems to=20 >>> identify which packets should be favoured over others, where=20= >>> networks assign packets to Diffserv classes they often use = packet=20 >>> inspection of application flow identifiers or deeper = inspection of=20 >>> application signatures. Thus, nowadays, Diffserv doesn't = always=20 >>> sit well with encryption of the layers above IP. So users = have to=20 >>> choose between privacy and QoS.=20 >>> =E2=80=9C=20 >>> This might, or might not be true, but there are no RFCs or = references sited, and I am unsure why this para is actually needed in an = RFC: If it is, then it needs some specific discussio by the WG, but = maybe it would be more prudent to not say this.=20 >=20 > [BB] We have had discussion of this on the ML.=20 > =E2=80=A2 "Networks tend not to trust end systems to identify = which packets..." =3D> Diffserv bleaching discussions. > =E2=80=A2 Which then raised the question of how operators that = use Diffserv actually determine which packets get what. > =E2=80=A2 You may remember I used the example of the UK = ISP, PlusNet, which uses DPI to put traffic into different Diffserv = classes. Also, I've pointed to home gateways that do this - I remember I = referred to a specific AL-Lu home gateway that my operator-employer used = to use, which communicated back to a regularly updated central database = of traffic signatures. > =E2=80=A2 And mobile networks use PI or DPI (see = RFC8404) > =E2=80=A2 The final logical step was discussed on the tsvwg ML = during the BoF stages of L4S (you may recall Kevin Smith's involvement), = and it's the inevitable conclusion of the previous steps. Also "doesn't = always" ensures it is not overstated.=20 > =E2=80=A2 Also, this issue was discussed at length in the IETF = more widely during the preparation of RFC 8404 (see =C2=A72.2.2). >=20 > We've done this: > s/they often use packet inspection of application flow = identifiers/ > /they tend to use packet inspection of application flow = identifiers/ > And we've cited RFC8404 at the end of the sentence about encryption. >=20 >>>=20 >>> ---=20 >>> 12) Issue: This seems not quite true:=20 >>>=20 >>> =E2=80=9C A. Per-flow forms of L4S like FQ-CoDel are = incompatible with full=20 >>> end-to-end encryption of transport layer identifiers for=20= >>> privacy and confidentiality (e.g. IPSec or encrypted VPN=20= >>> tunnels), because they require packet inspection to access = the=20 >>> end-to-end transport flow identifiers.=20 >>> =E2=80=9C=20 >>> I am also unsure why this para is actually needed in this ID!=20 >>>=20 >>> A network flow ID can be used with FQ, and I think should be noted. = I also thought this seemed slightly misleading by not mentioning that = TLS over UDP still exposes the port info needed to classify in FQ. I=E2=80= =99s suggest removing this ... it seems a well known effect of = classification/scheduling on traffic aggregates.=20 >=20 > [BB] A couple of years ago, we removed all the arguments about FQ that = were not relevant to this doc. But we left this one because it is one of = the main reasons why it's important to have an architecture that is a = superset of FQ and DualQ.=20 >=20 > Adding a network flow ID defeats the whole point of a = confidentiality/privacy solution like IPsec ESP that deliberately = encapsulates the transport layer flow ID to obfuscate against inferences = made by analysing application flow behaviour. So I'd rather not suggest = that, as it will get thrown out again during SECDIR review. >=20 > At the end of the parenthesis we've added: > (e.g. IPSec or encrypted VPN tunnels, as opposed to TLS over UDP) >=20 >=20 >>> ---=20 >>> 13) Ref=20 >>> In reference to scavenger service, can you add a reference to an RFC = as well as [LEDBAT_AQM]?=20 >=20 > [BB] I've realized we should have said "e2e scavenger behaviours = [RFC6817]".=20 > We didn't mean "services" like the LE PHB here, which can be = identified by FQ. >=20 >>> ---=20 >>> 14) Query:=20 >>> =E2=80=9C Indeed BBRv2 [BBRv2] uses L4S ECN...=E2=80=9D=20 >>> - Is this necessarily true that ECN is used, and the final spec for = BBR2 specifies this? Might it be fairer to say:=20 >>> =E2=80=9C Indeed BBRv2 [BBRv2] can use L4S ECN...=E2=80=9D=20 >=20 > [BB] "can use L4S ECN where available" >=20 >>> ---=20 >>> 15) NiT: In 6.4.1:=20 >>> =E2=80=9C L4S AQMs will not have to be deployed throughout the = Internet before=20 >>> L4S will work for anyone.=E2=80=9D=20 >>> - Is this =E2=80=9Cwork=E2=80=9D for anyone? or =E2=80=9Ccan benefit = anyone=E2=80=9D?=20 >=20 > [BB] correct >=20 >>> ---=20 >>> 16) NiT:=20 >>> =E2=80=9CDeployment in mesh topologies depends on how over-booked = the core is.=E2=80=9D=20 >>> - sentence ends in =E2=80=9Cis=E2=80=9D... which seems strange. Is = =E2=80=9Cover-booked=E2=80=9D well understood or should this be = =E2=80=9Cover-provisioned=E2=80=9D perhaps?=20 >>> I=E2=80=99d suggest something more like:=20 >>> =E2=80=9CDeployment in a mesh topology depends on how much the core = is over-provisioned.=E2=80=9D=20 >=20 > [BB] When I talk with network people, they use 'over-booked' quite = frequently. I can't think of a better alternative (it's the opposite of = over-provisioned, BTW). Searching for synonyms doesn't help, the best is = over-subscribed which is less good. Overbooked is also in wide general = use, e.g. the flight/hotel was overbooked. We're going to have to leave = it as is. Except apparently it doesn't need a hyphen.=20 >=20 >=20 >>> ---=20 >>> 17) NiT: In 6.4.2.:=20 >>> =E2=80=9CFor any one L4S flow to work, it requires 3 parts to = have been=20 >>> deployed.=E2=80=9D=20 >>> - what does =E2=80=9Cwork=E2=80=9D mean here? is this offer benefit = or is this dysfunctional in some way unless this happens or what?=20 >=20 > [BB] "provide benefit", particularly in the context of incentives = here. >=20 >>> ---=20 >>> 18) NiT:=20 >>> This seems wrong from a diffserv viewpoint:=20 >>> =E2=80=9Cthey are required not to change the L4S=20 >>> identifier, merely treating the traffic as best efforts traffic, = as=20 >>> they already treat traffic with ECT(1) today.=E2=80=9D=20 >>> - e.g. could we change to something like :=20 >>> =E2=80=9Cthey are required not to change the L4S=20 >>> identifier, merely treating the traffic in the same way as = not-ECT traffic, as=20 >>> they already treat traffic with ECT(1) today.=E2=80=9D=20 >=20 > [BB] Good catch. > But we've changed it to: "as they might already treat ECT(1) traffic = today." as well. >=20 >>> ---=20 >>> 19) NiT:=20 >>> =E2=80=9Cto police L4S traffic in order to=20 >>> protect competing Classic ECN traffic.=E2=80=9D=20 >>> - Is policing in an order - if so explain, otherwise please rephrase = to =E2=80=9Cto police L4S traffic to protect competing Classic ECN = traffic=E2=80=9D=20 >=20 > [BB] -> "so as to protect" which avoids "to ... to" >=20 >>> ---=20 >>> 20) NiT:=20 >>> =E2=80=9CTheir packet classifier (item 2 in Figure 1) could identify=20= >>> such customers against some other field (e.g. source address = range)=20 >>> as well as ECN.=E2=80=9D=20 >>> /ECN/as classifying on the ECN Field/.=20 >>> ---=20 >>> 21) NiT:=20 >>> =E2=80=9CThe TCP authentication option (TCP-AO [RFC5925]) can be = used to=20 >>> detect tampering with TCP congestion feedback.=E2=80=9D=20 >>> -It seems strange to not also mention that QUICs use of TLS also = prevent this tampering.=20 >=20 > [BB] We've generalized as well: > Transport layer authentication such as the TCP authentication option = (TCP-AO [RFC5925]) > or QUIC's use of TLS [RFC9001] can detect any tampering with = congestion feedback. >=20 >>> ---=20 >>> 22)Query: I think some references could need to be normative:=20 >>> RFC3168=20 >>> RFC8311=20 >>> I-D.ietf-tcpm-accurate-ecn=20 >=20 > [BB] We had not included any normative language at all in l4s-arch, = being informational. We took a similar approach to the Diffserv = architecture [RFC2475]. One could possibly argue that this sentence = pulls in the relevant part of RFC3168 by reference: > For L4S, the names used for the four codepoints of the 2-bit IP- > ECN field are unchanged from those defined in [ > RFC3168 > ]: Not ECT, > ECT(0), ECT(1) and CE, where ECT stands for ECN-Capable = Transport > and CE stands for Congestion Experienced. =20 >=20 > But looking at that, it needs explaining better anyway, which makes it = clearer that the normative inclusion of RFC3168 is being done by = ecn-l4s-id, not l4s-arch: >=20 > "L4S uses the ECN field as an identifier = [I-D.ietf-tsvwg-ecn-l4s-id]=20 > with the names for the four codepoints of the 2-bit IP-ECN field=20= > unchanged from those defined in [RFC3168]: Not ECT, ECT(0),..." >=20 > The references to RFC8311 are commentary about RFC8311, not normative. = Whereas in ecn-l4s-id, it's definitely normative - as the authority for = the ID to be assigned. Similarly, l4s-arch references to AccECN are = commentary about it, not normative. >=20 > _____ > Finally, this was a really useful review, which has hopefully removed = some potential trip-wires. Thank you again. > Amazingly, I hadn't already included you in the ACKs - we have now. >=20 > Cheers >=20 >=20 > Bob >=20 >=20 >>>=20 >>> -----=20 >>>=20 >>> On 29.07.21, 18:18, "tsvwg on behalf of Wesley = Eddy" wrote:=20= >>>=20 >>> This message is starting a combined working group last call on 3 = of the=20 >>> L4S drafts:=20 >>>=20 >>> - = Architecture:https://datatracker.ietf.org/doc/draft-ietf-tsvwg-l4s-arch/=20= >>>=20 >>> - DualQ:=20 >>> https://datatracker.ietf.org/doc/draft-ietf-tsvwg-aqm-dualq-coupled/=20= >>>=20 >>> - ECN = ID:https://datatracker.ietf.org/doc/draft-ietf-tsvwg-ecn-l4s-id/=20 >>>=20 >>> The WGLC will last through 4 weeks from today, and then we'll = see what=20 >>> to do next. Please submit any comments you have on these to the = TSVWG=20 >>> list in that timeframe.=20 >>>=20 >>> The chairs are considering a possible virtual interim following = the=20 >>> close in order to work through feedback received.=20 >>>=20 >>> The work on the L4S operational guidance draft is continuing in=20= >>> parallel, but that draft is not being last called yet.=20 >>>=20 >>>=20 >>=20 >=20 > --=20 > ________________________________________________________________ > Bob Briscoe =20 > http://bobbriscoe.net/