Re: [tsvwg] I-D Action: draft-ietf-tsvwg-l4sops-00.txt

Greg White <g.white@CableLabs.com> Fri, 07 May 2021 15:27 UTC

Return-Path: <g.white@CableLabs.com>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7073E3A266D for <tsvwg@ietfa.amsl.com>; Fri, 7 May 2021 08:27:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.101
X-Spam-Level:
X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cablelabs.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rreEtZZXS0Qo for <tsvwg@ietfa.amsl.com>; Fri, 7 May 2021 08:27:46 -0700 (PDT)
Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2134.outbound.protection.outlook.com [40.107.220.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CC5523A27E3 for <tsvwg@ietf.org>; Fri, 7 May 2021 08:26:57 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HMgrLBwmoqkcGbKARpgtOJdAKF5BvQYYDQYauMzaGntmZ+fqaJLjPRzOfnwuBJ1ePkSPR5bF371umJ0F4HCZCmsQugDVZlyJTJvUUmfZQUyQnD9+CBr6B9TpKPKl8whMmRMLHlH2NMDf5mPt3/IsVum0sIjxM7z42rYt14AX4Pt2cL68O8CuBAEJJa6QB56jk9Za4THANOcXWUjkxcSd8ymdSX5XuB7cElhUoJ6rNaJIKccSV8YnIObrgpHVbBAu/Wz6OdsuIOCEQyoVZP4QuXri4H/qSxs0JXxoT6QLhKfsJ1QnG63tEG1PZClVeUWy5zprikbloJhI5ZV+50F/Sw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/yH+ZBS5wTdB/6Ctk+v71c9HgxMi630/DPffvB3lRyM=; b=ILJk84E72/o5HLKJA6qvIu2g7B+ZI6YcE694Uuj1KzzWImOBGn4hNXBnL0V8WkTguhKZr/bt9xbFj3jMI+NMEw+VL7qpBGcr0+2HStCS+WKQ0Df5FMt/5mHs8rKxKm+MkAXFG4nY/c7SVoo2sn8wYBX4oeh4ci/sMdU08uaDrQiy0WR2I0iFkhOtzSHF/2rOE4hnVGtN5AKmt7BmTARuiS12HM0nanmj4gGj9n6gtHOdKqpj74aCFxE567bses49azwP12q1CbXYY+trq3cGtA4lxL0vmlsHRBFCpPmbFBMPvV1wfKLpbgh3DGBAV1gq6rI7mC2vXhw2b1ki7cqx/w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cablelabs.com; dmarc=pass action=none header.from=cablelabs.com; dkim=pass header.d=cablelabs.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cablelabs.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/yH+ZBS5wTdB/6Ctk+v71c9HgxMi630/DPffvB3lRyM=; b=PXCOciurzysvK8Of8OOtWQmF9yFQG2DPGyFQegG1ojVp1WY2ZbAGODO4gHO1344373v6jRbRcxKyzwWuLxWCOXuNwvicBtbw+PyiJeJ5iqP/UaE4fHmcg+L/GX3rpYVu70S9K4afYIAju10ogDBx2ZgIEyToI2fAj7hcxleyloBvBjq77SKe60lcc1ZkQtYkZ2rSTG4MnHtYrLGwDKP3+85nB8ev+4K8YByNvHEJ/RqEJoyXpqXGHNbEdayCbnda5/Cye5YeY6QkefdZtonqC905XGHes5o6Dq0xzKk8/XpF6g6F4GnNAvDEUJKTIByPrrNZvV2IKmeFl6a22H2adg==
Received: from CY4PR06MB2821.namprd06.prod.outlook.com (2603:10b6:903:130::11) by CY4PR0601MB3602.namprd06.prod.outlook.com (2603:10b6:910:8b::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.41; Fri, 7 May 2021 15:26:55 +0000
Received: from CY4PR06MB2821.namprd06.prod.outlook.com ([fe80::8161:6d07:dea0:8696]) by CY4PR06MB2821.namprd06.prod.outlook.com ([fe80::8161:6d07:dea0:8696%8]) with mapi id 15.20.4087.047; Fri, 7 May 2021 15:26:55 +0000
From: Greg White <g.white@CableLabs.com>
To: Sebastian Moeller <moeller0@gmx.de>
CC: "tsvwg@ietf.org" <tsvwg@ietf.org>
Thread-Topic: [tsvwg] I-D Action: draft-ietf-tsvwg-l4sops-00.txt
Thread-Index: AQHXQg+ithPpZgnifUuy9aSRG662RqrVbjGAgADJB4CAAYrlAA==
Date: Fri, 07 May 2021 15:26:55 +0000
Message-ID: <3830918D-0DF5-43C0-AA91-B5C4300EE1E3@cablelabs.com>
References: <162026122806.29902.12130492993431477595@ietfa.amsl.com> <0D313261-6F18-45AA-A40E-7CBCDF229EAD@cablelabs.com> <55F0F23A-EBCA-4703-A2B8-176AD26789C9@gmx.de>
In-Reply-To: <55F0F23A-EBCA-4703-A2B8-176AD26789C9@gmx.de>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/16.44.20121301
authentication-results: gmx.de; dkim=none (message not signed) header.d=none;gmx.de; dmarc=none action=none header.from=CableLabs.com;
x-originating-ip: [73.243.9.160]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 43561f90-fdf7-45f1-d34b-08d9116c8c2e
x-ms-traffictypediagnostic: CY4PR0601MB3602:
x-microsoft-antispam-prvs: <CY4PR0601MB36026FC748F2A5E083B0697FEE579@CY4PR0601MB3602.namprd06.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: sOjoGGBB1xtlXJpSQycEgm8iu965hx6DV1ZUOPQ6AyINScFCvKlqEQjNKoVbJ+sTqX8Pz/BxiE5bQ3lwlC110I6MAXYiPrWJjvSS57443gjMehJUIv8OotDEX4lHZ24eRHDC3pax1laON3D4iMm/TETgHIA8ydH02e1Hw6K6Rm4WYxNo+5uhLdu1f9nvdhScVR+cAwe3OU5LnSCjmxR+JBx7QSZOkz/bPiA0Q61LWfEGElaM4PM8qh22WbP5iqH/Ng9+v68p4kx1Wg/wJFLFfl37+uJQWtEeG3LtFiJQrXoSApef86gblXwrO2mNnzs3tBTsmkwKfkBCN9ofWuR9opVIJ2w5wY+KzYanFyXEDXqUZ8/IX5vwdvuKao898lHBBxqcfgJw/lt8e+5OPGQ1PxxTJhBaF8EEFCoOgGQXh6G8HOIPNbonLuV0ly3xGdq+pXdtV20ZoyP3O2sAAjkdghomAEG2ZXc27ejqVrquudN2jmXtBZEcsrCCYM51Un0KjT/1z3mnoQv+STHAb9VsEk3JCqtXXsbnZa+ixFW30ao1BFcNSZ83qhGSORD3mctsebvnPqGx66+5OgYFPq0L6uTAcSsBaqnyxltVwesP+tJIZ6CWhR5uIJKB6836/7Q3+/eXZlg35HAHUjdU3vSo6oAkLc6EweT1022GMN6zcOhfgDmN2QOk3ZmN0DtdHdRTSTyCKPD0ECxCmekXJsLEUr9x8mcg0r2/MON5ILGRjCQnYKGjkW5UGTfdbJd+qUoY
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CY4PR06MB2821.namprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(376002)(396003)(39840400004)(136003)(346002)(366004)(76116006)(26005)(66446008)(316002)(83380400001)(36756003)(186003)(66946007)(86362001)(6506007)(966005)(2616005)(53546011)(38100700002)(6512007)(4326008)(8676002)(66574015)(8936002)(478600001)(6916009)(71200400001)(66476007)(6486002)(122000001)(66556008)(64756008)(5660300002)(2906002)(33656002)(45980500001)(85282002); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata: PJrfHVBrzaY9OrVvNtCKD57CsL/KLjsRYCI/zNIKwXrw40ZjNMANgMihmx/RX3pVAmj+iW359QT10AWX/pSZyf77KmM/2y37LT0xHG7tfPGsE7HvS/u7UkoGwe5gYTBJ+Rb0JP9Ec1imOCsE13N29LbW8zGmLK+3ahXi9R/WFy+L3I5/HSCMO0LQpP2aD++iarIXvdNZe+Nteo+Psx029TOJRGf1c4G3pN7DDPNhyo17N8oiu6JD+6RpmKZMBwBC77dCcOpdObDyu4Tlgut8zq7VgPJNfKbfU55Ubmxtga0xXB7ups+g+uKbejOsbJp65JdiQqekLblPPgtVlCGvi/2yAXzqdPBlLGpatcVu/JedrfMup6ohJFHpxvhgPnpAnhvR8V+udKgZOxdMjMpJTcZGu9SZatyRfv1UMpynmoXeyvNLo/AcROozQsWhIl4h7WqueWArg5CWWAUZQvU/CiWyojqVdm8eQDZxyk1gCWBT41j3d5+kVP0hPv5OdAd++xIifuDOzq4Dz2gBWbhqDTL1QoSZfXqQw1gI/cmYnJYuEhRjSKO7DIMPPHwaoMV+BtgiFgo4or+rcBfPBFrPkLfq5OuaHV2k3hMSExcLLxZ7DqlRBz3Xar96nx83oPxG+xsmZKsjyewCJ2srR94wORCiZpa33VaBCsx83qXRmStlaSqWVn8b2EYZTA5ZIrsQ6RAwxpdlaoYlxmXsz0lL077S/9Nhcw5y07zoNW9OfVXDsqCqazWy22sqKND0vvaDge6aI09d+AteFKKk9pk7Ky8aZOb4LxZMlMRxhX/BMcmvdqd0y0sc/f+Ym6IiB/eiU4mmTHEFKzMOlAEsCv+H3E/905DadfcHueezpJdT4I3e2PDcaBg0j+acUEIz2PA9gx9UuBYdEAp8VeOg+OUpJK0ZUUWORnBt473PN4dca63LboMc6SCaPWiMPVs5tTVRT3GeUorXJIcaCvSPoKwNa0H/jVULdVsScDhztLvOciDT6PiUrmpsT7cYZ/0MhrbRbZ9bp4kP5Sf50RwqWs3tq3XOjB5b9+lBubbBszPbPuxutSeCY7m5LB/08tXYWl0jOWIQkKb3452Ps7M2Bex2EVcmU9W39f15e407GpTES72o0bOpwQdRFwPnlVPxnishuKWa/iCL6KI8/bwqZv37UlIQDs4ZGi+m7v4LEbEGJaBKo4L6bV09jngXEECVM6zgq0BpUE21ymXS2SpNjZn5A7XJHkqFzDjXkp8gW2r0yyOydBeXa0ma0DD1YenS4+dOJVolJFOd4izMi+KiTer/oVItoQT9Mnc0vM1hIPYRJtbNQbiXIm1Zb3VoTp1+Ob8o
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <96748EB172091542A79979498F7054AD@namprd06.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: cablelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: CY4PR06MB2821.namprd06.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 43561f90-fdf7-45f1-d34b-08d9116c8c2e
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 May 2021 15:26:55.3565 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: ce4fbcd1-1d81-4af0-ad0b-2998c441e160
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: VxEnMf02Py/P7b0KkM/2dD1x5sos+4d3C3ilH4QgVqSERMI6UGpp9ct/wHxbqNZ+Nxv2NpvgLuuhyXxnCPk7Uw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR0601MB3602
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/sbKfJ_g3PHgcM-VQp0QAQflHNXo>
Subject: Re: [tsvwg] I-D Action: draft-ietf-tsvwg-l4sops-00.txt
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 07 May 2021 15:27:52 -0000

Hi Sebastian,

Thanks for pointing this paper out.  I had not seen it previously, and will include a mention of it in the draft.  The results seem a bit puzzling though.  They report that, of the ~5% of IPv4 packets with a non-zero ECN field, 94% of them were CE!  It is hard to believe that this is actual evidence of ECN marking in the wild.  It seems more like evidence of some kind of ToS munging happening for traffic on this ISP's link between NYC and Sao Paulo.  

While in terms of total packets that CAIDA dataset is huge, in terms of visibility to traffic in the wild (roughly 80-90% of which is directly peered locally as I understand it) it might not be that representative.  Does anyone have further insight into this paper, or the peculiarities of the CAIDA equinix-nyc trace?

So far, I think the data from Akamai and Apple have been the most useful in understanding the situation.

-Greg



On 5/6/21, 3:53 AM, "Sebastian Moeller" <moeller0@gmx.de> wrote:

    Hi List, hi Greg,

    sorry for being late, but...

    ...speaking of evidence for ECN marking in the wild; here an analysis of CAIDA internet "back-bone link " traffic from 2018 https://ieeexplore.ieee.org/document/8909187 that has not beed discussed as far as I can tell. The data showed between ~5% of IPv4 packets carrying something other than NotECT in the ECN bitfield, a more detailed look at port resolution showed 0.26% (port 80) or 0.3% (port 443) in IPv4. I do not think this was broken down by network, but indicates non-zero ECN usage in the wild in 2018. IMHO the percentage of traffic might be a better proxy for ECN prevalence than number of networks. That said for figuring out how many/which operators should be pro-actively informed of the coming changes identifying the networks seems helpful.



    > On May 6, 2021, at 05:54, Greg White <g.white@CableLabs.com> wrote:
    > 
    > All,
    > 
    > I had hoped to upload this further in advance of the interim meeting next week so that participants would have more chance to review it beforehand. 
    > 
    > There are quite a few changes from the last individual draft, as can be seen in the diff: https://www.ietf.org/rfcdiff?url1=draft-white-tsvwg-l4sops-02&url2=draft-ietf-tsvwg-l4sops-00 
    > And, there remain a few TODO items as well.  These changes/TODOs were the mainly result of review comments on the mailing list, though I received some offline comments and suggested text as well.
    > 
    > I'll do a relatively quick summary of the changes during the timeslot on Monday, but for now, the most notable changes are:
    > 
    > New Section 3.1 summarizing recent studies of RFC3168 deployment (Jake, Pete, please review and see if the text is an accurate summary).

    [SM] See above for an additional relevant study with 2018 data that might be worth adding in the next revision.

    > New Section 6 discussing actions that can be taken by an operator of FQ bottlenecks.

    [SM] There I find:
    "This unfairness is compounded by the fact that the FQ	
     	   scheduler will already be causing unfairness to flows within the	
     	   tunnel relative to flows that are not tunneled."

    This argues that the AQM "should" divine the number of true capacity seeking flows in the compound tunnel aggregate and assign the proper share. I argue that this is impossible to robustly solve in general, and, in the case of encrypted tunnels also arguably the wrong thing to do, as doing so would introduce additional, potential severe re-ordering in the encrypted flow. But a partial solutions already exist, on IPv6 a tunnel could make use of the flow-label field to give additional flow identifier to AQMs. Arguably that is what tunnels should do, as that keeps the policy decision how much reordering one will accept in the hand of the tunnel operators.



    > New Section 7 discussing conclusion of the L4S experiment (Sebastian, for now I went with the text proposed on the mailing list, we can revise it in the next draft if needed).

    [SM] Much appreciated, while I have my quibbles, getting this section in feels quite helpful (and I do not pretend my initial draft was in any way complete and/or perfect). But for completeness, I still think it a good idea to lay out the gamut of potentially required/recommended actions at network termination. The argument will be better the more objective it is, and it already carries a noticeable pro-L4S bias.


    Regards
    	Sebastian


    > 
    > -Greg
    > 
    > 
    > On 5/5/21, 6:34 PM, "tsvwg on behalf of internet-drafts@ietf.org" <tsvwg-bounces@ietf.org on behalf of internet-drafts@ietf.org> wrote:
    > 
    > 
    >    A New Internet-Draft is available from the on-line Internet-Drafts directories.
    >    This draft is a work item of the Transport Area Working Group WG of the IETF.
    > 
    >            Title           : Operational Guidance for Deployment of L4S in the Internet
    >            Author          : Greg White
    >    	Filename        : draft-ietf-tsvwg-l4sops-00.txt
    >    	Pages           : 18
    >    	Date            : 2021-05-05
    > 
    >    Abstract:
    >       This document is intended to provide guidance in order to ensure
    >       successful deployment of Low Latency Low Loss Scalable throughput
    >       (L4S) in the Internet.  Other L4S documents provide guidance for
    >       running an L4S experiment, but this document is focused solely on
    >       potential interactions between L4S flows and flows using the original
    >       ('Classic') ECN over a Classic ECN bottleneck link.  The document
    >       discusses the potential outcomes of these interactions, describes
    >       mechanisms to detect the presence of Classic ECN bottlenecks, and
    >       identifies opportunities to prevent and/or detect and resolve
    >       fairness problems in such networks.  This guidance is aimed at
    >       operators of end-systems, operators of networks, and researchers.
    > 
    > 
    >    The IETF datatracker status page for this draft is:
    >    https://datatracker.ietf.org/doc/draft-ietf-tsvwg-l4sops/
    > 
    >    There is also an HTML version available at:
    >    https://www.ietf.org/archive/id/draft-ietf-tsvwg-l4sops-00.html
    > 
    > 
    >    Please note that it may take a couple of minutes from the time of submission
    >    until the htmlized version and diff are available at tools.ietf.org.
    > 
    >    Internet-Drafts are also available by anonymous FTP at:
    >    ftp://ftp.ietf.org/internet-drafts/
    > 
    > 
    >