Re: [tsvwg] Fwd: New Version Notification for draft-heist-tsvwg-ecn-deployment-observations-02.txt

Pete Heist <pete@heistp.net> Tue, 23 March 2021 09:35 UTC

Return-Path: <pete@heistp.net>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4E3033A24C9 for <tsvwg@ietfa.amsl.com>; Tue, 23 Mar 2021 02:35:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.199
X-Spam-Level:
X-Spam-Status: No, score=-0.199 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=heistp.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z0UO5xZCTV0l for <tsvwg@ietfa.amsl.com>; Tue, 23 Mar 2021 02:35:05 -0700 (PDT)
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ADCFE3A24C2 for <tsvwg@ietf.org>; Tue, 23 Mar 2021 02:35:04 -0700 (PDT)
Received: by mail-wm1-x334.google.com with SMTP id r10-20020a05600c35cab029010c946c95easo10400757wmq.4 for <tsvwg@ietf.org>; Tue, 23 Mar 2021 02:35:04 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=message-id:subject:from:to:cc:date:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=HCnqFlyPnhjTtVvSF2W44bTKbMssyibldLkmS51kPZY=; b=HoBeLgKtGzw2PsXTgjGZTEE+p04Y0vze+FvEKd5iKrAlc4SRxuH666ALtqBzpbW1AG Ii4JndtXQ+I4x99axv3CFUCdbNhDMwsZUgE0QUtRRf1geIsl81hnrQ+dQgOd6bLzfySx QkZVfR6adwUojEkWYQlopG7T73s2vPGEMNKt1FZ3kzqCkLHLDWtMtPSJ+VTHjKJ+4BFl nOJGottOG/cx5XyD4JbRWiXb8kTJAgZNEwraMt3YRauFIeqzW1sNndoSL3+BceRqvqmG irJ71HjoZr83gKKKtquttWcdUHWnP8IqLZosPuSnUuTWmAdac/L8HaGi/ZeZlSbhRnuj pVvw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=HCnqFlyPnhjTtVvSF2W44bTKbMssyibldLkmS51kPZY=; b=TNLVg0ZikwN+MusJIhfXYSoNfiL9pyz4tEvEswjOLTmqIIM5Ju37BzRWEqy4nJoRvC raHVfIw9z6IhQ2vP39XhBOXG1HQ3Y0+P97UbSyKYdboe/OKfgeWvERJ16ZINLDmELybH dNupdZszh1/ggOs8xP/SfHjYeO4bJMGy7G1ivEES30NMaUJgH3rXqJewYsQSsV7XfH1U gUQ2kgosqRBACE5C4QEEMTWP2ZfCcT5hJy+9HPThduub4MqgiBEP/7CWR95uL0DQ/uYy 7FpwPSFiUyBybSUF2OdQsGrFAH51MN+GVLO53RGkiKa8T7gKYSXwhXJPaEg5/XMlGy6a AvPw==
X-Gm-Message-State: AOAM531wDz8h/09dK3UYDvZT+MhpNEpCDpsxK2h2WtpvVs9nw92FttVT 4dXig4XzYuJZN/Mo87NmTXHc/A==
X-Google-Smtp-Source: ABdhPJw4ZVXcj0UCnnXwiq/aF93yrVyUWd0xWAMWIISIJpke1erHIbaOvPG7QfncK+jMfrxfGAtZDg==
X-Received: by 2002:a1c:a985:: with SMTP id s127mr2405022wme.158.1616492101810; Tue, 23 Mar 2021 02:35:01 -0700 (PDT)
Received: from [10.72.0.88] (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id i17sm22133004wrp.77.2021.03.23.02.35.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Mar 2021 02:35:01 -0700 (PDT)
Message-ID: <a60871f9541c0aca95a6990fcf3a30bf6f67376b.camel@heistp.net>
From: Pete Heist <pete@heistp.net>
To: Bob Briscoe <ietf@bobbriscoe.net>
Cc: tsvwg IETF list <tsvwg@ietf.org>, maprg@ietf.org
Date: Tue, 23 Mar 2021 10:35:00 +0100
In-Reply-To: <ab5add35-529e-6d12-5768-693123004338@bobbriscoe.net>
References: <161519742798.12373.8424747645620734074@ietfa.amsl.com> <4cc306ea278dab68741b0c27713bfb7c84522e11.camel@heistp.net> <72467bfe9a38edee74c4ab8e12ec350e23315ec9.camel@heistp.net> <ce2c8397-1bb9-8634-7822-88b9ba6d3b22@bobbriscoe.net> <0f8a406dc56a24d90a76a337074f62a07b4d4aec.camel@heistp.net> <ab5add35-529e-6d12-5768-693123004338@bobbriscoe.net>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.4
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/awI8wHEtNteLqaD2FpplV_cg65k>
Subject: Re: [tsvwg] Fwd: New Version Notification for draft-heist-tsvwg-ecn-deployment-observations-02.txt
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 23 Mar 2021 09:35:09 -0000

Bob,

On Mon, 2021-03-22 at 16:53 +0000, Bob Briscoe wrote:
> Pete,
> 
> On 09/03/2021 01:13, Pete Heist wrote:
> > Hi Bob, comment below...
> > 
> > On Tue, 2021-03-09 at 00:25 +0000, Bob Briscoe wrote:
> > > Pete,
> > > 
> > > Thanks for the updated draft a fortnight ago, with the extra
> > > columns
> > > I
> > > asked for.
> > > 
> > > I've been excluding the rows with obvious pathologies.
> > > 
> > > Of the 90 IPs in your TCP results, 74 have at least one obvious
> > > pathology, leaving only 16 clean.
> > > And actually, there was a pathology suggested in your slides
> > > today
> > > that
> > > isn't in the above,
> > >       If I check for that too, 83 IPs have at least one
> > > pathology,
> > > leaving only 7 clean.
> > > 
> > > Any idea what is going on?
> > > 
> > > 
> > > 
> > > ==Details==
> > > 
> > > In the TCP results, unlike with QUIC, the reverse feedback is
> > > visible.
> > > So I can check for more pathologies.
> > > 
> > > I checked for this obvious pathology, that wasn't directly
> > > mentioned
> > > in
> > > the slides today:
> > > * Congestion experienced (CE) in one direction but no echo
> > > congestion experienced (ECE) feedback in the other.
> > > * Or conversely, echo congestion experienced (ECE) feedback in
> > > one
> > > direction but no congestion experienced (CE) in the other.
> > I believe I've got your question right, but let's see. From the
> > vantage
> > point of the gateway, we don't always get to see CE in one
> > direction
> > and ECE in the other. In fact, it seems a lot of the time we don't.
> > 
> > Say downstream traffic passes through the gateway as ECT(0) and
> > gets
> > marked in an AQM later with CE, before arriving at the user
> > endpoint.
> > We haven't seen that CE at the gateway, but we still see the ECEs
> > in
> > the opposite direction. The same can happen upstream, if CE is
> > marked
> > somewhere after packets leave the gateway. That's why we don't
> > require
> > the presence of CE marks to mark an entry "possible AQM", but we
> > use
> > them if they're present.
> 
> [BB] OK that's a feasible explanation... but the numbers still seem
> weird...
> 
> Can you provide more insight into the topology of this ISP. You say
> it 
> is a co-operative wireless ISP. Are all the access links symmetric?

>From what I've seen, many seem to have roughly symmetric rates, but
this can vary. Whereas my connection tends to be about 2:1 download to
upload, there are some members whose upload rates are typically higher
than download. There are also some with fiber.

Pete

> I'm asking because, in the direction from the LAN, it seems highly 
> unlikely that 38 of the 90 IPs experienced no ECN marking in the 
> upstream access link or the backhaul, but they did experience ECN 
> marking somewhere further into the network (which could include
> another 
> access link for p2p traffic).
> 
> The results for the 38 IPs backhauled via the FQ-CoDel nodes (known
> to 
> support ECN) are even more unexpected (again taking only the TCP 
> results, because they show both directions):
> * In the direction from the LAN, 10 of these 38 IPs saw no ECN
> marking 
> from the LAN side (including none from those FQ-CoDel nodes), but
> they 
> did see the feedback of ECN marking from somewhere deeper into the
> network.
> * The results in the direction from the WAN are more what I would 
> expect: 30 of the 38 IPs saw no ECN marking from the WAN side, but
> they 
> did see the feedback of ECN marking that must have occurred on the
> LAN 
> side (where the FQ-CoDel nodes are).
> 
> 
> Bob
> 
> > 
> > Pete
> > 
> > > Of the 90 IPs in the TCP results, 40 from the WAN have this
> > > pathology
> > > and 38 from the LAN
> > > (taking the direction as that of the data)
> > > Only 16 IPs, don't have this pathology in either direction.
> > > 
> > > I would have thought you guys would have picked up these when
> > > checking
> > > for the ECE/CE >= 2
> > > (and/or zeros).
> > > 
> > > If I include the pathology you mention where ECE/CE < 2
> > > In all, of the 90 IPs in the TCP results, after I exclude any IP
> > > with
> > > a
> > > pathology, I only have 7 IPs left.
> > > 
> > > 
> > > 
> > > Bob
> > > 
> > > PS. I agree it's v strange for an ISP to share capacity between
> > > its
> > > members by how many flows they are using.
> > > 
> > > On 08/03/2021 22:19, Pete Heist wrote:
> > > > I'm following up in this thread to one question that's come up-
> > > > why
> > > > is
> > > > an ISP deploying fq_codel in its backhaul?
> > > > 
> > > > In short, this ISP encourages research and experiments by its
> > > > members,
> > > > and this was a way to both see how much congestion exists, and
> > > > attempt
> > > > to manage it with AQM when needed. Experiments with fq_codel
> > > > were
> > > > started a couple of years ago, and I helped add support for it
> > > > to
> > > > their
> > > > custom router interface early last year. However, it's only
> > > > actually
> > > > been enabled on a couple of routers, because:
> > > > - it takes a while to update routers in this environment
> > > > - some of the routers run kernels prior to fq_codel support
> > > > - congestion doesn't usually last long enough to give it much
> > > > urgency,
> > > > and sometimes it's fixed by just adding capacity
> > > > 
> > > > IMO, fq_codel isn't ideal in this situation. When there
> > > > actually is
> > > > congestion to manage, flow-fairness means that one user can
> > > > dominate
> > > > the queue with many flows, which is exactly one of the problems
> > > > we'd
> > > > like to avoid. Host fairness could improve on this, or better
> > > > yet,
> > > > member fairness using the member database, as members can have
> > > > more
> > > > than one IP/MAC. But, fq_codel is what's available now, and
> > > > generally
> > > > usable enough in this case to get started with AQM.
> > > > 
> > > > On Mon, 2021-03-08 at 11:09 +0100, Pete Heist wrote:
> > > > > -------- Forwarded Message --------
> > > > > Subject: New Version Notification for draft-heist-tsvwg-ecn-
> > > > > deployment-
> > > > > observations-02.txt
> > > > > Date: Mon, 08 Mar 2021 01:57:07 -0800
> > > > > 
> > > > > > A new version of I-D, draft-heist-tsvwg-ecn-deployment-
> > > > > > observations-
> > > > > > 02.txt
> > > > > > has been successfully submitted by Peter G. Heist and
> > > > > > posted to
> > > > > > the
> > > > > > IETF repository.
> > > > > > 
> > > > > > Name:           draft-heist-tsvwg-ecn-deployment-
> > > > > > observations
> > > > > > Revision:       02
> > > > > > Title:          Explicit Congestion Notification (ECN)
> > > > > > Deployment
> > > > > > Observations
> > > > > > Document date:  2021-03-08
> > > > > > Group:          Individual Submission
> > > > > > Pages:          30
> > > > > > URL:
> > > > > > https://www.ietf.org/archive/id/draft-heist-tsvwg-ecn-deployment-observations-02.txt
> > > > > > Status:
> > > > > > https://datatracker.ietf.org/doc/draft-heist-tsvwg-ecn-deployment-observations/
> > > > > > Html:
> > > > > > https://www.ietf.org/archive/id/draft-heist-tsvwg-ecn-deployment-observations-02.html
> > > > > > Htmlized:
> > > > > > https://tools.ietf.org/html/draft-heist-tsvwg-ecn-deployment-observations-02
> > > > > > Diff:
> > > > > > https://www.ietf.org/rfcdiff?url2=draft-heist-tsvwg-ecn-deployment-observations-02
> > > > > > 
> > > > > > Abstract:
> > > > > >      This note presents data gathered at an Internet
> > > > > > Service
> > > > > > Provider's
> > > > > >      gateway on the observed deployment and usage of ECN.
> > > > > > Relevant
> > > > > > IP
> > > > > >      counter and flow tracking data was collected and
> > > > > > analyzed
> > > > > > for
> > > > > > TCP
> > > > > > and other protocols.
> > > > > A few updates in this version:
> > > > > 
> > > > > * Improved statistics for the separated "known" and "unknown"
> > > > > AQMs.
> > > > > 
> > > > > * Clarified the detection method for possible AQMs.
> > > > > 
> > > > > * Added possibility of QUIC-ECN traffic in non-TCP data.
> > > > > 
> > > > > Pete
> > > > > 
> > > > > 
> > 
>