Re: [tsvwg] draft minutes from interim

Jonathan Morton <chromatix99@gmail.com> Sat, 07 March 2020 08:39 UTC

Return-Path: <chromatix99@gmail.com>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5636E3A0D92 for <tsvwg@ietfa.amsl.com>; Sat, 7 Mar 2020 00:39:44 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.052
X-Spam-Level:
X-Spam-Status: No, score=0.052 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iIUjLXj05av7 for <tsvwg@ietfa.amsl.com>; Sat, 7 Mar 2020 00:39:42 -0800 (PST)
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [IPv6:2a00:1450:4864:20::136]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2CFA73A0D8F for <tsvwg@ietf.org>; Sat, 7 Mar 2020 00:39:41 -0800 (PST)
Received: by mail-lf1-x136.google.com with SMTP id p5so3771434lfc.7 for <tsvwg@ietf.org>; Sat, 07 Mar 2020 00:39:41 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Z+SfO1JUtZh1AV7yaAPPVm/JBU8MWpxu25prM6uY+RY=; b=ZTbqKCRmgXRJrDUJEyHdrfgO57v8hRNRBTu2/XKgBUfaGHHlr6IUDiUC0pl3xbBSDs kzEcvqYUDK2zNJzc5Z45LFQTSqJQFpPorlZdxEMQROJ8eyFWH7IXs8sanBfukRRBBNY+ sjM1IlrXuh+uc4Tds87fCPOOuTho3EMBJrEIijfX3bQYSVadIaefjmYoIQwsK74Zf4FT mig+ZrID3r69pto1myBpbNIQVe2fD6xp/uIHa/v3Dt1sjZa0R8xa3WCtFwskjgatCNnj yE8CmYEzf45bcJuLRUmVXY45ww3aKyh0KuxWHbVDgoiDzm/XJXFVC3A0K/yfU2T9oQ+l 15zQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Z+SfO1JUtZh1AV7yaAPPVm/JBU8MWpxu25prM6uY+RY=; b=CXlyTyhW6yCjmNB2gZ1iXaE4ZXUDpEoLJrijC/i6wRqH/4e6kQzC8gFnt3R5QtvDSd oLTzK0gJUEk3YNqSre+56PG2IxZCprJ2dP2jITGYLV6BIWPkFeDif1kuF4nQSF4SMnjL CcYe0p9S4Fp7lzkGHy4J1pUL5ryp19RJ8uG0c6vme8STw4TAw8EstjmuYyO9mV+tQG6c j0e7Tc0X6wrmRARNX7ftz2Jw7h7Os2TNy9dOORbyqJex9LM8xFj1kGxFnR4gRU+0Wuh8 p8LJWGd8ZrRjcU+ldV0mH1z+WkV5UbppuJAmOpzlNrH2S/zcNLRqVHD6aokGDb42jGJQ HVNg==
X-Gm-Message-State: ANhLgQ1XKBClitZNuMuE2hEaXOeiVdle+BgJhWsm004dH04ECHEP7ZBP yIHl8EaajS+NN/tRMAfH5X0X0LP8
X-Google-Smtp-Source: =?utf-8?q?ADFU+vvRSrhrxvM7swTkmapHPJh2cxO4OK5BNry5RMVE?= =?utf-8?q?eciw7j9qxFO7Dx1+AR0u1FLugi0E85orzw=3D=3D?=
X-Received: by 2002:a05:6512:44a:: with SMTP id y10mr4420309lfk.85.1583570379976; Sat, 07 Mar 2020 00:39:39 -0800 (PST)
Received: from jonathartonsmbp.lan (83-245-250-250-nat-p.elisa-mobile.fi. [83.245.250.250]) by smtp.gmail.com with ESMTPSA id w3sm9230842lfe.9.2020.03.07.00.39.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 07 Mar 2020 00:39:39 -0800 (PST)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
From: Jonathan Morton <chromatix99@gmail.com>
In-Reply-To: <46003417-5ff0-b7b2-4038-0285b85a769f@bobbriscoe.net>
Date: Sat, 7 Mar 2020 10:39:25 +0200
Cc: Wesley Eddy <wes@mti-systems.com>, "tsvwg@ietf.org" <tsvwg@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <2A943E39-1924-4D13-8B5B-83DA7AA57F72@gmail.com>
References: <504d5b50-03bd-fd50-b968-b2def7ed6335@mti-systems.com> <46003417-5ff0-b7b2-4038-0285b85a769f@bobbriscoe.net>
To: Bob Briscoe <in@bobbriscoe.net>
X-Mailer: Apple Mail (2.3445.9.1)
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/vmKnw36nauebey-_la5UdAWeJM0>
Subject: Re: [tsvwg] draft minutes from interim
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 07 Mar 2020 08:39:44 -0000
X-List-Received-Date: Sat, 07 Mar 2020 08:39:44 -0000

> On 6 Mar, 2020, at 7:27 pm, Bob Briscoe <in@bobbriscoe.net> wrote:
> 
> During Jonathan's slides, there was a conversation that seems to be completely missing where Greg asked for SCE scenarios to not just show experiments as selected individual time series that could be "cherry picked", but to usually give results over a range of realistic link rates and RTTs and a range of traffic scenarios. Also there was a request to present summary "figures of merit" e.g. queue delay, utilization, rate ratio, packet drop, ECN marking, normalized flow completion time, etc. Also to show "whiskers", e.g. 99-%ile as well as mean, across a number of experiment runs.

I don't think the conversation was *completely* missed in the minutes:

> - Greg White asked if data is anecdotal cases and time series, rather than wider range, CDFs.

At the time, I remember pointing out that for a half-hour talk, we of course had to be selective, but that we could discuss specific tests and presentation formats offline.  Our code is also available for you to perform your own tests and reproduce ours.

The reason we prefer to present time-series charts is because they show the evolution behaviour of the congestion control in a way that makes behavioural anomalies obvious.  In particular, this was the best way to show the robustness of SCE to a shift from an SCE-aware bottleneck to an otherwise identical RFC-3168 bottleneck and back, a test which had been specifically requested by one of the Chairs; we have not yet seen a demonstration of an equivalent scenario in L4S.  We do this to project confidence that SCE works in a wide variety of circumstances, not just the ones actually presented.

Our choices of scenario to present were in part directed towards cases which are considered "difficult", either traditionally with conventional congestion, or through experience with high-fidelity congestion in particular.  We wanted to show that we could handle these "difficult" cases cleanly.  At least one of those cases was one that we have previously had difficulty with, and had been asked to find ways to improve on - so we did, and showed the improvement.

We are now in a process of generating more comprehensive results and with additional forms of data presentation.  These should be ready to show at Vancouver, though it looks increasingly likely that we will have to participate remotely this time.  And of course we will specifically address the question over ECT(1) proposed by the Chairs.

 - Jonathan Morton