Re: [Tsv-art] Tsvart last call review of draft-ietf-bmwg-b2b-frame-03

"Black, David" <David.Black@dell.com> Wed, 25 November 2020 14:34 UTC

Return-Path: <David.Black@dell.com>
X-Original-To: tsv-art@ietfa.amsl.com
Delivered-To: tsv-art@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8BE013A145B; Wed, 25 Nov 2020 06:34:39 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.1
X-Spam-Level:
X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=dell.com header.b=w4X5/0ko; dkim=pass (1024-bit key) header.d=dell.onmicrosoft.com header.b=kQ+sC8JK
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U8xnbZKQyD8U; Wed, 25 Nov 2020 06:34:37 -0800 (PST)
Received: from mx0a-00154904.pphosted.com (mx0a-00154904.pphosted.com [148.163.133.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5C79B3A1458; Wed, 25 Nov 2020 06:34:37 -0800 (PST)
Received: from pps.filterd (m0170392.ppops.net [127.0.0.1]) by mx0a-00154904.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0APETQD4005297; Wed, 25 Nov 2020 09:34:37 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=smtpout1; bh=FTO9/FT5ji+7QtVfEDlYzjYuLQN5EXVEwUnhagLBWvs=; b=w4X5/0koHEDWVicApisZ1g/++bAEX31Bazg2nOZmhx7yo7I/Q8pML9OJnyZO1oQI+o7e DMLMjfQSnUCQRh9aWqBg/eDIoxpWm+lQNB5Ws4nm58MbK79pvhJqnrV9juQRP5rkd6BT C+z5W7ff8UW9g3dLA67n0eH9V/3NuPsTBtfhEBcbur0ruRQZ/+yqjyG+NIZMToklJ69B D7YEe9QzZLs1r7us801F5MJ20pR5ppk1wpfRp8MsXBU4lF/eUofBq75m9cTGLi+NXBVg 3qFQJ/5UTvRjM8UbuM4AzALeNYvtbwKS1WtNN+nfATcco3tbgcg/TLbp5XiTRuObN03m Lw==
Received: from mx0b-00154901.pphosted.com (mx0a-00154901.pphosted.com [67.231.149.39]) by mx0a-00154904.pphosted.com with ESMTP id 34xxxj6q0h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Nov 2020 09:34:36 -0500
Received: from pps.filterd (m0090350.ppops.net [127.0.0.1]) by mx0b-00154901.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0APEQboE130686; Wed, 25 Nov 2020 09:34:36 -0500
Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2174.outbound.protection.outlook.com [104.47.59.174]) by mx0b-00154901.pphosted.com with ESMTP id 351116cybv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 25 Nov 2020 09:34:36 -0500
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kKwGB3RBXOEdXec/nJhU04hYfd/SAxxNlMHnkXIWfjuops+DCoSEwuUpi7uBt3D2Gy3Iya6DmX3/geXrcZoidYRLqSGAOgiZtaKMEW8ulzxECYhAfAqWZP+LiOjMuKyUsoM+9NEOTCFaPu9GiqnyKo1HEafsAZv938S3UoOCkcpwg7+RRdTR/zhM+Q11Hx5KI3/DKb9B3eVmbDy2UvqlwvntjTr+lUpj3+gS3U7YQ0p8R5IptkczRMmZTRC2IUoWQ8px2nWT0jENj42SLtxNaY7H0HIpEY9mJioxjFyg5Mztb0gvv0s4S+i0hmGcOenfCBvn4KDzFsbRvjAim5rkSw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FTO9/FT5ji+7QtVfEDlYzjYuLQN5EXVEwUnhagLBWvs=; b=ikXC3UPN40WmrVqW/YlrhuII56R8vjy3OprTfXC312X2AeqjKifuh7ZV3M2DMXjnQd+99LHjR5SMczIsz6QyzjbXCT5aSiP7HJ/nWk60IvxQc79D8Ik83EVDyjImx/+AUcNx49FSH5rLJLV76QVjifktu5BxlBd9OpmeAHOt/7OSKq6WFRPWvXtKAr224wiAqZhO+54qPlewRreqKGrOmCmFI6sauMkewU+UFJXXYUzBiY4F4xCRWMqO35EwGSjMn35BiiQeENxT3Zy4MTXV8pPfSmh/upqYZH8vSzGZK6h9SMQkWmOxFBLta0lV7qzt9hPFgAg9eqp7WrRMKibsog==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=dell.com; dmarc=pass action=none header.from=dell.com; dkim=pass header.d=dell.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Dell.onmicrosoft.com; s=selector1-Dell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FTO9/FT5ji+7QtVfEDlYzjYuLQN5EXVEwUnhagLBWvs=; b=kQ+sC8JKmNR1XhmisxYIP5GA9Zq/JW1Brf+dkINw20wKhPON/KZSjrB6eFu5aXDcpP7ccAx7083heMgjKs8uZYA5sjuz1rG5JULPJahYWtVnFBtQhu2xM3Ipy4+PW7lHeYgYg3a0qjGEP1wc+kFSEjFArX6J4yDkl94BRcnBZ8M=
Received: from MN2PR19MB4045.namprd19.prod.outlook.com (2603:10b6:208:1e4::9) by MN2PR19MB4080.namprd19.prod.outlook.com (2603:10b6:208:1e6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3589.22; Wed, 25 Nov 2020 14:34:33 +0000
Received: from MN2PR19MB4045.namprd19.prod.outlook.com ([fe80::2853:5ccc:b023:dce4]) by MN2PR19MB4045.namprd19.prod.outlook.com ([fe80::2853:5ccc:b023:dce4%3]) with mapi id 15.20.3611.020; Wed, 25 Nov 2020 14:34:33 +0000
From: "Black, David" <David.Black@dell.com>
To: "MORTON, ALFRED C (AL)" <acm@research.att.com>, "tsv-art@ietf.org" <tsv-art@ietf.org>
CC: "bmwg@ietf.org" <bmwg@ietf.org>, "draft-ietf-bmwg-b2b-frame.all@ietf.org" <draft-ietf-bmwg-b2b-frame.all@ietf.org>, "last-call@ietf.org" <last-call@ietf.org>, "Black, David" <David.Black@dell.com>
Thread-Topic: Tsvart last call review of draft-ietf-bmwg-b2b-frame-03
Thread-Index: AQHWwnx0aANa5zmswk+xVn2oQ5//DqnXqHMQgACiSQCAAJwM4A==
Date: Wed, 25 Nov 2020 14:34:32 +0000
Message-ID: <MN2PR19MB4045F0FF2FC651C799B61C4C83FA0@MN2PR19MB4045.namprd19.prod.outlook.com>
References: <160623159725.20249.9390987464844223889@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF014764C4F2@njmtexg5.research.att.com> <MN2PR19MB4045E52E2E321662721891C683FB0@MN2PR19MB4045.namprd19.prod.outlook.com> <4D7F4AD313D3FC43A053B309F97543CF014764EAA2@njmtexg5.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF014764EAA2@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels: MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_Enabled=True; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_SiteId=945c199a-83a2-4e80-9f8c-5a91be5752dd; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_Owner=david.black@emc.com; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_SetDate=2020-11-25T14:33:59.3437135Z; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_Name=External Public; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_Application=Microsoft Azure Information Protection; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_ActionId=7b8d4d6e-7eaf-4b19-8b90-c0eafbff6653; MSIP_Label_17cb76b2-10b8-4fe1-93d4-2202842406cd_Extended_MSFT_Method=Manual
authentication-results: research.att.com; dkim=none (message not signed) header.d=none;research.att.com; dmarc=none action=none header.from=dell.com;
x-originating-ip: [72.74.71.221]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: ec325140-d240-41d2-4342-08d8914f39ce
x-ms-traffictypediagnostic: MN2PR19MB4080:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <MN2PR19MB40805FFD8CDAD4F0356421E083FA0@MN2PR19MB4080.namprd19.prod.outlook.com>
x-exotenant: 2khUwGVqB6N9v58KS13ncyUmMJd8q4
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: loazOhlTEfbFCyNFOCNBsQ8wwKu+ZVNpKqZsB3lPuVU+crxCQrFQ6qeKih0ehYK3LWLaK3aVl+IKShZAZuTTf38gN/szxiGHIaiH+hJ6jfkQEdvfU2cxcVrwmeKLTE4kw6aC/Z/GPygPyiJy5fQzUn1Qjq9ycrvKczdZipiVFinsrbNlXaBAIBuST7UsFFfdvf98RFikYdWKBNYxJjHbFOj3lf0lOQOdoofxkockE2Tuuf2QTh2g6QR3TTwLayGQRozmS74o79KddnUftnmUEewi693Ee/eZ7bCQZbuYxUpIO6K1Cs5zlzxYihFP0glX5oajVq6VUr1e4h6g6l60HJgFJ/C+nHj2f71UvPxWLbBXC3QTiK3GmJFgXcXpXrnEtI8PwSTgHGYq9WkQqglQgw==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR19MB4045.namprd19.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(346002)(376002)(39860400002)(396003)(136003)(8676002)(53546011)(7696005)(66946007)(71200400001)(66446008)(66556008)(5660300002)(66476007)(76116006)(52536014)(64756008)(30864003)(26005)(786003)(110136005)(316002)(54906003)(186003)(86362001)(2906002)(6506007)(9686003)(107886003)(478600001)(8936002)(966005)(83380400001)(4326008)(33656002)(55016002); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata: =?utf-8?B?aXRKQmM0dTBkVkdwZ084N2wyYmhKUzVjbUVlNCtLQk5xVmJtWnZsL2I1ZUFP?= =?utf-8?B?S20vczlHZWY1RDVMUDNoZHVJS0VYbEk4ZnliSlpSWk9zREljRG9iL2hsNVpT?= =?utf-8?B?MUlZd0lhU3hqdUdVNXVNby9iU3pmVmkxRlA5Z3BjVEtZVi9VdS9BR1lteXpk?= =?utf-8?B?MHN1NlBtZWtJZFNMSjJGYVZEMnp6cGZMLzh3YVpVd2theHVJY3d6eWFqMVNK?= =?utf-8?B?a1N0Y29XOWo2S3hWTVZOd3dXNkcwWXhjR3ZSOTJKcjE3eTUxeWdHNkQ2RnhG?= =?utf-8?B?Z0ZOaURITXJ6aGxJbzJsMG9PSmN0WWFSdUI4TWYweFhjdHdhRnZQbTlxYlZz?= =?utf-8?B?Rnd0a0pkV0ZCRUhNVUFSZitScFRTYUpQWFc5Q3IzWVFwM1gxNzlyZWNUZmVr?= =?utf-8?B?QlZueVprcDRUdGlVb3FHNjdDbk0rWDR4b3VkNkk0VDFWdE9Nc3cvSXRZbmQ4?= =?utf-8?B?S3E4VlBLYWNVT1craWx4RUd6U2pLNGtxVlJEV21va2w4QjRZV3R0dTZvbUJu?= =?utf-8?B?QVVkTDJaTnliSGJjeUN1MEN5V3pxZkFJV1Z6L2p1eGU1aHArTUJyb0IvSnN5?= =?utf-8?B?b3UrNEYyN3lkNXhtOXRPdjhTNldPeEIyWll6NXlpOVY5bFU1bmxJS3VRM1pa?= =?utf-8?B?d2tiU0JGeDU2RWtsMm1aUmgrcVlrc1ZnbHI3eTFaNjhoRERpTWJ3NWtieEI5?= =?utf-8?B?andnSFRLTFlpWWV4VzFCbTluRlJPZnErUWVoRmlUbUNCSDdKaWRHbkpzeUFm?= =?utf-8?B?NUxPU0JJdFdIcjh6WDYvdXAwYzZXYTgyK2pGeVhtZFpIeWR0ODA5N3drQkVl?= =?utf-8?B?bUJYT2N3dll3YTJyc2ppS0dEbFRxQ2xBLzdPUzNsZDgyYUwwam1GenJjMHZR?= =?utf-8?B?QjE5c09WNS82ZmthUUxnUUZsLzJENHVWNVFTOHpUVzBpOWM0clhWUWl2NnJX?= =?utf-8?B?bmlTQlY2ajVCalNYbDNYVDR4QzM4MTFPMVhSWnl0bjJwc2VGZ2svb09rTWl3?= =?utf-8?B?Ylo2MW5LRE5KYncxVGVTd2lWZkNveCtCdE5PTFlkVW9PNk9wZGdWSERkYk9s?= =?utf-8?B?UUJPcXNsT3FQRnd6b3JnZ3lMc2VUK3ZOTW83blU1ckFRMTF6ZzF6TVJyTEdM?= =?utf-8?B?cC8wVHBRYWRQeEJnUVZPaHFXdkF5eUpodTUydTUzWTdwNHVuMjE0dWN6eVBQ?= =?utf-8?B?VlY1SXlIWmpwK2lZeHJ1emVERFh2Vm5FdTE2UFYzSWxlaWZrMnJaaGErMExv?= =?utf-8?B?WTZSQUdkNFlTTHZwTnRmaG5nOFBUbDNBNnNjM1JhVFFOZGJKcTVNY2hVUzl1?= =?utf-8?Q?ZlZ9hkl+/VnUQ=3D?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: Dell.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MN2PR19MB4045.namprd19.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ec325140-d240-41d2-4342-08d8914f39ce
X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Nov 2020 14:34:33.0421 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 945c199a-83a2-4e80-9f8c-5a91be5752dd
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: VQjMGFkitwtJt9nVwq9z/nFTEmbiV4sD5hpssvtX4fRFw7LVC+h7e6S6e5LZhjCerQPrkoxQXmeYf/zOE+/Yiw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR19MB4080
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-11-25_08:2020-11-25, 2020-11-25 signatures=0
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 mlxlogscore=999 impostorscore=0 bulkscore=0 spamscore=0 suspectscore=0 lowpriorityscore=0 adultscore=0 clxscore=1015 mlxscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2011250091
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 mlxlogscore=999 suspectscore=0 adultscore=0 spamscore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2011240121
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsv-art/x2kkKm7nv6II6lVKoBTaIE534Es>
Subject: Re: [Tsv-art] Tsvart last call review of draft-ietf-bmwg-b2b-frame-03
X-BeenThere: tsv-art@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Review Team <tsv-art.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsv-art>, <mailto:tsv-art-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsv-art/>
List-Post: <mailto:tsv-art@ietf.org>
List-Help: <mailto:tsv-art-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsv-art>, <mailto:tsv-art-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 25 Nov 2020 14:34:40 -0000

Al,

We appear to not be communicating on a key aspect of the concern:

> A burst of 1.7 seconds falls well within the coverage of a 2 second trial to count the
> frames and determine that loss has occurred. We don't need a big margin time-
> wise. Also, the trial duration increases with the search parameters: IOW, if the
> tester is still sending frames after 2 seconds, then obviously we can't allow the trial
> to terminate prematurely.

The 1.6 seconds value is *not* an upper bound - it was measured in practice on an operational conference network as part of what appears to have been a just-in-time slide preparation exercise.  Hence that 1.6 seconds of buffering ought to be viewed as being somewhere in the midst of the expected distribution of buffer times.  For that reason, the upper bound on test duration needs to be set with a safety factor to ensure that even if a bizarre DUT has 10 seconds of buffering, the B2B test depletes that buffering so that the B2B test is testing the forwarding path as intended, not the buffering capacity.

> I want to avoid over-reacting to the buffer bloat case. We all agree that bloated
> buffers are bad, and knowing a DUT has buffering >=1 second may be enough. I
> certainly do not think that trial durations for the B2B Frame benchmark need to be
> as long as you originally suggested, where the overwhelming number of test
> conditions can use a <<1 second burst and 2 second buffer depletion time:

I agree with this approach provided that there is a test procedure (in the B2B test and/or its prerequisites) that will discover the behavior of the bizarre DUT with 10 seconds of buffering and adjust the B2B test time to ensure that this excessive buffering is depleted.

> I hope the approach of the single sentence above works for you.

Based on this discussion, in addition to that sentence (which I think has to contain a "MUST" for situations in which excessive buffering is found), I would like to see a brief explanation of how the procedures in the draft, along with additional test procedures pulled in from RFC 2544 or elsewhere, avoid a 2 second test duration on a DUT with 10 seconds of buffering.

Thanks, --David

> -----Original Message-----
> From: MORTON, ALFRED C (AL) <acm@research.att.com>
> Sent: Wednesday, November 25, 2020 12:00 AM
> To: Black, David; tsv-art@ietf.org
> Cc: bmwg@ietf.org; draft-ietf-bmwg-b2b-frame.all@ietf.org; last-call@ietf.org
> Subject: RE: Tsvart last call review of draft-ietf-bmwg-b2b-frame-03
> 
> 
> [EXTERNAL EMAIL]
> 
> Hi David, some more clarity below.
> Al
> 
> 
> > -----Original Message-----
> > From: Black, David [mailto:David.Black@dell.com]
> > Sent: Tuesday, November 24, 2020 3:02 PM
> > To: MORTON, ALFRED C (AL) <acm@research.att.com>om>; tsv-art@ietf.org
> > Cc: bmwg@ietf.org; draft-ietf-bmwg-b2b-frame.all@ietf.org; last-
> > call@ietf.org; Black, David <David.Black@dell.com>
> > Subject: RE: Tsvart last call review of draft-ietf-bmwg-b2b-frame-03
> >
> > Al,
> >
> > > [acm] I agree with your observation that there are cases where trial
> > duration
> > > should be increased to accommodate the encountered in the DUT, but
> > > not
> > as a
> > > mandate for all testing. I have four factors in mind:
> > > 1. Some of the virtual network DUTs we are testing now have very
> > > small
> > buffers,
> > > and the B2B stream of frames is quite short -- less than 2000
> > frames@10GE in
> > > some cases -- so 2 seconds fully sufficient.
> > > 2. The trial duration is a factor in total test duration, where each
> > trial is one step in
> > > the Binary Search. We need to manage the tension between the time
> > > needed
> > to
> > > reach a search result and confidence that we have depleted the queues.
> > > 3. The RFC 2544 Latency benchmark will tell us if bufferbloat is
> > present.
> > > 4. The current text says "at least 2 seconds".
> > >
> > > So I suggest adding the following text:
> > >
> > >     The duration of the trial MUST be at least 2 seconds, to allow DUT
> > >     buffers to deplete. When RFC2544 Latency measurements indicate that
> > >     large buffers are present in the DUT, the trial duration SHOULD be
> > >     increased to ensure that buffer depletion takes place, without unduly
> > >     extending the total test time.
> >
> > The overall approach of collecting evidence that there is a problem
> > before increasing the 2 second minimum duration is fine, but the
> > details appear to need more attention:
> >
> > 1. Obtaining the RFC 2544 Latency measurements would need to be added
> > to Section 4 (Prerequisites) of this draft to ensure that the buffer
> > size information is available.
> [acm]
> Section 26.2 Latency is a very primitive procedure by today's standards. It is
> common for testers to measure delay on every packet, and report a true average.
> This is enough to alert the operator to the presence of bufferbloat, but there's no
> assurance that the Latency benchmark measures the extent of the buffer time.
> 
> >
> > 2. I did not see any requirements in the RFC 2544 Latency test
> > (Section
> > 26.2) to deplete buffers.  Did I miss something?
> [acm]
> It's an overall requirement in Section 23, after running the trial:
> 
>    d) Wait for two seconds for any residual frames to be received.
> 
> >
> > 3. I would think that the trial duration MUST be increased, not just
> > SHOULD be increased if there is evidence of large buffer size, as
> > buffer depletion appears to be a necessary characteristic of this B2B
> > measurement.
> [acm]
> 
> We need to keep in mind that the sentence(s) in this discussion pertain to the trials
> of the B2B Frame Benchmark testing, in the section that begins:
> 
>   Each trial in the test requires the tester to send a burst of frames (after idle time)
> with the minimum inter-frame gap, and to count the corresponding frames
> forwarded by the DUT.
> 
> Let's say we have a DUT that offers 1.6 seconds of buffering. The Binary Search will
> increase the length of the burst until the buffer drops frames, and then try to find
> the longest burst where frame loss is zero. This simple procedure always waits the
> minimum trial duration for frames to exit the DUT.
> 
> A burst of 1.7 seconds falls well within the coverage of a 2 second trial to count the
> frames and determine that loss has occurred. We don't need a big margin time-
> wise. Also, the trial duration increases with the search parameters: IOW, if the
> tester is still sending frames after 2 seconds, then obviously we can't allow the trial
> to terminate prematurely.
> 
> So, I suggest we use this wording instead:
> 
>        The duration of the trial MUST be at least 2 seconds in addition to
>        the time to send each burst of frames, to allow DUT buffers to deplete.
> 
> and we establish an adaptive waiting time for extreme cases, consistent with
> RFC2544 section 23 item d). For a DUT with 1.6 second buffers, the Trial duration
> would be 3.6 seconds.
> 
> BTW, one tester issue we are trying to fix here is the case when the frame header
> processing rate is sufficiently high that no buffer limit is measureable at the large
> frame sizes, and we implore people not to waste time with the B2B Frame
> Benchmark at those frame sizes. One commercial tester sent bursts up to 30
> seconds in length, and then happily reported the impossible result of 30 second
> buffers.
> 
> I want to avoid over-reacting to the buffer bloat case. We all agree that bloated
> buffers are bad, and knowing a DUT has buffering >=1 second may be enough. I
> certainly do not think that trial durations for the B2B Frame benchmark need to be
> as long as you originally suggested, where the overwhelming number of test
> conditions can use a <<1 second burst and 2 second buffer depletion time:
> 
> > > > Hence, the 2 second minimum duration ought to be increased by at
> > > > least a factor of 10.  I'd suggest changing it to 30 seconds or 60 seconds...
> 
> I hope the approach of the single sentence above works for you.
> 
> 
> 
> >
> > It also looks like the link to the entire slide deck didn't make it
> > into my original review correctly - that slide deck is at:
> > https://urldefense.com/v3/__http://www.taht.net/*d/lca_tcp3.odp__;fg!!
> > BhdT !0oWabwvUrtVlGQY7ZKnFnMa-
> un_e95tfCJcRKIWG7wECiJkVISb4sjd9Salwehw$
> > .  In addition to slide 6 from this slide deck (Figure 1 in the APNIC
> > blog), slide 14 is also relevant to this discussion.
> >
> > Thanks, --David
> >
> > > -----Original Message-----
> > > From: MORTON, ALFRED C (AL) <acm@research.att.com>
> > > Sent: Tuesday, November 24, 2020 11:11 AM
> > > To: Black, David; tsv-art@ietf.org
> > > Cc: bmwg@ietf.org; draft-ietf-bmwg-b2b-frame.all@ietf.org; last-
> > call@ietf.org
> > > Subject: RE: Tsvart last call review of draft-ietf-bmwg-b2b-frame-03
> > >
> > >
> > > [EXTERNAL EMAIL]
> > >
> > > Hi David, Thanks for your review and comment!
> > >
> > > Please see a proposed resolution below, [acm] Al
> > >
> > > > -----Original Message-----
> > > > From: David Black via Datatracker [mailto:noreply@ietf.org]
> > > > Sent: Tuesday, November 24, 2020 10:27 AM
> > > > To: tsv-art@ietf.org
> > > > Cc: bmwg@ietf.org; draft-ietf-bmwg-b2b-frame.all@ietf.org; last-
> > > > call@ietf.org
> > > > Subject: Tsvart last call review of draft-ietf-bmwg-b2b-frame-03
> > > >
> > > > Reviewer: David Black
> > > > Review result: Ready with Issues
> > > >
> > > ...
> > > >
> > > > This draft updates the back-to-back frame testing procedure in RFC
> > > > 2544 to take account of experience.
> > > >
> > > > The draft is in good shape, with one notable exception in Section 5.2:
> > > >
> > > >    The duration of the trial MUST be at least 2 seconds, to allow DUT
> > > >    buffers to deplete.
> > > >
> > > > That duration of 2 seconds has been carried forward from RFC 2544
> > > > without change.  A 2 second duration may have been sufficient to
> > > > deplete buffers in 1999, but that is no longer reliably the case.
> > > > For example, on-site measurement of the network for the 2020 Linux
> > > > Conference in Australia indicated a at least 1.6 seconds of
> > > > buffering, as indicated by Figure 1 at
> > > >
> > https://urldefense.com/v3/__https://blog.apnic.net/2020/01/22/bufferbl
> > oat-
> > may-be-solved-but-its-not-over-
> > yet/__;!!BhdT!wOuQE0NajXs4dT7tdIMVQU5FFpb0JiU0-
> > yK2DOVVn0ecoYjf7mFEABLmlwDk$ ,
> > > > which is slide 6 in from the complete slide deck at:
> > > >
> > https://urldefense.com/v3/__https://blog.apnic.net/2020/01/22/bufferbl
> > oat-
> > may-be-solved-but-its-not-over-
> > yet/__;!!BhdT!wOuQE0NajXs4dT7tdIMVQU5FFpb0JiU0-
> > yK2DOVVn0ecoYjf7mFEABLmlwDk$
> > > > .  Experience with bufferbloat suggests that one network device
> > > > was primarily responsible.  Also, see slide 14 in that slide deck.
> > > >
> > > > That 1.6 seconds measured on an actual network is entirely too
> > > > close to 2 seconds for confidence that buffers will be depleted in
> > > > any
> > tested device.
> > > > Hence, the 2 second minimum duration ought to be increased by at
> > > > least a factor of 10.  I'd suggest changing it to 30 seconds or 60
> > > > seconds as convenient round numbers, and providing the rationale
> > > > that increased buffering in WiFi devices, e.g., home "routers," as
> > > > indicated by experience with bufferbloat measurements, is the
> > > > reason
> > for the
> > > increased duration.
> > > >
> > > [acm]
> > > [acm] I agree with your observation that there are cases where trial
> > duration
> > > should be increased to accommodate the encountered in the DUT, but
> > > not
> > as a
> > > mandate for all testing. I have four factors in mind:
> > > 1. Some of the virtual network DUTs we are testing now have very
> > > small
> > buffers,
> > > and the B2B stream of frames is quite short -- less than 2000
> > frames@10GE in
> > > some cases -- so 2 seconds fully sufficient.
> > > 2. The trial duration is a factor in total test duration, where each
> > trial is one step in
> > > the Binary Search. We need to manage the tension between the time
> > > needed
> > to
> > > reach a search result and confidence that we have depleted the queues.
> > > 3. The RFC 2544 Latency benchmark will tell us if bufferbloat is
> > present.
> > > 4. The current text says "at least 2 seconds".
> > >
> > > So I suggest adding the following text:
> > >
> > >     The duration of the trial MUST be at least 2 seconds, to allow DUT
> > >     buffers to deplete. When RFC2544 Latency measurements indicate that
> > >     large buffers are present in the DUT, the trial duration SHOULD be
> > >     increased to ensure that buffer depletion takes place, without
> > unduly
> > >     extending the total test time.
> > >
> > > I hope this suggestion resolves your issue; thanks for highlighting it!
> > > Al