Re: [bmwg] Martin Duke's Discuss on draft-ietf-bmwg-b2b-frame-03: (with DISCUSS and COMMENT)

"MORTON, ALFRED C (AL)" <acm@research.att.com> Sat, 12 December 2020 15:18 UTC

Return-Path: <acm@research.att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 26F593A1180; Sat, 12 Dec 2020 07:18:50 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.92
X-Spam-Level:
X-Spam-Status: No, score=-1.92 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SMLr1loOXvgj; Sat, 12 Dec 2020 07:18:48 -0800 (PST)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A09D73A116E; Sat, 12 Dec 2020 07:18:48 -0800 (PST)
Received: from pps.filterd (m0049459.ppops.net [127.0.0.1]) by m0049459.ppops.net-00191d01. (8.16.0.43/8.16.0.43) with SMTP id 0BCFFVJh031250; Sat, 12 Dec 2020 10:18:44 -0500
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049459.ppops.net-00191d01. with ESMTP id 35ctjwbfpd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Dec 2020 10:18:43 -0500
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id 0BCFIgDB127951; Sat, 12 Dec 2020 09:18:43 -0600
Received: from zlp30493.vci.att.com (zlp30493.vci.att.com [135.46.181.176]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id 0BCFIaAI127867 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 12 Dec 2020 09:18:36 -0600
Received: from zlp30493.vci.att.com (zlp30493.vci.att.com [127.0.0.1]) by zlp30493.vci.att.com (Service) with ESMTP id 68017400A014; Sat, 12 Dec 2020 15:18:36 +0000 (GMT)
Received: from clph811.sldc.sbc.com (unknown [135.41.107.12]) by zlp30493.vci.att.com (Service) with ESMTP id 454CD400A013; Sat, 12 Dec 2020 15:18:36 +0000 (GMT)
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clph811.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id 0BCFIaXp031769; Sat, 12 Dec 2020 09:18:36 -0600
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clph811.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id 0BCFIRO4031178; Sat, 12 Dec 2020 09:18:28 -0600
Received: from exchange.research.att.com (njmtcas1.research.att.com [135.207.255.86]) by mail-azure.research.att.com (Postfix) with ESMTP id 05C7610A18E0; Sat, 12 Dec 2020 10:18:27 -0500 (EST)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas1.research.att.com ([fe80::e881:676b:51b6:905d%12]) with mapi id 14.03.0487.000; Sat, 12 Dec 2020 10:18:27 -0500
From: "MORTON, ALFRED C (AL)" <acm@research.att.com>
To: Martin Duke <martin.h.duke@gmail.com>, The IESG <iesg@ietf.org>
CC: "draft-ietf-bmwg-b2b-frame@ietf.org" <draft-ietf-bmwg-b2b-frame@ietf.org>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>, Sarah Banks <sbanks@encrypted.net>
Thread-Topic: Martin Duke's Discuss on draft-ietf-bmwg-b2b-frame-03: (with DISCUSS and COMMENT)
Thread-Index: AQHWzn+aIGGQh/sasUSbSHjRLrydaqnvZ33A
Date: Sat, 12 Dec 2020 15:18:26 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF014766D179@njmtexg5.research.att.com>
References: <160755503926.27888.3173906725876085467@ietfa.amsl.com>
In-Reply-To: <160755503926.27888.3173906725876085467@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [24.148.42.167]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2020-12-12_04:2020-12-11, 2020-12-12 signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 clxscore=1011 mlxscore=0 adultscore=0 bulkscore=0 spamscore=0 priorityscore=1501 suspectscore=0 mlxlogscore=999 impostorscore=0 lowpriorityscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012120118
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/UwoU5Cw4X4Iy-5MGJ-yp_h7zJOQ>
Subject: Re: [bmwg] Martin Duke's Discuss on draft-ietf-bmwg-b2b-frame-03: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 12 Dec 2020 15:18:50 -0000

Hi Martin, thanks for your review and comment,
please see my reply, [acm] below,
Al

> -----Original Message-----
...
> 
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
> 
> Thank you for engaging with the TSVART review. Despite the wordsmithing that
> has gone on, I am not sure that we have captured the correct text.
> 
> The proposed change is:
> > I clarified:
> > The duration of the trial MUST include at least 2 seconds in addition to the time
> > required to send and receive each burst of frames, to ensure that DUT buffers to deplete. 
> > and I'll add: 
> > The upper search limit for the time to send each burst MUST be configurable as 
> > high as 30 seconds (buffer time results
> > reported at the configured upper limit are likely invalid, and the test MUST
> > be repeated with a higher search limit).
> 
> But IIUC it's the additional time that needs to scale up. 
[acm] 

In the revised text where David and I reached agreement, we identified 3 time components of the trial duration, making the duration variable: no longer static and at "at least 2 seconds".

1. the time to send the burst of frames (at the back-to-back rate), determined by the search algorithm
2. the time to receive the transferred burst of frames (at the RFC2544 Throughput rate), possibly truncated by buffer overflow, but certainly including the latency of the DUT with or without buffer-bloat
3. at least 2 seconds in addition to the time to receive the burst (2.), to ensure that DUT buffers have depleted.

So, both components 1. and 2. are variables, and the burst receive time component (2.) compensates for large buffers, non-back-to-back burst egress, and anything else that contributes to DUT latency. The final "at least 2 seconds" is simply about making sure the trial is really over before moving on in an automated test - we won't make an error if frames trickle-out very late for some unfortunate reason.

> A layman's reading of
> the document, IMO, suggests that the burst length has a binary search but the 2
> seconds of waiting can be fixed.
[acm] 
Yes, that's right, plus all the other factors above.

So, let's try this, but I'm trying not to extend or complicate the buffer time << 2 seconds testing for the sake of the buffer-bloat case:

-=-=-=-=-=-=-

The duration of the trial includes three REQUIRED components:

1. the time to send the burst of frames (at the back-to-back rate), determined by the search algorithm
2. the time to receive the transferred burst of frames (at the RFC2544 Throughput rate), possibly truncated by buffer overflow, and certainly including the latency of the DUT 
3. at least 2 seconds not overlapping the time to receive the burst (2.), to ensure that DUT buffers have depleted.

The upper search limit for the time to send each burst MUST be configurable as high as 30 seconds (buffer time results reported at or near the configured upper limit are likely invalid, and the test MUST be repeated with a higher search limit).

-=-=-=-=-=-=-=-=-

Does that wording do it?

>  
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
> 
> Other than that, this a well-written document. Thanks!
[acm] 
Thank you!

> 
>