Re: [bmwg] Martin Duke's Discuss on draft-ietf-bmwg-b2b-frame-03: (with DISCUSS and COMMENT)

"MORTON, ALFRED C (AL)" <acm@research.att.com> Tue, 15 December 2020 00:25 UTC

Return-Path: <acm@research.att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 35D713A0827; Mon, 14 Dec 2020 16:25:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.003
X-Spam-Level:
X-Spam-Status: No, score=0.003 tagged_above=-999 required=5 tests=[RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MO9fp61UsOxA; Mon, 14 Dec 2020 16:25:09 -0800 (PST)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0031D3A0803; Mon, 14 Dec 2020 16:25:08 -0800 (PST)
Received: from pps.filterd (m0083689.ppops.net [127.0.0.1]) by m0083689.ppops.net-00191d01. (8.16.0.43/8.16.0.43) with SMTP id 0BF0FhGe000964; Mon, 14 Dec 2020 19:25:07 -0500
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0083689.ppops.net-00191d01. with ESMTP id 35dbnfbpae-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 14 Dec 2020 19:25:07 -0500
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id 0BF0P6HR037877; Mon, 14 Dec 2020 18:25:06 -0600
Received: from zlp30496.vci.att.com (zlp30496.vci.att.com [135.46.181.157]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id 0BF0P1Gd037792 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 14 Dec 2020 18:25:01 -0600
Received: from zlp30496.vci.att.com (zlp30496.vci.att.com [127.0.0.1]) by zlp30496.vci.att.com (Service) with ESMTP id 92539403A432; Tue, 15 Dec 2020 00:25:01 +0000 (GMT)
Received: from clph811.sldc.sbc.com (unknown [135.41.107.12]) by zlp30496.vci.att.com (Service) with ESMTP id 665BC403A42A; Tue, 15 Dec 2020 00:25:01 +0000 (GMT)
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clph811.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id 0BF0P1GI039065; Mon, 14 Dec 2020 18:25:01 -0600
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.255.15]) by clph811.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id 0BF0OqJU037430; Mon, 14 Dec 2020 18:24:52 -0600
Received: from exchange.research.att.com (njmtcas1.research.att.com [135.207.255.86]) by mail-green.research.att.com (Postfix) with ESMTP id 3205910A18F0; Mon, 14 Dec 2020 19:24:51 -0500 (EST)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas1.research.att.com ([fe80::e881:676b:51b6:905d%12]) with mapi id 14.03.0487.000; Mon, 14 Dec 2020 19:24:52 -0500
From: "MORTON, ALFRED C (AL)" <acm@research.att.com>
To: "Scott O. Bradner" <sob@sobco.com>
CC: Martin Duke <martin.h.duke@gmail.com>, The IESG <iesg@ietf.org>, "draft-ietf-bmwg-b2b-frame@ietf.org" <draft-ietf-bmwg-b2b-frame@ietf.org>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] Martin Duke's Discuss on draft-ietf-bmwg-b2b-frame-03: (with DISCUSS and COMMENT)
Thread-Index: AQHWzn+aIGGQh/sasUSbSHjRLrydaqnvZ33AgAT4xYCAAuA24A==
Date: Tue, 15 Dec 2020 00:24:51 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF014766EB8C@njmtexg5.research.att.com>
References: <160755503926.27888.3173906725876085467@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF014766D179@njmtexg5.research.att.com> <BC09F3BD-B046-44D8-8063-3EA10E9DE574@sobco.com>
In-Reply-To: <BC09F3BD-B046-44D8-8063-3EA10E9DE574@sobco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [24.148.42.167]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2020-12-14_13:2020-12-11, 2020-12-14 signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 mlxscore=0 mlxlogscore=999 malwarescore=0 clxscore=1011 bulkscore=0 priorityscore=1501 phishscore=0 impostorscore=0 suspectscore=0 spamscore=0 lowpriorityscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012150000
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/TSRm4ezyygxXSk7Rhmaw_I8dO_Y>
Subject: Re: [bmwg] Martin Duke's Discuss on draft-ietf-bmwg-b2b-frame-03: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 15 Dec 2020 00:25:11 -0000

Hi Scott, thanks for helping with this discussion.

I'm trying to formulate adaptive extra time based on the time it takes to *receive* the burst, with the additional "at least 2 seconds"  waiting time to be sure we received all the packets that might arrive.  Let me try drawing the timeline that's in my mind, and I'll use a buffer-bloat case example of a 1 second buffer (which dominates all other buffers in the DUT). 

One of the key contributions of this memo is recognizing that the buffer is being emptied while the burst of back-to-back frames is simultaneously trying to fill the buffer.  

Assume that the RFC 2544 Throughput is only half of the back-to-back frame rate for the frame size used. 

>From the draft:
   4.  A helpful concept is the buffer filling rate, which is the
       difference between the Max Theoretical Frame Rate (ingress) and
       the Measured Throughput (HeaderProc on egress).  If the actual
       buffer size in frames was known, the time to fill the buffer
       during a measurement can be calculated using the filling rate as
       a check on measurements.  However, the Buffer in the model
       represents many buffers of different sizes in the DUT data path.

So (danger: calculating while typing and drawing!), a 1 second burst of B2B frames only raises the occupation buffer to 50%, and another second of transmission is needed before reaching 100% occupation.

Trial
Time, sec: 0          1          2          3         4          5          6

Sender:    |==========|==========|
Receiver:  |= = = = = |= = = = = |= = = = = |= = = = =|
Waiting Time                                          |          |          |
                                                                            Trial
                                                                            Ends

In the ideal example timeline above, the back-to-back burst stopped exactly when the buffer reached capacity, so there is no loss. The buffer fill rate is half the back-to-back rate. Also, it takes 2 seconds to deplete the buffer and for frames to stop arriving at the receiver. Only then do we start the 2 second waiting time to ensure no more frames will arrive!

While we're here, let's look at a calculation from the memo:

   Corrected DUT Buffer Time =
                          /                                         \
           Implied DUT    |Implied DUT       Measured Throughput    |
        =  Buffer Time -  |Buffer Time * -------------------------- |
                          |              Max Theoretical Frame Rate |
                          \                                         /
        =  2 - [ 2 * 0.5 ] seconds
        =  1 second

and we avoid the error of calculating buffer time based on the sender's burst duration alone.

hope this helps,
Al


> -----Original Message-----
> From: Scott O. Bradner [mailto:sob@sobco.com]
> Sent: Saturday, December 12, 2020 5:18 PM
> To: MORTON, ALFRED C (AL) <acm@research.att.com>
> Cc: Martin Duke <martin.h.duke@gmail.com>om>; The IESG <iesg@ietf.org>rg>;
> draft-ietf-bmwg-b2b-frame@ietf.org; bmwg-chairs@ietf.org; bmwg@ietf.org
> Subject: Re: [bmwg] Martin Duke's Discuss on draft-ietf-bmwg-b2b-frame-03:
> (with DISCUSS and COMMENT)
> 
> this would seem to work if 2 seconds is significantly longer than it takes
> to send the burst - but if it takes 2 second to send the burst
> then 2 seconds extra buffer could easily lose packets - seems to me that
> he extra time should be related to the time it takes to send the burst
> 
> e.g 50% of the burst time but not less than 2 seconds
> 
> Scott
> 
> 
> > On Dec 12, 2020, at 10:18 AM, MORTON, ALFRED C (AL)
> <acm@research.att.com> wrote:
> >
> > Hi Martin, thanks for your review and comment,
> > please see my reply, [acm] below,
> > Al
> >
> >> -----Original Message-----
> > ...
> >>
> >> ----------------------------------------------------------------------
> >> DISCUSS:
> >> ----------------------------------------------------------------------
> >>
> >> Thank you for engaging with the TSVART review. Despite the wordsmithing
> that
> >> has gone on, I am not sure that we have captured the correct text.
> >>
> >> The proposed change is:
> >>> I clarified:
> >>> The duration of the trial MUST include at least 2 seconds in addition
> to the time
> >>> required to send and receive each burst of frames, to ensure that DUT
> buffers to deplete.
> >>> and I'll add:
> >>> The upper search limit for the time to send each burst MUST be
> configurable as
> >>> high as 30 seconds (buffer time results
> >>> reported at the configured upper limit are likely invalid, and the
> test MUST
> >>> be repeated with a higher search limit).
> >>
> >> But IIUC it's the additional time that needs to scale up.
> > [acm]
> >
> > In the revised text where David and I reached agreement, we identified 3
> time components of the trial duration, making the duration variable: no
> longer static and at "at least 2 seconds".
> >
> > 1. the time to send the burst of frames (at the back-to-back rate),
> determined by the search algorithm
> > 2. the time to receive the transferred burst of frames (at the RFC2544
> Throughput rate), possibly truncated by buffer overflow, but certainly
> including the latency of the DUT with or without buffer-bloat
> > 3. at least 2 seconds in addition to the time to receive the burst (2.),
> to ensure that DUT buffers have depleted.
> >
> > So, both components 1. and 2. are variables, and the burst receive time
> component (2.) compensates for large buffers, non-back-to-back burst
> egress, and anything else that contributes to DUT latency. The final "at
> least 2 seconds" is simply about making sure the trial is really over
> before moving on in an automated test - we won't make an error if frames
> trickle-out very late for some unfortunate reason.
> >
> >> A layman's reading of
> >> the document, IMO, suggests that the burst length has a binary search
> but the 2
> >> seconds of waiting can be fixed.
> > [acm]
> > Yes, that's right, plus all the other factors above.
> >
> > So, let's try this, but I'm trying not to extend or complicate the
> buffer time << 2 seconds testing for the sake of the buffer-bloat case:
> >
> > -=-=-=-=-=-=-
> >
> > The duration of the trial includes three REQUIRED components:
> >
> > 1. the time to send the burst of frames (at the back-to-back rate),
> determined by the search algorithm
> > 2. the time to receive the transferred burst of frames (at the RFC2544
> Throughput rate), possibly truncated by buffer overflow, and certainly
> including the latency of the DUT
> > 3. at least 2 seconds not overlapping the time to receive the burst
> (2.), to ensure that DUT buffers have depleted.
> >
> > The upper search limit for the time to send each burst MUST be
> configurable as high as 30 seconds (buffer time results reported at or
> near the configured upper limit are likely invalid, and the test MUST be
> repeated with a higher search limit).
> >
> > -=-=-=-=-=-=-=-=-
> >
> > Does that wording do it?
> >
> >>
> >> ----------------------------------------------------------------------
> >> COMMENT:
> >> ----------------------------------------------------------------------
> >>
> >> Other than that, this a well-written document. Thanks!
> > [acm]
> > Thank you!
> >
> >>
> >>
> >
> > _______________________________________________
> > bmwg mailing list
> > bmwg@ietf.org
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/bmwg__;!
> !BhdT!1uRJDJBUadSunB4ZCkgOTzg3ZssPtiufcyrsTcxEc1F67df5q4YNUa9IYHacnsA$