Re: [bmwg] AD Review of draft-ietf-bmwg-evpntest

Sarah B <sarah.banks@gmail.com> Mon, 02 December 2019 22:54 UTC

Return-Path: <sarah.banks@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EC0F012009C for <bmwg@ietfa.amsl.com>; Mon, 2 Dec 2019 14:54:19 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Level:
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yRiA_qFInt8s for <bmwg@ietfa.amsl.com>; Mon, 2 Dec 2019 14:54:16 -0800 (PST)
Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A4A72120098 for <bmwg@ietf.org>; Mon, 2 Dec 2019 14:54:16 -0800 (PST)
Received: by mail-pg1-x52f.google.com with SMTP id x7so512280pgl.11 for <bmwg@ietf.org>; Mon, 02 Dec 2019 14:54:16 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=qiH7BdpfgSjy3nYlisssmUH2L8a0hwgbs56mUCkJxQ0=; b=V6WKKD5jCQU4mHJrPB/bwIbmPr7CFquBjx/7kSww2P//DUOYMPHnQEnWwkUKR5LB+d jaMxDfF7ibwLDESAEp020joMhpHPCxJ8/eyME7H7EEnhkJ0OqOHdBD+u/S/khkA3D4LU naRaYvAqlmbxXZl26hunuuwoKQfDqbQYRYrJ/7t94E3J+zoEaNzcp3EWjosVh/orucEO Fn8fUFyeneyeZiiY8H+bqWrCmY7hD2Amk+0+IGyxbpvXkROQ59mL6Xwy6rzZQE9/3tqK lnpFSCvPa4ahgnXez4bfyOfu+jpNmAs/CSJefbsWEeVuGkfWMd4phF+xgipxtOomBa8u 6YdA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=qiH7BdpfgSjy3nYlisssmUH2L8a0hwgbs56mUCkJxQ0=; b=Ditrv5+qXx5XumMBdr6MSIzrgfEpOI/p+YEjr7WuOtHW1ZiBoPzMSZQdVdcJV73n0A PFgXmH+AHErZBqTXwgYn0HlMuy5AK0fBZfxbe7wZyXOGYTAlFmyaZyobBBYOGtJRsxuA Zdd1yUv+mAjbEnrjO5cK4n0GYQ36PQWEXMi3uWnZZei1dILUJerYaMdeuk5JAFjRadxC f7LIaP7wA4DTWUjbklJH4L0j6Dvza4NrDzZYwtQqT/7a8YhV7QNKQN/e9Q5fc+2/7skr +p5fZ9g/tsedk2go2ZY1g6k9JBPbzv4pW+0n+O9gqG4ZNKNTYnvAnfvQy06qyBVRF08d pmlQ==
X-Gm-Message-State: APjAAAXwpObZBloQEFnHBmCwjXAps7uKHAfMqWqcfnn63hF8D4zKSno4 I4P8xu24H+hjZpfV+RciVuU=
X-Google-Smtp-Source: APXvYqwehxxx3GuXj7L2PbPDkmlzQXpSCU9wpQsdrSjS27LfseKjAFXuwp5kRGZldAOjdsxqM2SyNw==
X-Received: by 2002:a62:6001:: with SMTP id u1mr1196087pfb.158.1575327255583; Mon, 02 Dec 2019 14:54:15 -0800 (PST)
Received: from [192.168.1.246] (corelight.static.monkeybrains.net. [208.90.215.182]) by smtp.gmail.com with ESMTPSA id i11sm545131pfo.80.2019.12.02.14.54.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Dec 2019 14:54:14 -0800 (PST)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
From: Sarah B <sarah.banks@gmail.com>
In-Reply-To: <CAHw9_i+biry0-sH8Ez1NZKo3V5rBkPL2amTk74akXH8E27Bttg@mail.gmail.com>
Date: Mon, 2 Dec 2019 14:54:13 -0800
Cc: bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <3FFADB16-2C79-48B5-A1BE-5C8CBC2E72D3@gmail.com>
References: <CAHw9_i+biry0-sH8Ez1NZKo3V5rBkPL2amTk74akXH8E27Bttg@mail.gmail.com>
To: Warren Kumari <warren@kumari.net>
X-Mailer: Apple Mail (2.3445.104.11)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/f3CUHINR1STqUP0m_QsINSJThQ0>
Subject: Re: [bmwg] AD Review of draft-ietf-bmwg-evpntest
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 02 Dec 2019 22:54:20 -0000

Hi Warren,
	Thanks for your thorough review. The authors will review the document with your feedback and edit accordingly. While We (and I, specifically) have looked at this document a LOT, we missed some obvious things, and our authors are first time authors; I'll work with them to sort out the obvious catches (thanks for those).

Cheers
Sarah

> On Dec 2, 2019, at 2:42 PM, Warren Kumari <warren@kumari.net> wrote:
> 
> Hi all,
> 
> Unfortunately, I do not think that this document is ready to be
> progressed -- there are a large number of areas where the document is
> *very* unclear, and / or incomplete. I usually break my reviews up
> into "substantive" and "Nits / Editorial"; but in this case I didn't,
> largely because many of the editorial-type issues make the text
> unclear; there are also a sufficient quantity that I do not feel
> comfortable asking all of the IETF to review, nor would I feel
> comfortable asking the directorates / IESG to review.
> 
> I've annotated my comments with "WK: ", but many of the issues occur
> multiple times -- for example, there are many instances of "SA", which
> from context is "Source Address" but this could also be "SA" in the
> "Single-Active" sense.
> 
> W
> -------------------
> 
> 
> Abstract
> 
>   This document defines methodologies for benchmarking EVPN and PBB-
>   EVPN performance.  EVPN is defined in RFC 7432, and is being deployed
>   in Service Provider networks.  Specifically this document defines the
> 
> WK (nit) : Specifically, this (missing comma)
> 
>   methodologies for benchmarking EVPN/PBB-EVPN convergence, data plane
>   performance, and control plane performance.
> 
> 
> 
> 1.  Introduction
> 
>   EVPN is defined in RFC 7432, and describes BGP MPLS- based Ethernet
>   VPNs (EVPN).  PBB-EVPN is defined in RFC 7623, discusses how Ethernet
>   Provider backbone Bridging can be combined with EVPNs to provide a
>   new/combined solution.  This draft defines methodologies that can be
>   used to benchmark both RFC 7432 and RFC 7623 solutions.  Further,
>   this draft provides methodologies for benchmarking the performance of
>   EVPN data and control planes, MAC learning, MAC flushing, MAC ageing,
> 
> WK: s/ageing/aging  (RFC index has 258 instances of 'aging' and only 4 'ageing')
> 
>   convergence, high availability, and scale.
> 
> 1.1.  Requirements Language
> 
>   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
>   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
>   document are to be interpreted as described in RFC 2119 [RFC2119].
> 
> WK: Replace RFC2119 with RFC8174 language.
> 
> 
> 
> WK: Please fix formatting, and sort alphabetically. Also, this isn't
> "Terminology", it is mainly Abbreviations.
> 
> 1.2.  Terminologies
> 
>   MHPE Multi homed Provide Edge router.
> 
>   RR Route Reflector.
> 
>   P Provider Router.
> 
>   CE Customer Router/Devices/Switch.
> 
>   MHPE2 Multi homed Provider Edge router 2.
> 
>   MHPE1 Multi homed Provider Edge router 1.
> 
>   SHPE3 Single homed Provider Edge Router 3.
> 
>   AA EVPN Terminologies AA All-Active.
> 
>   SA EVPN Terminologies SA Single-Active.
> 
>   RT Router Tester.
> 
>   Sub Interface Each physical Interfaces is subdivided in to Logical
>   units.
> WK: s/in to/into/
> 
>   EVI EVPN Instances which will be running on sub interface or physical
>   port of the provider Edge routers.
> 
>   DF Designated Forwarder.
> 
>   ESI Ethernet Segment Identifier.
> 
> 2.  Test Topology
> 
>   EVPN/PBB-EVPN Services running on SHPE3, MHPE1 and MHPE2 in Single
>   Active Mode:
> 
> 
>         | [Traffic Generator ] Router Tester traffic sender/receiver
> of layer 2 traffic with multiple vlan.
> 
> WK: VLANs. Also, this doesn't really parse - is 'Router Tester'
> supposed to be in the brackets?
> 
> +----------+
> |          |
> |  SHPE3   |
> |          |
> +----------+
>    |
>    |Core link
> +----------+
> |          |
> |  RR      |
> |          | Route Reflector/Core router
> 
> WK: Core router or Provider router? (Core router isn't defined in the
> terminology section)
> 
> +----------+-------------|
>   |                     |
>   |     Core links      |
> +----------+       +-----------+
> |          |       |    MHPE2  |
> |   DUT    |       |           |
> |  MHPE1   |       |           |
> +----------+       +-----------+
>     |    PE-CE link    |
> +----------+------------
> |          |
> |  CE      |
> |  layer2  |
> |bridge    |
> +----------+------------ [Traffic Generator](Router Tester
> sender/reciever of layer 2 traffic with multiple vlan)
> 
> WK: As above. Aslo, receiver is a typo
> 
> 
> WK: The formatting of this table is very poor, esp the header.
> 
> 
> 
> +-----------------+---------------------+---------------------+---------------------+----------------------+-----------------------+
> |                 |                     |                     |
>             |                      |                       |
> |                 |                     |                     |
>             |                      |                       |
> |                 |                     |                     |
>             |                      |                       |
> |                 |                     |                     |
>             |                      |                       |
> | Mode            |                     |                     |
>             |Receiver              |                       |
> |                 |  Test               |Traffic Direction    |Sender
>             |                      |                       |
> |                 |                     |                     |
>             |                      |                       |
> |                 |                     |                     |
>             |                      |                       |
> |                 |                     |                     |
>             |                      |                       |
> +----------------------------------------------------------------------------------------------------------------------------------+
> |                 |                     |                     |
>             |                      |                       |
> |                 |                     |                     |
>             |      SHPE3           |                       |
> |Single Active    |  Local Mac          |                     |CE
>             |                      |Layer 2 traffic        |
> |                 | Learning            | Uni                 |
>             |                      |                       |
> |                 |                     |                     |
>             |                      | multiple MAC          |
> |                 |                     |                     |
>             |                      |                       |
> +-----------------------------------------------------------------------------------------------------------------------------------+
> |                 |                     |                     |
>             |                      |                       |
> |Single Active    | Remote MAC          |                     |
>             |         CE           |Layer 2 traffic        |
> |                 | Learning            | uni                 | SHPE3
>             |                      |                       |
> |                 |                     |                     |
>             |                      |multiple MAC           |
> |                 |                     |                     |
>             |                      |                      ++
> +----------------------------------------------------------------------------------------------------------------------------------+
> |                 |                     |                     |
>             |                      |                       |
> |Single Active    | Scale Convergence   |   Bi                |
>             |  CE/SHPE3            |                       |
> |                 |                     |                     |
> CE/SHPE3          |                      |Layer 2 traffic        |
> |                 | Local& Remote       |                     |
>             |                      |multiple mac& vlans    |
> |                 | Learning            |                     |
>             |                      |                       |
> +-----------------+---------------------+---------------------+--------------------------------------------+-----------------------+
> 
>             |
> --- SNIP ---
> 
>   Test Setup Configurations:
> 
>   There are five routers in the Test setup.  SHPE3, RR/P, MHPE1 and
>   MHPE2 emulating a service provider network.  CE is a customer device
>   connected to MHPE1 and MHPE2, it is configured with bridge domains in
>   multiple vlans.  The router tester is connected to CE and SHPE3.The
>   MHPE1 acts as DUT.The RT will be used as sender and receiver of
>   traffic.The measurement will be taken in DUT.
> 
> 
> WK: What exactly is a "Router Tester"? This term doesn't seem to be defined
> anywhere -- is it just a traffic generator?
> Please also fix the formatting above (missing spaces after periods) -
> this is throughout the document.
> 
>   All routers except CE is configured with OSPF/IS-IS,LDP,MPLS,BGP with
>   EVPN address family.
> 
> WK: All routers except CE are configured with. "Configured with"
> doesn't really provide
>  sufficient detail to allow repeatable testing.
> 
> 
>   All routers except CE must have IBGP configured with RR acting as
>   route reflector.
> 
> WK: iBGP. Also, expand acronyms.
> 
>   MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN/PBB-EVPN instances
>   depends up on the cases.
> 
> WK: s/MHPE1,MHPE2,SHPE3/MHPE1, MHPE2, SHPE3/ (and elsewhere)
> 
> 
>   MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on IFD.
> 
> WK: What is an IFD?
> 
> 
>   MHPE1 and MHEPE2 are running Single Active mode of EVPN.
> 
>   CE is acting as bridge configured with vlans that is configured on
>   MHPE1,MHPE2,SHPE3.
> 
>   Depends up on the test traffic will be flowing uni directional or bi
>   directional depends on the test performed.
> 
> WK: This is unparsable (and even if I could parse it I'm not sure what
> it is related to)
> 
> 
>   The above configuration will be serving as the base configuration for
>   all test cases.
> 
> 3.  Test Cases for EVPN Benchmarking
> 
> 3.1.  Local MAC Learning
> 
>   Objective:
> 
>   To Record the time taken to learn the MAC address locally in DUT.
> 
>   Topology : Topology 1
> 
>   Procedure:
> 
>   The data plane MAC learning can be measured using the parameters
>   defined in RFC 2889 section 5.8.  Send "X" unicast frames from CE to
>   MHPE1(DUT) working in SA mode with "X" different source and
>   destination address from RT.  The DUT must learn these "X" macs in
>   data plane.
> 
> WK: MACs (and elsewhere)
> 
> 
>   Measurement :
> 
>   Measure the time taken to learn "X" MACs in DUT evpn mac table.  The
>   data plane measurement is taken by considering DUT as black box the
>   range of X MAC is known from RT and the same must be learned in DUT,
>   the time taken to learn "X" macs is measured.
> 
> WK: This is also largely unparsable.
> 
> 
> 
>   Repeat these test and plot the data.  The test is repeated for "N"
>   times and the values are collected.  The mac learning time is
>   calculated by averaging the values obtained from "N" samples.
> 
>   Mac learning in sec = (T1+T2+..Tn/N)
> 
> WK: Presumably (T1+T2+..Tn)/N ?
> 
> 
> 3.2.  Remote MAC Learning
> 
>   Objective:
> 
>   To Record the time taken to learn the remote macs.
> 
>   Topology : Topology 1
> 
>   Procedure:
> 
> 
>   Send X frames with X different SA and DA to SHPE3 from RT.
> 
> WK: Please define / expand SA / DA. Also, SA conflicts with "SA EVPN
> Terminologies SA Single-Active."
> 
>   SHPE3
>   will advertise these locally learned macs to MHPE1 and MHPE2 via
>   control plane.Measure the time taken to learn these X MACs from
>   remote peer in DUT EVPN MAC address table.The DUT and MHPE2 are
>   running SA mode.
> 
>   Measurement :
> 
>   Measure the time taken by the DUT to learn the "X" MACs in the data
>   plane.Repeat these test and plot the data.The test is repeated for
> 
> WK: Repeat these tests...
> 
>   "N" times and the values are collected.The mac learning time is
>   calculated by averaging the values obtained from "N" samples.
> 
>   Mac learning in sec = (T1+T2+..Tn/N)
> 
> WK: As above.  Also, what *exactly* do I report? This isn't (AFAICT)
> "Mac learning in sec", it is (presumably) MAC Learning Rate, and I'd
> *assume* that I report a line graph showing MACs learnt vs time? This
> (and the other tests) need to be *much* more explicit about what
> exactly is being reported.
> 
> --- SNIP ----
> 
> 3.9.  ARP/ND Scale
> 
>   These tests are conducted to Record the scaling parameter of ARP/ND
>   of the DUT.
> 
>   Objective:
> 
>   To Record the ARP/ND scale of the DUT.
> 
>   Topology : Topology 1
> 
>   Procedure:
> 
>   Send X arp/icmpv6 request from RT to DUT with different sender ip/
>   ipv6 address to the same target gateway ip address.  Measure whether
>   X MAC+IPv4 address/MAC+IPv6 address of the hosts are learned in DUT.
> 
> WK: I don't think I can run this test -- how fast do I send these? What
>  *exactly* is the packet I'm sending (icmpv6 is not a packet, it's a protocol)
>  Where do I send it? (to DUT? to the same target gateway IP? What
> does that even mean?)
>  The X MAC+IPv4 bit doesn't make sense.
> 
> 
>   Measurement :
> 
>   The DUT must learn X MAC+IPV4/MAC+IPv6 and it must advertise the X
>   MAC+IPV4/MAC+IPV6 to the remote router.
> 
> 3.10.  Scaling of Services
> 
>   Objective:
> 
>   To measure the scale limit of DUT for EVPN.This is to measure the
>   performance of DUT in scaling to "X" EVPN instances.
> 
>   Topology : Topology 1
> 
>   Procedure:
>   The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.Clear BGP neighbors of
>   the DUT.  Once adjacency is established in the DUT.
> 
> WK: Fragment?
> 
>    Measure the
>   routes received from MHPE2 and SHPE3 for "N" EVI in the DUT.
> 
>   Measurement :
> 
>   There should not be any loss of route types 1,2,3 and 4 in DUT.  DUT
>   must relearn all type 1,2,3 and 4 from remote routers.  The DUT must
>   be subjected to various values of N to find the optimal scale limit
> 
> WK: Do the first 2 sentences mean the same thing? Presumable I *do*
> lose all the routes, but then I should recover them? Why is this being
> compared to what I had before clearing, and not just the maximum /
> number sent from the tester? What ratio should there be for types 1,
> 2, 3, 4? Is it important?
> 
> 
> 3.11.  Scale Convergence
> 
>   Objective:
> 
>   To measure the convergence time of DUT when the DUT is scaled with
>   EVPN instance along with traffic.
> 
>   Topology : Topology 1
> 
>   Procedure:
> 
> 
>   Scale N EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
>   using traffic generator with X different SA and DA for N EVI's.  Send
>   F frames from traffic generator to SHPE3 with X different SA and DA.
>   There will be 2X number of MAC address will be learned in DUT EVPN
>   MAC table.
> 
> WK: Doesn't parse.
> 
>   There is a bi directional traffic flow with F pps in each
>   direction.  Then clear the BGP neighbors in the DUT.  Once the
>   adjacency is restored in DUT.  Measure the time taken to learn 2X MAC
>   address in DUT MAC table.
> 
> WK: Also doesn't parse.
> 
> 
>   Measurement :
> 
>   The DUT must learn 2X MAC address.  Measure the time taken to learn
>   2X MAC in DUT.  Repeat these test and plot the data.The test is
>   repeated for "N" times and the values are collected.The convergence
>   time is calculated by averaging the values obtained by "N" samples.
> 
>   Convergence time in sec = (T1+T2+..Tn/N)
> 
> 3.12.  SOAK Test.
> 
>   Objective:
> 
>   This test is carried out to measure the stability of the DUT in a
>   scaled environment with traffic over a period of time "T'".  In each
>   interval "t1" the DUT CPU usage, memory usage are measured.  The DUT
>   is checked for any crashes during this time period.
> 
>   Topology : Topology 1
> 
>   Procedure:
> 
> 
>   Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
>   using traffic generator with different X SA and DA for N EVI's.  Send
>   F frames from traffic generator to SHPE3 with X different SA and DA.
>   There will be 2X number of MAC address will be learned in DUT EVPN
>   MAC table.  There is a bi directional traffic flow with F pps in each
>   direction.  The DUT must run with traffic for 24 hours, every hour
>   check for memory leak, CPU usage and crash.
> 
>   Measurement :
> 
>   Take the hourly reading of CPU, process memory.  There should not be
>   any leak, crashes, CPU spikes.
> 
> WK: What all am I supposed to be reporting for this test? Can N be 1
> and F be 1 and X be 1? I'm happy to do that and report no issues for
> 24 hours....
> 
> 
> 4.  Test Cases for PBB-EVPN Benchmarking
> 
> WK: ----- Are these the same as for "Test Cases for EVPN
> Benchmarking"? If so, why are they repeated?
> 
> 4.8.  High Availability
> 
>   Objective:
> 
>   To record traffic loss during routing engine failover.
> 
>   Topology : Topology 1
> 
>   Procedure:
> 
> 
>   Send X frames to DUT with X different SA and DA from CE using the
>   traffic generator.  Send X frames from traffic generator to SHPE3
>   with X different SA and DA so that 2X MAC address will be Learned in
>   DUT.  There is a bi directional traffic flow with X pps in each
>   direction.  Then do a routing engine fail-over.
> 
>   Measurement :
> 
>   There should be 0 traffic loss which is the ideal case, No change in
>   the DF role.  DUT should not withdraw any routes.Repeat the test "N"
>   times and plot the data.The packet loss is calculated by averaging
>   the values obtained from "N" samples.
> 
>   Packet loss in sec = (T1+T2+..Tn/N)
> 
> WK: What exactly am I reporting? "There should be 0 traffic loss which
> is the ideal case" - am I reporting
> the number of seconds with dropped packets? What if it *does* withdraw
> routes? What do I report if I don't have multiple REs? Presumably if X
> == 2 I can do this easily, but if X == 1e8 it's harder - but all I'm
> reporting is "Packet loss".
> 
> 
> 
> --- SNIP ---
> 
> 4.11.  Soak Test
> 
>   Objective:
> 
>   To measure the stability of the DUT in a scaled environment with
>   traffic.
> 
>   Topology : Topology 1
> 
>   Procedure:
> 
>   Scale N PBB-EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE
>   using traffic generator with X different SA and DA for N EVI's.  Send
>   F frames from traffic generator to SHPE3 with X different SA and DA.
>   There will be 2X number of MAC address will be learned in DUT PBB-
>   EVPN MAC table.  There is a bi directional traffic flow with F pps in
>   Each direction.  The DUT must run with traffic for 24 hours, every
>   hour check the memory leak, crashes.
> 
>   Measurement :
> 
>   Take the hourly reading of CPU process, memory usages.  There should
>   not be any memory leak, crashes,CPU spikes.
> 
> WK: "any memory leak, crashes,CPU spikes" is not at all clear - e.g:
> How are you defining a 'spike'? If the CPU jumps from 2% to 10% for 3
> seconds, is that a spike? Many routers run a BGP reaper every N
> seconds / minutes, which looks like a spike - is this an issue?
> 
> 7.  Security Considerations
> 
>   There is no additional consideration from RFC 6192.
> 
> WK: I thought that all BMWG documents included standard boilerplate
> for Security Considerations?
> 
> ----------------
> 
> -- 
> I don't think the execution is relevant when it was obviously a bad
> idea in the first place.
> This is like putting rabid weasels in your pants, and later expressing
> regret at having chosen those particular rabid weasels and that pair
> of pants.
>   ---maf
> 
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg