Re: [Tsv-art] [Last-Call] Tsvart last call review of draft-ietf-bmwg-evpntest-07

Sarah Banks <sbanks@encrypted.net> Tue, 25 May 2021 16:24 UTC

Return-Path: <sbanks@encrypted.net>
X-Original-To: tsv-art@ietfa.amsl.com
Delivered-To: tsv-art@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 608293A13EC; Tue, 25 May 2021 09:24:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.898
X-Spam-Level:
X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1yFiNw9QOWFL; Tue, 25 May 2021 09:24:42 -0700 (PDT)
Received: from xyz.hosed.xyz (xyz.hosed.xyz [71.114.67.91]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B3BAD3A13EF; Tue, 25 May 2021 09:24:41 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by xyz.hosed.xyz (Postfix) with ESMTP id E58F313C1D86; Tue, 25 May 2021 12:24:40 -0400 (EDT)
X-Virus-Scanned: Debian amavisd-new at xyz.hosed.xyz
Received: from xyz.hosed.xyz ([127.0.0.1]) by localhost (xyz.hosed.xyz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4D6xUkFLxxLL; Tue, 25 May 2021 12:24:40 -0400 (EDT)
Received: from [172.16.12.111] (c-73-71-250-98.hsd1.ca.comcast.net [73.71.250.98]) by xyz.hosed.xyz (Postfix) with ESMTPSA id 362F213C09A8; Tue, 25 May 2021 12:24:40 -0400 (EDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\))
From: Sarah Banks <sbanks@encrypted.net>
In-Reply-To: <162187790509.16740.4321707014094930863@ietfa.amsl.com>
Date: Tue, 25 May 2021 09:24:39 -0700
Cc: tsv-art@ietf.org, last-call@ietf.org, draft-ietf-bmwg-evpntest.all@ietf.org, bmwg@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <A7886642-D364-4864-AF04-1852C0FBEB6A@encrypted.net>
References: <162187790509.16740.4321707014094930863@ietfa.amsl.com>
To: Jana Iyengar <jri.ietf@gmail.com>
X-Mailer: Apple Mail (2.3654.60.0.2.21)
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsv-art/ZdJ8LN8YzmFVJDPuTxTCGKLIrDY>
Subject: Re: [Tsv-art] [Last-Call] Tsvart last call review of draft-ietf-bmwg-evpntest-07
X-BeenThere: tsv-art@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Review Team <tsv-art.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsv-art>, <mailto:tsv-art-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsv-art/>
List-Post: <mailto:tsv-art@ietf.org>
List-Help: <mailto:tsv-art-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsv-art>, <mailto:tsv-art-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 25 May 2021 16:24:43 -0000

Hi Jana,
   Thank you for your review. The authors are reviewing the feedback, and plan to update the document shortly. Your catch is a good one - I hadn't noticed this the first time through, and I agree with your feedback. It might be also worthwhile simply documenting the CPU usage; conversely, establishing a recorded baseline before the test starts, and then noting the CPU usage might help the tester draw conclusions around "spikes", but with either approach, the data is captured and documented. Sudhin, this is something for you to consider as you work through the document feedback.

Thank you,
Sarah

> On May 24, 2021, at 10:38 AM, Jana Iyengar via Datatracker <noreply@ietf.org> wrote:
> 
> Reviewer: Jana Iyengar
> Review result: Ready with Issues
> 
> This document has been reviewed as part of the transport area review team's
> ongoing effort to review key IETF documents. These comments were written
> primarily for the transport area directors, but are copied to the document's
> authors and WG to allow them to address any issues raised and also to the IETF
> discussion list for information.
> 
> When done at the time of IETF Last Call, the authors should consider this
> review as part of the last-call comments they receive. Please always CC
> tsv-art@ietf.org if you reply to or forward this review.
> 
> The document seems fine overall. There are some minor grammar and consistency
> things, but I expect that the RFC-editors will handle those.
> 
> The one thing that stuck out to me is the following: It helps in documents such
> as this to be more precise about exactly what a measurement tool or tester
> should consider success or failure. One piece of text where this precision
> should be improved is in the Soak Test (both 3.12 and 4.11):
>  "The CPU spike is determined as the CPU usage which shoots at 40 to 50
>  percent of the average usage.
>    The average value vary from device to device.  Memory leak is determined
>   by increase usage of the memory for EVPN process.  The expectation is
>   under steady state the memory usage for EVPN process should not
>   increase."
> Perhaps something like the following for defining CPU spikes might be helpful:
> "A CPU spike is defined as a sudden increase and subsequent decrease in usage
> from average usage to about 150% of average usage." Similarly, memory leak is
> very weakly defined. Do you mean _any increase_ in memory usage, or is there a
> threshold that you want to propose? Do you mean consistent increase over time?
> Can you define a leak more precisely in the context of your test?
> 
> 
> 
> -- 
> last-call mailing list
> last-call@ietf.org
> https://www.ietf.org/mailman/listinfo/last-call