Re: [bmwg] resending: Re: [iotdir] telechat Review for draft-ietf-bmwg-ngfw-performance-13

t petch <ietfa@btconnect.com> Sun, 30 January 2022 21:09 UTC

Return-Path: <ietfa@btconnect.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D5C823A15A4; Sun, 30 Jan 2022 13:09:51 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.614
X-Spam-Level:
X-Spam-Status: No, score=-2.614 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, NICE_REPLY_A=-0.714, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=btconnect.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yC3pVJWR5dMs; Sun, 30 Jan 2022 13:09:50 -0800 (PST)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40101.outbound.protection.outlook.com [40.107.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0D61C3A15A1; Sun, 30 Jan 2022 13:09:48 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=R6onnIAbplRw4CMdrXJMso/0B/QKq4yUh7VVN5sGYSuLy4q6OFR6JC3o1qnkuJUSlwD4/0mrNiug6g3+l9OO0t27+BO0TqM1sgS78QzE4XHDkNvwx3RgKQmqUG0O4SLvjkMYhRV4PsuQQI0ciPPvxL4gJ0MTyzd347Eci2vtrbYvTz6jmlMgKAKCk2ECbO2TLYUfnNJAIRc27wNVyZ2dyfpEn1R/Wk+qqfAYf/zHzbw0naDBHun653AFjufIeLBQJaeTY8zYSVqy5oQ4SPFwLBZkRY3lwCVKbAEEw5az0h95r3CqH+dnqLMUkYUy/GVpit+CUmNIGbxGA1WtdEobtg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sxfkge6sCoXX2FL2i+LXamDNMquayymFe05i1DkcjsI=; b=WrLsiHh5Es4yaKPzOv5Nvy6RRoKbfZfwESfUBHduhMjFaGzQrXmLuUl7Ev3kk9K+aBrr17c2NWnFNyOb7kFOlYNNRKKHK03ZxSEorB2zOY3X1aUEvO0W5cCPxOs4T3vS11bqK2P1Mfp+2AflzTluP+rYduq1Gos+qEENgX6UfnAKTsQFmo5FT+c9ZTFFRgtYrDHAlMz3/3E5jPgZgO1NVlDNbwJEyOEFdX/ohzN3pIUCfdAL2YM0jkKByIfIyTxNx7Nrns6kKg79rY7XHk/N5B0DVcLJyQxyeAP1yIXqmbitVMg1NsBFZ9px/FA/9RA4pE3ao+YSpWaXHO1ajyJubA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=btconnect.onmicrosoft.com; s=selector2-btconnect-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sxfkge6sCoXX2FL2i+LXamDNMquayymFe05i1DkcjsI=; b=Cx5yqGuMzVbq7JezfZZYjB4EmJnOs5balFUm9Lz8MpCCPcKSb2thhEYqp0F4gwMGugi1pf7mjlLfINyFLg3Z0B7FDEQn6oV9HrMdG/gNwtt31dFg+taheKgGemWfQBSMBkanFpoT90BjaaU7kjY2eF9y1PZWlXCB1Ipz46BCs0w=
Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=btconnect.com;
Received: from DB7PR07MB5546.eurprd07.prod.outlook.com (2603:10a6:10:73::23) by VI1PR07MB5054.eurprd07.prod.outlook.com (2603:10a6:803:90::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.10; Sun, 30 Jan 2022 21:09:45 +0000
Received: from DB7PR07MB5546.eurprd07.prod.outlook.com ([fe80::299b:c48:2d33:bf4]) by DB7PR07MB5546.eurprd07.prod.outlook.com ([fe80::299b:c48:2d33:bf4%5]) with mapi id 15.20.4951.010; Sun, 30 Jan 2022 21:09:45 +0000
To: Toerless Eckert <tte@cs.fau.de>, iot-directorate@ietf.org, evyncke@cisco.com
References: <YfaC59S87eoegvdx@faui48e.informatik.uni-erlangen.de>
Cc: draft-ietf-bmwg-ngfw-performance.all@ietf.org, mariainesrobles@googlemail.com, bmwg@ietf.org
From: t petch <ietfa@btconnect.com>
Message-ID: <61F6FE95.5050600@btconnect.com>
Date: Sun, 30 Jan 2022 21:09:41 +0000
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:38.0) Gecko/20100101 Thunderbird/38.5.0
In-Reply-To: <YfaC59S87eoegvdx@faui48e.informatik.uni-erlangen.de>
Content-Type: text/plain; charset="windows-1252"; format="flowed"
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: LO2P265CA0073.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:8::13) To DB7PR07MB5546.eurprd07.prod.outlook.com (2603:10a6:10:73::23)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 18791c17-e9c5-469c-50b5-08d9e434d754
X-MS-TrafficTypeDiagnostic: VI1PR07MB5054:EE_
X-Microsoft-Antispam-PRVS: <VI1PR07MB50544AC8CB384192972F2336A2249@VI1PR07MB5054.eurprd07.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: PjCKTe90nNJdwEHGqr9g0ZpTqTHDiEdU1WrUVvMgADTzDTYy/yM5aCTYa5oaeCU+lK82KairyvmkuPcqo+SLCMMz7Tze78Geey/ld4TYw9uOPjm2t0iy0t6EnPijQf2iKr1kiWrrYjgcdIWDX73Gp1D3GEPkd9KS5LGKwMBB54u9uSA5/GBAKhg+gOfynmuFf48uIgGLKXsx1CDWtLUVou5TPf6cg28lKdzhpiBBjQgCbR2CMxGrFKnwfuno/VrkbfRhiQ9OIm5WvgGpplGl43Q5U6IS+8oBCY2yvx/ZNXXlLmspFik/vEG3AQnnA4WZOj4OQj4UJN+J5djDAdg5QW+7sGaem4mRblyuiTu7ijXJxZr5BYWdBhCE0i0p4KJb7/wwRyhjolSlMIoASclqXszuzs/iCHbnZSuxoaWpwASMzExFF9TxwD8FXbK3Z7gYAcJOFpXtYN8fP3Co9dhHDR23hSGOaESua+OAR40cv3nbN5ji+N09hKpifI1K1chdg+PcoQeK8JFadQd/Qckw7SfaxRYUAO9t0kVyUNMCp5A1YVb2b9m+uZEipoegsaDOcdO7QipaXw1p0f/4ZukwqnVQGw0iR/z4vtA7gN2EbviOKaeF+qfZCShDiTLOBYfPrmZwcj6eH4S6pDZYJG5sB9vH0bLR7v0/JCLzVUqk6fvVCInC5OBwJuvy6Tr1PehNSaxAspGTV6wabYBSBdIglI/wEPfGopwofG9ow/V+rIvA96Wo1km01jSQMlwxRc19isdnbAfCfrZvPCWZwKAOt3w3MlodOmLTGCFGCmbZWxy09Jf8jCdk97rAPaPpFzdY
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB7PR07MB5546.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(8936002)(8676002)(4326008)(6512007)(66946007)(316002)(66556008)(66476007)(33656002)(2616005)(66574015)(6666004)(87266011)(6506007)(53546011)(52116002)(508600001)(5660300002)(186003)(82960400001)(86362001)(36756003)(26005)(83380400001)(966005)(2906002)(6486002)(38350700002)(38100700002)(20210929001); DIR:OUT; SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: yra5PiqL+IKbofpAGSnPx6GnKXsRM1MBGyk2/qT+f9OC/NPrNbFljNilZsHIQKkVk1W/C1eFjJvSKamprmznLv9uWz/7GV3XaRxCBIISL8/0Y4+yf9rP6Dy08bpS4/TZv5aA2PX9G3lEH9DXVDEvnUN574wR6pBrDiT3klZClxO2kyO+cPrxOzBYvCuwT3ioI3ra730AkhD1lLUgducbRGbmM4b6/79Xi9GEd50RtbZIukwyquPft22fWEfe02KZFBvFbUlP6qRrg0045gTGcAHgBvTdzrJM0E+jMH/2Nl70K8M+b5s7V5/0v2UqcQsg3SZkx4h4FeYFO9cCWn8HpQXl7VaVJfcCmMdeIY22eUCr6Nq64JkxNo1lCoAGpaTXar0VsvnLl62lp7WyTWYgXk71xj+v2fQF178RrmvPcy62EzaBCvKLhBYcabR0hNoJOattaT68JKeIDjvc8X3m6IR3nq6mvmgsefyCDd2j+R6sf8/08qVkRS7XUxdQJDF/8RHe4ltpj7Wb7Iza5tWdPvq7OyXQBu915Y/YiLf+6IVyVKVPacICFK1IQXUvmVSWyw4q870DYaXQcI+aIvxYQiqjRQrtbioYSr9eFOF4lKM58nFPli2kbD0VbuPLaGZznWapnuZ12FwK8KWCEJlf478QPs2iHWbNP+01cvcupdRpWUAGcOasqeZ9D2nKbDIHVd/L+uwZMqf3cu8r+nig+JrNdrRQPlTxNib+CBHv3VX3a+2B075oaoRWzAk3PWIq8Y4vGIfkaa1uOxsg++XI3/ba5JHd6ROrHeylz2ECLibPNI+A8j+u2QIW0oE/WxZHypeZZwcvSRWRHwlSHB1HfC338Wu58QQSsk62/o57pP8YOB0ZEa0AL+Uap/6pmDITQM+dQLokk9+SFq2WwAisuwweycsPBqcV15kStcgw/eQ1ssuZ80iHEINn0MIn0VU7culrMeUCe5n6nLhq0Z93so46096CQMRds0GlrJkK2sNMEbvwQedxAhYV+8NS3HFEKbVyIjysTLBMks89bJvq1JuSnBwbmNBfG1wW3eaxiGPxaXNFZgZFPa8qZfFNK77yDbfa3KbPM2SStEOAUvvUMZ05+FWWHHrgJCH1RjGObm5Pf1Nh7f1Sj7XRHxX6GJXzhmk4cN1IfvWlxRKB626w9F9JbHHngqJTzGcgdX0SPSApgct0hSfXlmvtvWEk6WMHhWEP91417GXJSFN/EgcUtva29ipq1ZTqrVD/IHmZrKT++gQEa55iM8wigFvTn2u/CbPcquTwaWPMXrgLtyj1zATBljEbTMWW54BMSGGKoHgbkhbA9C/qzGD2wFRF1JbyvY6WmRimp3D6fxAGoD2nd4YQEWPZFfZBq9Pf92D4+lGJ/uXN+R7Z2IGGjEG9kPkUfuEH8R/0pqEdL0TE03WwbycW/reeW9XKkXTtWAR4KLeHCPskeIeDOg+AU70aNvyYeKcdTJlCoICnMumoAQHEpTQnZB2ocNvoVxVFOuRvKKut/hz11rOJRlBKyrf4bXwYZ+GjRQiMrDfM4bTaUTPEZnlv8ilrvFOmW8uxJC+PrIH2bUNuHtgmVixp2jfZqUzOCN06VcyjoNZE3EellAqcOKSePHsU8vzjviZGZf3V2gM=
X-OriginatorOrg: btconnect.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 18791c17-e9c5-469c-50b5-08d9e434d754
X-MS-Exchange-CrossTenant-AuthSource: DB7PR07MB5546.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2022 21:09:45.4663 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: cf8853ed-96e5-465b-9185-806bfe185e30
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: z/38qJd7QFdFF/Ax/SY5m97MM6npMwqO+5dBQxIVbeMC3PTELxs0gX4gPUzMhrYDT3R6e6+c1AZgahljfg3Tzg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR07MB5054
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/f9w2JfXAnI9QimFe9qajRCkgmGo>
Subject: Re: [bmwg] resending: Re: [iotdir] telechat Review for draft-ietf-bmwg-ngfw-performance-13
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 30 Jan 2022 21:09:52 -0000

On 30/01/2022 12:21, Toerless Eckert wrote:
> [resending, because bmwg@ietf.org is set up to not accept emails longer than 60kb,
>   a first for me (have not seen this in other IETF WGs. So this time around i will
>   try to add the review as an attachment.]

I have and think that it is an excellent idea, especially when it stops 
me getting .pdf of close to 1Mbyte and html which includes the MSO 
styleshift!

Tom Petch

> Reviewer: toerless eckert
> Review result: On the right track
>
> Summary:
> Thanks a lot for this work. Its an immensibley complex and important problem to
> tackle. I have in my time only measured router traffic performance and that already was an
> infinite matrix. This looks to me like some order of infinite bigger a problem.
>
> Meaning: however questioning my reviews feedback may be wrt. nitpicking about the document,
> i think that the document in its existing form is already a great advancement to measure
> performance for these security devices, and in doubt should be progressed rather faster than
> slower especially because in my (limited) market understanding, many security device vendors will
> only provide actual feedback once it is an RFC (community i think overall more conservative in
> adopting IETF work, most not proactively engaging during draft stage).
>
> But of course: feel free to improve the document with any of the feedback/suggestions
> in my review that you feel are useful.
>
> Maybe high level, i would suggest most importantly to add more explanations, especially in
> an appropriate section about those aspects known NOT to be considered (but potentially
> important) so that the applicability of the tests that are described are better put into
> perspective by adopters of the draft to their real-world situations.
>
> Favorite pet topic: Add req. to measure the DUT through a power meter and report consumption
> so we can start making sure products with lower power consumptions will see sales benefits
> when reporting numbers from this document (see details inline).
>
> Formal:
> I choose to keep the whole document inline to make it easier for readers to vet
> my comments without having to open in parallel a copy of the whole document.
>
> Rest see attachment
>
> Thanks!
>      Toerless
>
>
>
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg
>