Re: [bmwg] Murray Kucherawy's Discuss on draft-ietf-bmwg-ngfw-performance-13: (with DISCUSS and COMMENT)

John C Klensin <john-ietf@jck.com> Fri, 11 February 2022 16:13 UTC

Return-Path: <john-ietf@jck.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B93193A137A; Fri, 11 Feb 2022 08:13:28 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Level:
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f-OG6nBpYevX; Fri, 11 Feb 2022 08:13:24 -0800 (PST)
Received: from bsa2.jck.com (ns.jck.com [70.88.254.51]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EB0343A09E0; Fri, 11 Feb 2022 08:13:20 -0800 (PST)
Received: from [198.252.137.10] (helo=PSB) by bsa2.jck.com with esmtp (Exim 4.82 (FreeBSD)) (envelope-from <john-ietf@jck.com>) id 1nIYXs-000K6f-KE; Fri, 11 Feb 2022 11:13:16 -0500
Date: Fri, 11 Feb 2022 11:13:10 -0500
From: John C Klensin <john-ietf@jck.com>
To: Stewart Bryant <stewart.bryant@gmail.com>, Scott Bradner <sob@sobco.com>
cc: bmwg-chairs@ietf.org, IETF <ietf@ietf.org>, The IESG <iesg@ietf.org>, bmwg@ietf.org, draft-ietf-bmwg-ngfw-performance@ietf.org
Message-ID: <23A0A262ECC84B5B369983ED@PSB>
In-Reply-To: <5AAFE998-739E-48DA-B82D-2FBCD5989E42@gmail.com>
References: <164386586568.20734.14136987065393258244@ietfa.amsl.com> <fdb1df79-3a53-ea53-8532-ffb8d2545996@eantc.de> <CAL0qLwaDu9U1Kj_GtgNEY6JWpJncyC-t=NYA_uwOHs8zZmCHpQ@mail.gmail.com> <01D0C656-F323-4A7A-BF5B-764D646E2CD8@sobco.com> <5AAFE998-739E-48DA-B82D-2FBCD5989E42@gmail.com>
X-Mailer: Mulberry/4.0.8 (Win32)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
X-SA-Exim-Connect-IP: 198.252.137.10
X-SA-Exim-Mail-From: john-ietf@jck.com
X-SA-Exim-Scanned: No (on bsa2.jck.com); SAEximRunCond expanded to false
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/5QROTOUP18EZdX9JOw1vLQK4DKw>
Subject: Re: [bmwg] Murray Kucherawy's Discuss on draft-ietf-bmwg-ngfw-performance-13: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 11 Feb 2022 16:13:37 -0000


--On Friday, February 11, 2022 12:22 +0000 Stewart Bryant
<stewart.bryant@gmail.com> wrote:

> 
> 
>> On 11 Feb 2022, at 12:08, Scott Bradner <sob@sobco.com> wrote:
>> 
>> I do not think it is useful to use SHOULD without
>> specifically saying what the escape clause is -  i.e.
>> specifically say when its OK to not act as if it was a MUST
> 
> Hi Scott
> 
> The problem with that is it requires perfect foresight on the
> exceptions.
> 
> I have always taken SHOULD to mean that you MUST do this
> unless you have a good reason not to do this and understand
> the consequences for your system in its deployment environment.
> 
> In other words I think we need to trust the engineer
> implementing the system to apply good judgement when they
> ignore the MUST element of the SHOULD.

Stewart,

As Scott probably remembers, I've advocated several times  --
with the IESG, IAB, and with the RFC Editor -- for a rule that
is a little more flexible than the one Scott suggests above but
that treats "good judgment" differently.  That suggestion has
been something like the following:

If SHOULD is used, then it must be accompanied by at least one
of:
 (1) A general description of the character of the
	exceptions and/or in what areas exceptions are likely to
	arise.  Examples are fine but, except in plausible and
	rare cases, not enumerated lists. 
 (2) A statement about what should be done, or what the
	considerations are, if the "SHOULD" requirement is not
	met. 
 (3) A statement about why it is not a MUST.

The first option was not just because of your "perfect
foresight" comment above but because attempts to enumerate all
cases can easily lead into "what is not explicitly forbidden is
permitted" thinking which is never good for standards.  And,
especially if there is any ambiguity, all three need to be read
through the lens of the original intended interpretation of the
Robustness Principle.

I have no opinion about what should be done for the case of this
particular specification but, because this is not the first time
a problem like this has arisen _and_ because there have
occasionally been objections from within the IESG to documents
that discuss what to do if an implementation does not follow a
given specification, I would encourage the IESG and the
community to think about the issue more broadly.

   john