Re: Quality of Directorate reviews

Phillip Hallam-Baker <phill@hallambaker.com> Wed, 13 November 2019 15:08 UTC

Return-Path: <hallam@gmail.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2F06F120926 for <ietf@ietfa.amsl.com>; Wed, 13 Nov 2019 07:08:03 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.646
X-Spam-Level:
X-Spam-Status: No, score=-1.646 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kAnEqV15Et3M for <ietf@ietfa.amsl.com>; Wed, 13 Nov 2019 07:08:02 -0800 (PST)
Received: from mail-ot1-f52.google.com (mail-ot1-f52.google.com [209.85.210.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E701F120946 for <ietf@ietf.org>; Wed, 13 Nov 2019 07:08:01 -0800 (PST)
Received: by mail-ot1-f52.google.com with SMTP id b16so1867478otk.9 for <ietf@ietf.org>; Wed, 13 Nov 2019 07:08:01 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hahta1+9zvRDsBQ1jwq1VTFLHld50/Z5aYU/o86loHA=; b=rYTPUgH/Z2fx8vbKrioJEnQJc17luOxcYeov92/HMVwjyF88JJUhZQ6tzljl6tSCBp HiU66UEBEsJkMe1hzphCXAU4q2a7GHVNEkXEf7k14M/F/DTHHKMATPVvo5GfT9RSqG9v 680zx7M4yYlxvTht4ISWMRc40HDlju3sjydtGJ44Dgl11iLNszCau/lbDK6I5dNfJ21x 9ckh6Sh4ccHiHTSy4CtmxTcKttvLah8XXEfYvJJHucC/7Anmv0a77pwqf+k8jMWQX9cd B4plVUyrRdLVST3jITowTAoVZP7BXszivxZbA2kwm83L1uwcEJdOA2EC8jF14eq3FnCz wdcQ==
X-Gm-Message-State: APjAAAVuv3o6iVntGR/I4Mor4/3PjmeDOCBbXMah+otD6zv2ttSN3JE8 zn9a9fQONrVmBJtPDJdmMxkHTILWTOKZvn566TE=
X-Google-Smtp-Source: APXvYqxMd5R1VoW5XIotc4UHwdiS3jhoexwxLEQFjhFI6DgiEWIEziuMc7fGEJQLJe/6OEgvmH3pGD/XGwBrVTPk3cA=
X-Received: by 2002:a9d:6f15:: with SMTP id n21mr3758792otq.231.1573657681134; Wed, 13 Nov 2019 07:08:01 -0800 (PST)
MIME-Version: 1.0
References: <157279399807.13506.13363770981495597049.idtracker@ietfa.amsl.com> <0EF64763-BA25-468A-B387-91445A61D318@gmail.com> <CAJU8_nUovmFmgNiYx0ez_1f+GPdU9xGViDYWfowEEomrn0pyDw@mail.gmail.com> <alpine.LRH.2.21.1911040841160.27600@bofh.nohats.ca> <CE06CC6D-E37F-4C90-B782-D14B1D715D4B@cable.comcast.com> <38E47448-63B4-4A5D-8A9D-3AB890EBDDDD@akamai.com> <09886edb-4302-b309-9eaa-f016c4487128@gmail.com> <26819.1572990657@localhost> <2668fa45-7667-51a6-7cb6-4b704c7fba5a@isode.com> <2C97D18E-3DA0-4A2D-8179-6D86EB835783@gmail.com> <MN2PR11MB43669E4CEF13CDA51A764F9AB5790@MN2PR11MB4366.namprd11.prod.outlook.com> <20191.1573054128@localhost> <15BCDF05-FB13-45D2-A5DF-70618EBA1A5A@gmail.com> <9182.1573147520@localhost> <A3493C65-7F8A-407D-A9F4-FF36296C0920@gmail.com>
In-Reply-To: <A3493C65-7F8A-407D-A9F4-FF36296C0920@gmail.com>
From: Phillip Hallam-Baker <phill@hallambaker.com>
Date: Wed, 13 Nov 2019 10:07:47 -0500
Message-ID: <CAMm+LwiP4Ypuyh2xsd8qBjUfwuNzOYOfp3OrDnPmU-YwMH2pMw@mail.gmail.com>
Subject: Re: Quality of Directorate reviews
To: Stewart Bryant <stewart.bryant@gmail.com>
Cc: Michael Richardson <mcr+ietf@sandelman.ca>, Alexey Melnikov <alexey.melnikov@isode.com>, Bob Hinden <bob.hinden@gmail.com>, IETF <ietf@ietf.org>
Content-Type: multipart/alternative; boundary="00000000000015092b05973bbb20"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/j-lbbt_2-nChTDSMzezTvtiyUvU>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Nov 2019 15:08:09 -0000

Maybe we have the wrong model for security reviews and this is in part the
result of the RFC series being kitchen sink.

Some reviews are so trivial as to be irrelevant. Other cases you get a
document where the security considerations is a link to another document
that is nothing but security considerations.

Maybe what we need is a structure that assigns multiple reviewers for some
projects and rubber stamps others.

Also note that I am a designer and my skill set is quite different to those
of a hacker. I can show people a way to do a job so that there is a very
small chance of getting it wrong. But thats not the same as spotting their
mistake if they decide to do it their way.

Another concern I have is formal methods and before folk start squawking
about how they are necessary, my doctoral thesis is on formal methods. My
college Tutor was Tony Hoare, I know what they are capable of. What worries
me is that systems we can prove to be secure seem to be turning out to be
fragile in the real world.

If folk want an extreme example, look at BTC, the cryptography of the
Blockchain is unbreakable. Yet I spent this morning talking to a friend
dealing with the aftermath of a relative who has literally lost everything
to a corrupt exchange scheme.