Re: Structured request headers deployment issues

Mark Nottingham <> Mon, 22 June 2020 03:23 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 36BE93A0907 for <>; Sun, 21 Jun 2020 20:23:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.749
X-Spam-Status: No, score=-2.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, MAILING_LIST_MULTI=-1, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key) header.b=CQIcMkKI; dkim=pass (2048-bit key) header.b=CrocoERc
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id oPLpxkT3p-XC for <>; Sun, 21 Jun 2020 20:23:07 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 7F7D33A08FE for <>; Sun, 21 Jun 2020 20:23:06 -0700 (PDT)
Received: from lists by with local (Exim 4.92) (envelope-from <>) id 1jnD05-0008Bc-QK for; Mon, 22 Jun 2020 03:20:01 +0000
Resent-Date: Mon, 22 Jun 2020 03:20:01 +0000
Resent-Message-Id: <>
Received: from ([]) by with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <>) id 1jnD03-0008An-Cs for; Mon, 22 Jun 2020 03:19:59 +0000
Received: from ([]) by with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <>) id 1jnD01-0001BA-DQ for; Mon, 22 Jun 2020 03:19:59 +0000
Received: from compute4.internal (compute4.nyi.internal []) by mailout.nyi.internal (Postfix) with ESMTP id DAD1A5C1FE2; Sun, 21 Jun 2020 23:19:42 -0400 (EDT)
Received: from mailfrontend1 ([]) by compute4.internal (MEProxy); Sun, 21 Jun 2020 23:19:42 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm3; bh=9 JOjFW768J+fQMimM9ae6bIUJnb1VbSMB2MG6zEyRNg=; b=CQIcMkKIvauOZTuoL +6ZlNjEnVTTQLtwZ9fPxt02arHaaD/O+ltZTV6iQWWIfq/TVkxqyr0t/yKTgqpky 6h6p8Edobaz4pnNb/OMLBv9S2vxvz1erU5BoEBHq/LTGdkOiWzA0Ascbrp3rOqt/ X9XRtdOViyGhjdAIWRQ2aPfSD2xejH+8+qejT/fZeJ/rhBywM5geissDsI3eJ1J9 XuI3GYNMVcJ7QCxbBrpQrhFcuoAdOc7YMMEKQO7VvFRroZOxP8cG/bcH1XPRMaA0 vZpf+Lih6xSbumqSJcQoAI+KJdbVYJi0lat5FADSn+aJntlzCmrihbRiPGCUmzLR 1/LeQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=9JOjFW768J+fQMimM9ae6bIUJnb1VbSMB2MG6zEyR Ng=; b=CrocoERcoBZ//2stfJ8o1tNFU1Ia/ECtq6pQt7camZowXpp8EZXFDGU/Z dGZ0Nr6FDSFXla35w7nNuc3dHn6Va4dXG1lYVK/k4Y0vA1doqMFwqiH1orIis0tY uilth1RQ7mcCpslWTf6kOy9W0eiWT3PE0pA1fFrtdMGWZ765Lp3GNTBlnEZXabMj mJASEvtPPYu6NR0oBqSSYrM1boEf/23+nb43gYy3J5KbFTnsu8H+jjx76FsEzD1x RFmFKX7NVpzu8oeZYyFx0soU4LVbuAwEROv8bJ9QeptuCiK48w0M6m0guldckUyt cnksQPTj063jNZNcmsG2X0dtLVILw==
X-ME-Sender: <xms:TSPwXiGcEB4mMN31zjJozE0xFGSuiFGo5HHl4QRemQo1jElz4PWT9Q>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudekuddgjedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtvdenucfhrhhomhepofgrrhhk ucfpohhtthhinhhghhgrmhcuoehmnhhothesmhhnohhtrdhnvghtqeenucggtffrrghtth gvrhhnpeelffdvueevffffkeeggfffueegheelkeekteejlefhleekveekudeiieevvdet gfenucffohhmrghinhepghhithhhuhgsrdgtohhmpdhmnhhothdrnhgvthenucfkphepud duledrudejrdduheekrddvhedunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghm pehmrghilhhfrhhomhepmhhnohhtsehmnhhothdrnhgvth
X-ME-Proxy: <xmx:TSPwXjU8uo0ZoroG3dHjV55wjcy_BCIjUQuzvvzd13WyuSu5fMdigg> <xmx:TSPwXsKYnxjqaxf-VYo5sNUqOUq2U-ZUqDO40kcqda_Z9qo_W3LtKg> <xmx:TSPwXsHC7TJuzLNo7e3TcbqxceMeZPhRx2Nes4uG6UUG0-y-cs5gPQ> <xmx:TiPwXmxMHBXa6E24kfXO4fk7JCXbU5AkMCErzItQMa-aOv9wgIofOw>
Received: from ( []) by (Postfix) with ESMTPA id 43D033280059; Sun, 21 Jun 2020 23:19:40 -0400 (EDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.\))
From: Mark Nottingham <>
In-Reply-To: <>
Date: Mon, 22 Jun 2020 13:19:37 +1000
Cc: Yoav Weiss <>, " Group" <>, Tommy Pauly <>, Ilya Grigorik <>
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <>
To: Mike West <>
X-Mailer: Apple Mail (2.3608.
Received-SPF: pass client-ip=;;
X-W3C-Hub-Spam-Status: No, score=-9.8
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_DB=-1, W3C_IRA=-1, W3C_IRR=-3, W3C_WL=-1
X-W3C-Scan-Sig: 1jnD01-0001BA-DQ 53378eacf7b0feb2b16036c6267d7014
Subject: Re: Structured request headers deployment issues
Archived-At: <>
X-Mailing-List: <> archive/latest/37807
Precedence: list
List-Id: <>
List-Help: <>
List-Post: <>
List-Unsubscribe: <>

Hey Mike,

> On 19 Jun 2020, at 5:03 pm, Mike West <> wrote:
>> Personally, I think that outreach and waiting is the right approach; if browsers consistently send these headers, they'll adapt, and the numbers are still relatively small -- or at least small enough that it's not likely the numbers will be reduced if the syntax is changed (due to _other_ WAFs' opinions about what a "good" request is).
> The flip-side of this is that we break sites users rely upon. That's tough to do at scale. In this case in particular, I think I agree with the underlying assertion that we'd like this syntax to work, and it seems reasonable to me to roll it out to a small percentage of stable users. That said, if we end up breaking the internet for even a small percentage of users, it doesn't seem like a good idea to hold them hostage in the hopes of increasing pressure on WAF vendors.

I don't think we disagree. The problem is that there is always going to be a small percentage that break, unless we severely limit the evolution of the Web (i.e., don't use new headers at all, especially given the data Yoav is bringing!), so the relevant question is *how much*.

In other words, there's a theshold below which you're comfortable breaking a *few* sites, and those few sites presumably feel the pressure to change. Above that threshold, you're unwilling to cause too much damage, and the sites presumably feel comfortable in persisting their behaviour (acknowledging that there's some fuzziness on both sides).

> My expectation is that we'd roll this out to a small percentage of stable, see a spike in bug reports (especially from enterprise folks who are likely to have WAF deployments, unlikely to broadly deploy beta channels, and unlikely to choose to enable usage statistics or histograms), see a corresponding spike in error codes for top-level navigations, and roll it back.
> I think it might be reasonable to explore alternate spellings at the same time, perhaps with some A/B testing to evaluate how well each weaves its way through middleware.

What's missing here are the ability to engage the WAF community, and time. Unlike Web sites, there's a relatively smaller number of WAF vendors out there, so it's not unreasonable to engage with them to bring the numbers down below the threshold. That will take time, especially since WAFs don't yet have auto-update capabilities on par with browsers, but I think it's worth doing.

What I don't want to see is the Web become ossified on what a few WAFs happened to do in June 2020.

>> Related, we're also seeing more examples WAFs limiting how we can evolve the protocol (e.g., <>). There's been a bit of background chatter about writing something about this and creating better communication with that community; I'm not sure what that will look like yet, but if anyone has ideas or is interested, please say so.
> I am interested!

Great. I've heard from a few other folks too, so we'll try to figure out the appropriate venue for further discussion.


Mark Nottingham