Re: [OAUTH-WG] client certs and TLS Terminating Reverse Proxies (was Re: I-D Action: draft-ietf-oauth-jwt-introspection-response-08.txt)

"Richard Backman, Annabelle" <> Thu, 31 October 2019 18:55 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 60EDC12000F for <>; Thu, 31 Oct 2019 11:55:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -14.499
X-Spam-Status: No, score=-14.499 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id I8E7BaVZGoMg for <>; Thu, 31 Oct 2019 11:55:36 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id D044312006A for <>; Thu, 31 Oct 2019 11:55:33 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;;; q=dns/txt; s=amazon201209; t=1572548134; x=1604084134; h=from:to:subject:date:message-id:references:in-reply-to: mime-version; bh=9KIBbfXDeLG44nEMJtAM79/UOJN4cB1sS9iJEa98bOo=; b=UvsfvBisvO4WQfAEH+fIvdg+8rcjWdYpPECiYQ7phDdB4SDBCktV12Ij E8ruFiXG/0uLMS7g2XiNy4VeOmCcSUqyaAAjoq60C4lOTr7FEURUqZPZp TMMuoIyI91yL7KhrxGmLDWdOjDjouuoFKSSVc3ttzTRZkf3ddleD8+RHk c=;
IronPort-SDR: GEeAu+XnJ90BUHTMvaCYXxqhmA7hWWC5gd0hU+oKNmSI3xAMja1bYoo2dgq8SnbpdtcvO4ZRBj XDmfH2gc9NEQ==
X-IronPort-AV: E=Sophos;i="5.68,252,1569283200"; d="scan'208,217";a="2421032"
Received: from (HELO ([]) by with ESMTP; 31 Oct 2019 18:55:29 +0000
Received: from ( []) by (Postfix) with ESMTPS id 06B8FA338D for <>; Thu, 31 Oct 2019 18:55:28 +0000 (UTC)
Received: from ( by ( with Microsoft SMTP Server (TLS) id 15.0.1367.3; Thu, 31 Oct 2019 18:55:28 +0000
Received: from ( by ( with Microsoft SMTP Server (TLS) id 15.0.1367.3; Thu, 31 Oct 2019 18:55:28 +0000
Received: from ([]) by ([]) with mapi id 15.00.1367.000; Thu, 31 Oct 2019 18:55:28 +0000
From: "Richard Backman, Annabelle" <>
To: "" <>
Thread-Topic: [OAUTH-WG] client certs and TLS Terminating Reverse Proxies (was Re: I-D Action: draft-ietf-oauth-jwt-introspection-response-08.txt)
Date: Thu, 31 Oct 2019 18:55:28 +0000
Message-ID: <>
References: <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>
In-Reply-To: <>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/10.1b.0.190715
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: []
Content-Type: multipart/alternative; boundary="_000_E70795A45BC74450A1B4FFC3D332F1DCamazoncom_"
MIME-Version: 1.0
Archived-At: <>
Subject: Re: [OAUTH-WG] client certs and TLS Terminating Reverse Proxies (was Re: I-D Action: draft-ietf-oauth-jwt-introspection-response-08.txt)
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: OAUTH WG <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 31 Oct 2019 18:55:39 -0000

Relying on a fixed, random header name for security, even as a “defense in depth” measure, is dangerous. In order for this mechanism to be effective, the header name must be random (in the cryptographic sense) and must be kept secret. It needs to be withheld from request logs or error logs, either on the reverse proxy or on the service. It cannot be committed to code repositories. Including it as a signed header in request signing algorithms that require an explicit list of signed headers (such as AWS Signature Version 4<>, draft-cavage-http-signatures<>, draft-ietf-oauth-signed-http-request<>) turns that signature metadata into a secret, meaning that it cannot be logged, etc. If signatures are sent to a central authority for processing, that authority must also know not to log the list of signed headers. I could go on, but I hope this is enough to express that there are SO MANY ways that header names can and will be revealed, and they aren’t always obvious.

What we are talking about is a message authenticity problem. The best practices for providing message authenticity involve applied crypto, e.g., an HMAC or digital signature over the header contents. If an implementation does that, then the random header name is unnecessary. This approach is immune to the sort of misconfiguration scenarios that have been discussed in this thread, as they would result in the reverse proxy failing to provide a properly signed header to the service.

Annabelle Richard Backman
AWS Identity

From: OAuth <> on behalf of Vladimir Dzhuvinov <>
Organization: Connect2id Ltd.
Date: Thursday, October 31, 2019 at 4:27 AM
To: "" <>
Subject: Re: [OAUTH-WG] client certs and TLS Terminating Reverse Proxies (was Re: I-D Action: draft-ietf-oauth-jwt-introspection-response-08.txt)

This is a good story.

How about requiring reverse proxies to automatically scrub all inbound HTTP headers that start with "Sec-"?

If a sec header has the format "Sec-[NAME]-[random-chars]" we get:

* The "Sec-" prefix will  cause new compliant reserve proxies to automatically scrub the inbound HTTP header.

* The NAME part still makes the header easily identifiable (I think Rich Salz had this concern).

* The random chars part, potentially optional, will provide a line of defence against config errors in old legacy reverse proxies.

All concerns should then get covered.

On 31/10/2019 12:48, Hans Zandbelt wrote:
the way I see this is that stripping and setting the headers must be part of the implementation of the protocol itself, it should not be something left to a non-atomic piece of configuration by an administrator; I implemented it like this in the Apache implementation of Brian's TTRP spec [1] and have been doing this for the OIDC and OAuth Apache modules that set headers for backends to consume [2]; and yes, I have made mistakes [3], but IMHO it should not be a problem to use a well known header and make the implementer (not the admin) get it right by pointing this out in the spec, as is done for many other pitfalls



On Thu, Oct 31, 2019 at 11:37 AM Vladimir Dzhuvinov <<>> wrote:
All in all, I am in favor of this being defined in one standard way, in addition to secure communication between proxies and backends being standardized — but this latter bit really seems like a separate problem.

 — Justin

-1 for devising a well known header

+1 for a simple way to authenticate a reverse proxy with a web server

With a well known name, in a attack this will get probed first. The well known header name doesn't guarantee a correct config. And if we have a new standard for sec headers that must be stripped automatically from inbound HTTP requests, we cannot expect that will get implemented in all reverse proxy software overnight.

Because a correct config is not practical as the only line of defense, when we implemented mTLS we decided to add a length (>= 32 chars) and randomness check to the header config. I saw some concerns that this may cause deployment issues. Nobody has complained about that so far.

Regarding mistyping, this can be an issue, but has little to no effect if a long random header gets misstyped (from the inbound strip directive). With a well known header this will result in a immediate security hole, which can theoretically go unnoticed.

Here is an example Apache httpd config, to illustrate my point:

RequestHeader set Sec-Client-X509-Cert-liede5vaePeeMiYie0xu2jaudauleing ""

RequestHeader set Sec-Client-X509-Cert-liede5vaePeeMiYie0xu2jaudauleing "%{SSL_CLIENT_CERT}s"

(the first line strips the inbound headers)



Vladimir Dzhuvinov
OAuth mailing list<>

ZmartZone IAM -<>


Vladimir Dzhuvinov