Re: #227: Encoding advice for new headers and parameters

Willy Tarreau <> Sat, 01 October 2016 06:32 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 7B6ED12B03C for <>; Fri, 30 Sep 2016 23:32:23 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -9.237
X-Spam-Status: No, score=-9.237 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.316, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id nwIaHKfNitvK for <>; Fri, 30 Sep 2016 23:32:22 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 2CFC712B054 for <>; Fri, 30 Sep 2016 23:32:21 -0700 (PDT)
Received: from lists by with local (Exim 4.80) (envelope-from <>) id 1bqDmZ-0001C8-Ny for; Sat, 01 Oct 2016 06:28:23 +0000
Resent-Date: Sat, 01 Oct 2016 06:28:23 +0000
Resent-Message-Id: <>
Received: from ([]) by with esmtps (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <>) id 1bqDmX-0001An-Jq for; Sat, 01 Oct 2016 06:28:21 +0000
Received: from ([] by with esmtp (Exim 4.80) (envelope-from <>) id 1bqDlC-00074t-Th for; Sat, 01 Oct 2016 06:27:50 +0000
Received: (from willy@localhost) by pcw.home.local (8.15.2/8.15.2/Submit) id u916QWJN031692; Sat, 1 Oct 2016 08:26:32 +0200
Date: Sat, 1 Oct 2016 08:26:32 +0200
From: Willy Tarreau <>
To: Julian Reschke <>
Cc: Mark Nottingham <>, HTTP Working Group <>
Message-ID: <>
References: <> <>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <>
User-Agent: Mutt/1.6.0 (2016-04-01)
Received-SPF: pass client-ip=;;
X-W3C-Hub-Spam-Status: No, score=-5.5
X-W3C-Hub-Spam-Report: AWL=-0.575, BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_IRA=-1, W3C_WL=-1
X-W3C-Scan-Sig: 1bqDlC-00074t-Th 5381ef9062788e7301464ef639629d15
Subject: Re: #227: Encoding advice for new headers and parameters
Archived-At: <>
X-Mailing-List: <> archive/latest/32433
Precedence: list
List-Id: <>
List-Help: <>
List-Post: <>
List-Unsubscribe: <>

On Thu, Sep 29, 2016 at 03:48:29PM +0200, Julian Reschke wrote:
> > A more aggressive approach would be to also recommend that new parameters on existing fields (even if they specify use of 5987) SHOULD use such encoding.
> -1 to mixing different escaping rules in the same field.

That reminds me that there are some applications where you can specify a
field name for something, and the value will be passed there. What
applications do in practice is to emit "$name: $value" without checking
*anything* about $name (eg: content-length will work), and by applying
the same encoding on $value. So indeed, having possibly confusing
encoding rules is a bit tricky. BTW, I used to work on a project where
we had two different encodings depending on the field name, and one or
two fields having to carry the same value for legacy reasons, but with
a different encoding, and we found several time code parts like this :

           value = legacy_header_encode(value);
           req_ptr += sprintf(req_ptr, "legacy_name: %s\n", value);
           req_ptr += sprintf(req_ptr, "new_name: %s\n", value);

Here new_name should have been encoded with new_header_encode(). And that's
without telling about the number of times "value" is passed unencoded,
possibly carrying escape characters... Of course above it seems obvious
but when reading such code it's much less. Thus it's important to be very
careful about this.