Re: p1: whitespace in request-target

Amos Jeffries <squid3@treenet.co.nz> Thu, 18 April 2013 03:50 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 63A5021F8D94 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 17 Apr 2013 20:50:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.599
X-Spam-Level:
X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AtF06WbQtQk8 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 17 Apr 2013 20:50:19 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id 687D821F8D90 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Wed, 17 Apr 2013 20:50:19 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1USfq1-0003qO-K5 for ietf-http-wg-dist@listhub.w3.org; Thu, 18 Apr 2013 03:48:45 +0000
Resent-Date: Thu, 18 Apr 2013 03:48:45 +0000
Resent-Message-Id: <E1USfq1-0003qO-K5@frink.w3.org>
Received: from lisa.w3.org ([128.30.52.41]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <squid3@treenet.co.nz>) id 1USfpy-0003pj-Te for ietf-http-wg@listhub.w3.org; Thu, 18 Apr 2013 03:48:42 +0000
Received: from ip-58-28-153-233.static-xdsl.xnet.co.nz ([58.28.153.233] helo=treenet.co.nz) by lisa.w3.org with esmtp (Exim 4.72) (envelope-from <squid3@treenet.co.nz>) id 1USfpx-0000kZ-N0 for ietf-http-wg@w3.org; Thu, 18 Apr 2013 03:48:42 +0000
Received: from [192.168.2.7] (103-9-43-128.flip.co.nz [103.9.43.128]) by treenet.co.nz (Postfix) with ESMTP id 46334E711D; Thu, 18 Apr 2013 15:48:13 +1200 (NZST)
Message-ID: <516F6CF9.30709@treenet.co.nz>
Date: Thu, 18 Apr 2013 15:48:09 +1200
From: Amos Jeffries <squid3@treenet.co.nz>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5
MIME-Version: 1.0
To: Mark Nottingham <mnot@mnot.net>
CC: "ietf-http-wg@w3.org Group" <ietf-http-wg@w3.org>, Roy Fielding <fielding@gbiv.com>
References: <2183465A-F833-4701-A55C-EC105A36329E@mnot.net>
In-Reply-To: <2183465A-F833-4701-A55C-EC105A36329E@mnot.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: pass client-ip=58.28.153.233; envelope-from=squid3@treenet.co.nz; helo=treenet.co.nz
X-W3C-Hub-Spam-Status: No, score=-3.5
X-W3C-Hub-Spam-Report: AWL=-3.449, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001
X-W3C-Scan-Sig: lisa.w3.org 1USfpx-0000kZ-N0 c8cb9dac725ba53a7ed93ae01a74ed3e
X-Original-To: ietf-http-wg@w3.org
Subject: Re: p1: whitespace in request-target
Archived-At: <http://www.w3.org/mid/516F6CF9.30709@treenet.co.nz>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/17323
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

On 18/04/2013 12:49 p.m., Mark Nottingham wrote:
> p1 3.1.1 says:
>
>> Unfortunately, some user agents fail to properly encode hypertext references that have embedded whitespace, sending the characters directly instead of properly encoding or excluding the disallowed characters. Recipients of an invalid request-line SHOULD respond with either a 400 (Bad Request) error or a 301 (Moved Permanently) redirect with the request-target properly encoded. Recipients SHOULD NOT attempt to autocorrect and then process the request without a redirect, since the invalid request-line might be deliberately crafted to bypass security filters along the request chain.
>    http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-22#section-3.1.1
>
> I note that the practice of correcting this is fairly widespread; e.g., in Squid, the default is to strip the whitespace, and IIRC has been for some time:
>
>    http://www.squid-cache.org/Doc/config/uri_whitespace/
>
> I think that the Squid documentation needs to be corrected, because the text in RFC2396 (and later in 3986) is about URIs in contexts like books, e-mail and so forth, not protocol elements:
>
>    http://tools.ietf.org/html/rfc3986#appendix-C

The relevant portion there being:
  "

    For robustness, software that accepts user-typed URI should attempt
    to recognize and strip both delimiters and embedded whitespace.
"
Note that Squid *does* accept user-typed HTTP messages and the software traditionally causing whitespace issues
are usually the specialized clients sending URI through HTTP-compatible messages in the URL field (Outlook and Exchange with RCP, some shockwave ICY clients).

> My question is why this is a SHOULD / SHOULD NOT. We say that SHOULD-level requirements affect conformance unless there's a documented exception here:
>
>    http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-22#section-2.5
>
> ... but these requirements don't mention any exceptions. Is the security risk here high enough to justify a MUST / MUST NOT? If not, they probably need to be downgraded to ought (or an exception needs to be highlighted).

The biggest risk is software truncating portions of the URL, or (like 
Squid when there is a trailing SP on the line) determining that the 
HTTP/1 version label is part of the URL on a HTTP/0.9 syntax GET request 
- which results in any HTTP/1.x header features being ignored. The 
actual security worst-case risk of this undeterminable, but its not 
going to be good for the transaction at the best of times.

Amos