RE: legality of Transfer-Encoding: chunked bodies in HTTP/2
Mike Bishop <Michael.Bishop@microsoft.com> Thu, 07 August 2014 01:19 UTC
Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3B23B1A03A7 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 6 Aug 2014 18:19:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.303
X-Spam-Level:
X-Spam-Status: No, score=-6.303 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_52=0.6, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r0OkPpfT5rZN for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 6 Aug 2014 18:19:23 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BF2951A029D for <httpbisa-archive-bis2Juki@lists.ietf.org>; Wed, 6 Aug 2014 18:19:23 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1XFCKC-0003wY-Uu for ietf-http-wg-dist@listhub.w3.org; Thu, 07 Aug 2014 01:17:01 +0000
Resent-Date: Thu, 07 Aug 2014 01:17:00 +0000
Resent-Message-Id: <E1XFCKC-0003wY-Uu@frink.w3.org>
Received: from lisa.w3.org ([128.30.52.41]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <Michael.Bishop@microsoft.com>) id 1XFCJv-0003vp-UY for ietf-http-wg@listhub.w3.org; Thu, 07 Aug 2014 01:16:43 +0000
Received: from mail-bl2lp0207.outbound.protection.outlook.com ([207.46.163.207] helo=na01-bl2-obe.outbound.protection.outlook.com) by lisa.w3.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from <Michael.Bishop@microsoft.com>) id 1XFCJu-0000hc-IM for ietf-http-wg@w3.org; Thu, 07 Aug 2014 01:16:43 +0000
Received: from BN1PR03MB137.namprd03.prod.outlook.com (10.255.201.12) by BN1PR03MB038.namprd03.prod.outlook.com (10.255.225.146) with Microsoft SMTP Server (TLS) id 15.0.995.11; Thu, 7 Aug 2014 01:15:57 +0000
Received: from BN1PR03MB137.namprd03.prod.outlook.com ([169.254.11.16]) by BN1PR03MB137.namprd03.prod.outlook.com ([169.254.11.16]) with mapi id 15.00.0995.014; Thu, 7 Aug 2014 01:15:57 +0000
From: Mike Bishop <Michael.Bishop@microsoft.com>
To: Osama Mazahir <OSAMAM@microsoft.com>, Mark Nottingham <mnot@mnot.net>, Michael Sweet <msweet@apple.com>
CC: Zhong Yu <zhong.j.yu@gmail.com>, Martin Thomson <martin.thomson@gmail.com>, Johnny Graettinger <jgraettinger@chromium.org>, HTTP Working Group <ietf-http-wg@w3.org>
Thread-Topic: legality of Transfer-Encoding: chunked bodies in HTTP/2
Thread-Index: AQHPsN0/fuwrtcKgv0eB6zquYYqKXJvCcJcAgAAFngCAAAjDAIAAOyQAgABBBgCAAVHXAIAAClow
Date: Thu, 07 Aug 2014 01:15:56 +0000
Message-ID: <67ae17515e424ba39be3127edff351fe@BN1PR03MB137.namprd03.prod.outlook.com>
References: <CAEn92Tq72TK9OrFqnLFpsAVi2+EFjsMEwHtMBg_195NoJg150w@mail.gmail.com> <CABkgnnXjV1JqunhmanK_ZXH_5CuZzD3SF2vLKTgXE157Esf7aw@mail.gmail.com> <CACuKZqHkQKzHGJcOHFWVgZNU_pX88APDrO6+Yr_qjDXb=CW41Q@mail.gmail.com> <b197816b29f64fae847132a8945ce0de@SN2PR03MB046.namprd03.prod.outlook.com> <0D91B271-EA17-4906-A723-277C069656DF@apple.com> <18711EE1-458B-4172-80B1-DA9E2D3FDBB5@mnot.net> <214054b7df3d43ddaaeae2d83f6b2283@SN2PR03MB046.namprd03.prod.outlook.com>
In-Reply-To: <214054b7df3d43ddaaeae2d83f6b2283@SN2PR03MB046.namprd03.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [70.199.134.180]
x-microsoft-antispam: BCL:0;PCL:0;RULEID:
x-forefront-prvs: 029651C7A1
x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(377454003)(57704003)(189002)(24454002)(199002)(51704005)(13464003)(15975445006)(87936001)(81542001)(64706001)(15202345003)(21056001)(20776003)(76176999)(54356999)(106116001)(81342001)(19580395003)(101416001)(66066001)(107046002)(95666004)(19580405001)(83072002)(74316001)(85852003)(83322001)(92566001)(99286002)(76576001)(99396002)(105586002)(2656002)(80022001)(77096002)(4396001)(76482001)(46102001)(86362001)(79102001)(74662001)(74502001)(77982001)(50986999)(106356001)(93886004)(217423001)(31966008)(85306004)(33646002)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BN1PR03MB038; H:BN1PR03MB137.namprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en;
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.onmicrosoft.com
Received-SPF: pass client-ip=207.46.163.207; envelope-from=Michael.Bishop@microsoft.com; helo=na01-bl2-obe.outbound.protection.outlook.com
X-W3C-Hub-Spam-Status: No, score=-3.3
X-W3C-Hub-Spam-Report: AWL=-2.565, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001
X-W3C-Scan-Sig: lisa.w3.org 1XFCJu-0000hc-IM 012a8ca7f098c070d0baf5279c87c45b
X-Original-To: ietf-http-wg@w3.org
Subject: RE: legality of Transfer-Encoding: chunked bodies in HTTP/2
Archived-At: <http://www.w3.org/mid/67ae17515e424ba39be3127edff351fe@BN1PR03MB137.namprd03.prod.outlook.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/26551
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
Or 3b, Intermediary receives HTTP/2 response headers containing a Content-Length, and it just dumps the DATA payload into the TCP connection as it arrives. If the CL is incorrect, that's the origin's problem/fault. But yes, at the transition point, prohibiting Transfer-Encoding strongly suggests that the absence of Content-Length implies creating the chunks off the DATA frames. -----Original Message----- From: Osama Mazahir [mailto:OSAMAM@microsoft.com] Sent: Wednesday, August 6, 2014 5:35 PM To: Mark Nottingham; Michael Sweet Cc: Zhong Yu; Martin Thomson; Johnny Graettinger; HTTP Working Group Subject: RE: legality of Transfer-Encoding: chunked bodies in HTTP/2 So we are forcing 1.1<-->2.0 intermediaries to transform the body by stripping & adding chunked encoding instead of merely encapsulating the body into DATA frames? 1. Intermediary receives HTTP/1.1 chunked request body. Intermediary strips "Transfer-Encoding: chunked". 2. Intermediary strips chunk headers as it constructs the DATA frames. The final 0-chunk maps to an END_STREAM. 3. Intermediary receives HTTP/2 response body consisting of DATA frames (no "Transfer-Encoding: chunked"). 4. Intermediary adds chunk boundaries as it constructs the DATA frames. The END_STREAM maps to the final 0-chunk. I'm confused. Either "Transfer-Encoding: chunked" is illegal and intermediaries MUST convert or it's legal and HTTP2 receivers have to be able to handle "Transfer-Encoding: chunked". -----Original Message----- From: Mark Nottingham [mailto:mnot@mnot.net] Sent: Tuesday, August 5, 2014 9:25 PM To: Michael Sweet Cc: Osama Mazahir; Zhong Yu; Martin Thomson; Johnny Graettinger; HTTP Working Group Subject: Re: legality of Transfer-Encoding: chunked bodies in HTTP/2 Just to make sure we're all on the same page -- *logically* HTTP/2 is still chunked, because you still split things up into DATA frames. However, the "chunk" format specified in <http://httpwg.github.io/specs/rfc7230.html#chunked.encoding> will NOT be considered part of the message framing in HTTP/2; if that shows up, it'll be considered payload (and therefore visible to the application as content, which is probably NOT what you want). Cheers, On 6 Aug 2014, at 10:32 am, Michael Sweet <msweet@apple.com> wrote: > DATA frames are already equivalent to HTTP/1.1 chunking. Why do we need to chunk twice? > > On Aug 5, 2014, at 5:00 PM, Osama Mazahir <OSAMAM@microsoft.com> wrote: > >> +1 for allowing Chunked. >> >> Windows machines are going to end up output chunked responses (well...technically anything built on top of the Windows HTTP networking components). We cannot rewrite all the Windows applications to avoid chunking and having some magical translation layer doing unbounded buffering to convert into Content-Length prefixed form is bad. Stripping the chunk boundaries when encapsulating into DATA frames requires a reparser in the data send path, which is merely swapping one problem for another (and may require the peer to do the opposite if converting back into 1.1 form). >> >> It may be _ineffecient_ to have chunks but it should be legal, to maximize 1.1<-->2.0 fidelity. >> >> We can probably address this via editorial changes indicating that in pure HTTP2, using END_STREAM only is the preferred way to mark end-of-body (or, of course, trailers). Along with all the normal caveats of smuggling bytes in DATA frames after the final chunk, etc. >> >> -----Original Message----- >> From: Zhong Yu [mailto:zhong.j.yu@gmail.com] >> Sent: Tuesday, August 5, 2014 1:30 PM >> To: Martin Thomson >> Cc: Johnny Graettinger; HTTP Working Group >> Subject: Re: legality of Transfer-Encoding: chunked bodies in HTTP/2 >> >> On Tue, Aug 5, 2014 at 3:09 PM, Martin Thomson <martin.thomson@gmail.com> wrote: >>> On 5 August 2014 11:39, Johnny Graettinger <jgraettinger@chromium.org> wrote: >>>> Section 8.1.3 "Examples" provides guidance on how an intermediary >>>> should do so, however section 8.1.2.2 "Hop-by-Hop Header Fields" >>>> says that stripping Transfer-Encoding is a SHOULD, not a MUST. >>> >>> Yeah, so the only reason that it's not a MUST is that some >>> intermediaries don't look and we don't want to force them to look. >>> That's all. Transfer-Encoding has no meaning in HTTP/2 (unless an >>> extension restores that meaning). >>> >>> Seeing the chunked framing in DATA frames would be wrong. >>> >> >> That is quite confusing and dangerous. Should we require that anyone >> *generating* the initial HTTP2 message MUST NOT include the Transfer-Encoding header? >> >> Zhong Yu >> bayou.io >> > > _________________________________________________________ > Michael Sweet, Senior Printing System Engineer, PWG Chair > -- Mark Nottingham https://www.mnot.net/
- legality of Transfer-Encoding: chunked bodies in … Johnny Graettinger
- Re: legality of Transfer-Encoding: chunked bodies… Martin Thomson
- Re: legality of Transfer-Encoding: chunked bodies… Zhong Yu
- RE: legality of Transfer-Encoding: chunked bodies… Osama Mazahir
- Re: legality of Transfer-Encoding: chunked bodies… Martin Thomson
- Re: legality of Transfer-Encoding: chunked bodies… Martin Thomson
- Re: legality of Transfer-Encoding: chunked bodies… Ryan Hamilton
- Re: legality of Transfer-Encoding: chunked bodies… Ryan Hamilton
- Re: legality of Transfer-Encoding: chunked bodies… Greg Wilkins
- Re: legality of Transfer-Encoding: chunked bodies… Zhong Yu
- Re: legality of Transfer-Encoding: chunked bodies… Greg Wilkins
- Re: legality of Transfer-Encoding: chunked bodies… Michael Sweet
- Re: legality of Transfer-Encoding: chunked bodies… Mark Nottingham
- RE: legality of Transfer-Encoding: chunked bodies… Mike Bishop
- Re: legality of Transfer-Encoding: chunked bodies… Greg Wilkins
- Re: legality of Transfer-Encoding: chunked bodies… Jason Greene
- RE: legality of Transfer-Encoding: chunked bodies… K.Morgan
- RE: legality of Transfer-Encoding: chunked bodies… Osama Mazahir
- RE: legality of Transfer-Encoding: chunked bodies… Mike Bishop
- Re: legality of Transfer-Encoding: chunked bodies… Martin Thomson
- RE: legality of Transfer-Encoding: chunked bodies… Osama Mazahir
- Re: legality of Transfer-Encoding: chunked bodies… Martin Thomson
- RE: legality of Transfer-Encoding: chunked bodies… Mike Bishop
- Re: legality of Transfer-Encoding: chunked bodies… Martin Thomson
- Re: legality of Transfer-Encoding: chunked bodies… Mark Nottingham
- Re: legality of Transfer-Encoding: chunked bodies… Amos Jeffries
- Re: legality of Transfer-Encoding: chunked bodies… Martin Thomson