Re: New Version Notification for draft-kazuho-early-hints-status-code-00.txt

[Cory Benfield <[e-mail]>]

> On 31 Oct 2016, at 14:25, Kazuho Oku <[e-mail]> wrote:
> 
> The draft proposes a new _informational_ status code that can be used
> for indicating hints to help a client start making preparations for
> processing the final response, while the server struggles to build the
> final response.
> 
> Actually, we already have a working implementation. H2O, our HTTP/2
> server, recognizes Link: rel=preload headers within an informational
> response sent from upstream, and tries to push the specified resource.
> One reason I decided to write this draft is because I believe it would
> be better to standardize the mechanism.

Kazuho,

This draft generally seems like a really good idea. It helps to resolve some of the substantial issues around the processing of large or high-latency responses, and as you have identified seems like it would make a good companion to HTTP/2 server push. I think this is a draft worth pursuing, and though I will not be at the next IETF meeting I look forward to seeing the progress through the IETF.

While I’m here, I’d like to discuss section 3. To provide some data points, most of the HTTP/1.1 clients in the Python ecosystem will not handle a new 1XX status code very well. A quick survey of the ecosystem suggests that the following tools will all misbehave:

- every standard library http client (urllib, urllib2, httplib), all of which treat only “100 Continue” specially: the rest will treat the 103 header block as the final response and, because it MUST NOT have a content-length, will read until connection close.
- every third-party library built on top of those tools, including urllib3, Requests, and httplib2, which inherit their misbehaviour.
- aiohttp, which special-cases 100 and 101 but not any others
- Twisted’s client, which special-cases 100

In fact, so far I haven’t found a single Python http client that would handle this response in any way that approaches sensible. I would even tolerating *ignoring* the response as a sensible behaviour, but we don’t even have that. In all cases, these clients treat the 103 response as the final response and totally lose track of their framing.

I doubt Python is unique here. While other languages may have libraries that are somewhat better at handling this situation, certainly some-to-many of them will continue to have this problem. This problem is also likely to be present throughout the long-tail of HTTP implementations: clearly it has been received wisdom by implementers that non-100 1XX status codes do not need to be supported because they will not be encountered. Worse, many of these implementations may be unable to update. Even if Python’s standard library http.client library was fixed to handle 1XX codes sensibly, the most widely-deployed version of Python (2.7) would not receive any such fix, meaning that all Python 2.7 users would be utterly confused by the receipt of a 103 status code.

This means that your suggestion that this should be negotiated is extremely important. I think it’s not “desirable” to negotiate this behaviour: I think it’ll have to be mandatory if we want to avoid screwing up a majority of long-tail HTTP/1.1 clients in the world. For HTTP/1.1, that’ll have to be negotiation by header, presumably by means of either an extension to the “Expect” header field or an equivalent header field that is inserted by analogy. For HTTP/2 this could be negotiated at the connection level by means of a settings value, or by the same header at the cost of one byte per request.

I’ll start filing bugs against relevant repositories to address this behaviour so that the Python ecosystem is less embarrassingly unprepared for further 1XX status codes, but in the short-to-medium term it would be much better if we could negotiate the 103 status code rather than assume it will function correctly.

Cory