Re: New Version Notification for draft-kazuho-early-hints-status-code-00.txt

[Cory Benfield <[e-mail]>]

> On 1 Nov 2016, at 22:50, Roy T. Fielding <[e-mail]> wrote:
> 
>> On Nov 1, 2016, at 1:17 AM, Cory Benfield <[e-mail]> wrote:
>> 
>> I’m right there with you Julian. The 1XX response category gets to be another marker pointing us to the lesson the IETF has been learning for the past decade or so: extension points on a specification that no-one uses rust over time and become unusable.
> 
> No.  What I've learned is that every feature in every protocol is poorly
> implemented by some poor soul who thinks they deserve special consideration
> for their inability to interoperate with the future.  I have, in the past,
> consistently refused such considerations.

I don’t understand where you think anyone who wrote a broken implementation is asking for special consideration. The only HTTP implementation I fully wrote is a HTTP/2 implementation that can handle 1XX codes per the specification. I certainly don’t need the special consideration for libraries I maintain, because I’ll be shipping patches for them.

I am simply informing the IETF that the vast majority of widely deployed HTTP client libraries today will fail in the face of the 103 status code. Since I looked at Python I have gone back and looked at Ruby, Javascript, and Go, and the same remains true there. All of these languages have implementations that surface the 103 to the user and swallow the 200. So far curl and wget are the only widely-deployed non-browser implementations I have found that handle the 103 the way that RFC 7230 says they should. While we should praise those implementers, we should acknowledge that curl and wget are so widely deployed that they get bug reports for all the wacky edge cases in the way that the smaller implementations do not. (Yes, I called 1XX a wacky edge case, let’s all move on from it.)

>> In this case, I think the 1XX problem is more oversight than anything else. The problems in all these cases are tractable, and can be fairly easily fixed. It’s just that someone needs to spend that time.
> 
> They are easily fixed.  Force the broken implementations to die in a miserable
> way and teach people not to write crappy code.

That fixes nothing. While I’m sure it’s emotionally satisfying to metaphorically slap developers around the face and scream “OBEY. THE. RFC” at them, the very existence of RFCs 7230+ is a good reminder that developers almost universally do not pay attention to the RFCs: they pay attention to the protocol *as encountered on the web*. That protocol has had for its entire lifetime only two 1XX status codes: 100 and 101. Protocol implementations almost universally handle those status codes correctly: just no others in the range. And that hasn’t mattered because no-one is *using* those other codes.

And so, we have come to use an extension point that was written into the protocol 17 years ago and never used, and we have found that it is rusty. This should not be a surprise to us. We should be equally wary of all the other unused extension points in the protocol (chunk extensions, anyone?), because all will trigger this similar failure mode.

Actually fixing this problem requires developer effort to write patches and to land them. I had already acknowledged this in the quote above: I was merely pointing out that we are locking the stable door after the horse has bolted. Thanks to the magic of long-term-support Linux releases the vast majority of deployed code today cannot handle the 103 status code, and that code will be with us for at least five years regardless of whether or not we patch the software.

All of this *excludes* the alarming reality of the many millions of embedded HTTP/1.1 stacks which are implementing a protocol that is at best a second cousin to the one specified in RFCs 7230+. The odds of those tolerating this response are pretty low: I’d be pretty impressed if you could find one.

> There is absolutely no reason to negotiate 1xx codes.  If some application fails
> because their developers can't read, it is not our responsibility to work around them.
> If we do anyway, the entire Internet goes to crap (just like it has for HTML).
> At most, we use User-Agent or Server to flag non-compliant implementations and
> work around only specific versions of known-to-be-deployed breakage.

Well, firstly, let me note that all currently-specced 1XX status codes *are* negotiated. While 100 Continue is allowed to be sent without an “Expect” header requesting it, in practice it never is, so user-agents that don’t send the “Expect" header will never see it. Similarly, 101 is only sent in response to requests with an “Upgrade” header, so once again user-agents that never emit an “Upgrade: header will never see it. So while the spec sure as hell says that 1XX codes can be sent whenever, in practice they are only seen by user-agents that ask to see them.

Regardless, we can choose to flag non-compliant implementations. You’ll find, however, that the list of implementations is going to be pretty long. I have yet to bump into a HTTP implementation made available by a language standard library that handles 103 correctly, and most third-party implementations don’t either. That means the “known to be deployed” breakage basically includes almost every non-PHP web application deployed on the web today.

What that means is that the user-agent field will not be used to flag non-compliant implementations, it will be used to flag *compliant* ones, as there are vastly more of the former than the latter. That means that Chrome, Safari, Firefox, Opera (maybe), curl, and wget will all get a pass, and everyone else will be “guilty until proven innocent”. That means that we are rolling out a feature that is expressly a "browsers-only” feature if we deploy it in this way.

Let’s not pretend that the doors will be slowly opened as other implementations get their houses in order, because that’s not what will happen. What will happen, instead, is that the implementations that really are making a genuine effort to be of high quality but didn’t get around to this niche use-case yet because they’re somewhat less well-resourced than the others will be forever locked out of this feature, because there is no registry of “user-agents that support feature X” that can be updated. The only option they’ll have is to user-agent spoof, and I thought we were trying not to encourage that any more.

I like this 103 status code, and I want it to be as widely deployable as possible. It seems like the easiest way to do that *today*, without causing server administrators angst about whether they’ll break their clients, is to use a header field that flags support of the feature. It’s certainly not mandatory, but the cost of it is pretty damn low. In the meantime, for HTTP/2, where the implementations are generally newer and of higher quality at this time, we can safely avoid negotiation if that makes you happier.

Cory

(Vaguely relatedly, Roy, your assertion that these implementations fail because “their developers can’t read” is one of the most grossly petty things I’ve read on this mailing list. Most of the implementations I’ve tested are open-source implementations that are maintained by one-to-two developers, part time. It turns out that the real world of implementing HTTP/1.1 is sufficiently complex that writing code to support a feature that has been used literally zero times prior to this moment was not high up the priority list of implementers. So it’s not that they "can’t read", it’s that they prioritised their work items to actually solve the problems their users have, and “being served unexpected 1XX status codes” wasn’t on that list. Clearly all us independent implementers are amateur-hour hacks with no business being on the web.)