Re: Declarative HTTP Spec Test Suite

[Mohammed Al Sahaf <[e-mail]>]

On Monday, May 27th, 2024 at 4:43 PM, Daniel Stenberg <[e-mail]> wrote:

> On Sun, 26 May 2024, Mohammed Al Sahaf wrote:
> 
> > - Firefox fails the transfer on plaintext HTTP with
> > "NS_ERROR_NET_PARTIAL_TRANSFER"; but with HTTPS connection, it only reads
> > and displays payload per the number of bytes stated in the header value.
> 
> 
> As the lead developer of curl, I have found it easier to make curl be stricter
> than the browsers in some these areas - probably partly because we did it so
> from the beginning. But also: authoring a toolset that is often used to mimic
> or reproduce what a browser can do, we occasionally have to sacrifize spec
> compliance in order to allow the users to do exactly this. To rather mimic
> someone else's (non-compliant) behavior because that is what users expect.
> Because that's how the Internet works now. (Like in the case of URLs.)

I hear you. But how do you reconcile the need to mimic non-compliant behavior with behavior that varies across different agents? In the example I cited, when Content-Length reports fewer bytes than the payload, Chrome (sometimes) ignores the header, Firefox (sometimes) respects the header, while curl behaves consistently. The users' expectations aren't clear. I also wonder what other areas are there differing behavior as this case and what could be the impact, though they're likely edge-cases because no one complained loudly yet.

> > The first challenge is to find an HTTP client that implements HTTP Semantics
> > RFCs perfectly, otherwise its own idiosyncrasies will get in the way of the
> > validation. One would be tempted to default to curl, especially that hurl
> > uses curl under the hood. However, there is a chance curl may have its ownn
> > set of HTTP idiosyncrasies that may affect the results of the test
> > execution. Changes to curl are probably not desired unless the subject
> > behavior is confirmed to be a defect. Involvement of curl is voluntary to
> > curl, and the team may be looped and involved into this initiative for
> > comments if desired.
> 
> 
> It is not really in curl's interest to reach a highscore in a compliance test
> if that means that it interoperates less good. curl is not a HTTP compliance
> meter, it is an internet transfer tool and library.
> 
> This said, we of course want to comply as far as possible, but whenever there
> is a fork in the road, the decision might not always be to go with the
> strictest language in the most recent RFC. I'm also sure we can find downright
> bugs or just protocol silliness even in curl's implementation.
>
> Also, as has been discussed numerous times: the HTTP RFCs mostly describe how
> things should work and how to behave, not how to act when the other side does
> the wrong thing or how to fail etc.

Clear, and that's sensible and pragmatic. The goal is to find where the implementations disagree, identify the gaps, and perhaps retrofit the fixes into the protocol definition. I believe it'll be valuable to know the situations of "This is where X differs from {the rest, RFC}". Maybe it's a bug in the implementation. Maybe it's not an oversight, rather the devs have a good reason for sidestepping that part of the spec. Maybe it's a bug in the spec. What I hope to achieve with this is to shine a spotlight on the various implementations to find those dark corners. It'd be like the [Can I Use](https://caniuse.com/) comparison table.


All the best,
Mohammed