Re: Ambiguity on HTTP/3 HEADERS and QUIC STREAM FIN requirement

[Willy Tarreau <[e-mail]>]

On Fri, Jun 17, 2022 at 03:02:56PM +1000, Martin Thomson wrote:
> Hey Willy,
> 
> On Fri, Jun 17, 2022, at 14:53, Willy Tarreau wrote:
> > Probably, but I'm still seeing it as a workaround. I mean, since HTTP/1.0
> > we've been used to know when receiving a full headers block the entire
> > list of the header fields. And it looks like with H3 the headers block
> > is uncertain at the end of a HEADERS frame. 
> 
> I don't think that's right.  There are two layers at work here, but you do
> have a clear marker for the end of the header block.
> 
> The HEADERS frame has ended in this case, so you have a clear indication that
> you have all the headers.

Not exactly. In HTTP/1 we used to have Transfer-Encoding which was a
connection-level header field to bridge the gap between what was explicitly
advertised in headers and what could have been ambiguous at the connection
level (such as receiving a FIN late). In HTTP/2 we could get rid of this
header field because the HEADERS frame was able to arbor the END_STREAM
flag to indicate that there was no body following. And in H2, it's not the
same to send a HEADERS + ES and a HEADERS followed by DATA+ES. The first
one doesn't have a body, the second one has an empty body. Passing that
to HTTP/1 results in a transfer-encoding header being added to the second
one. So for H1 and H2 the presence or absence of a body has always been
explicit and known upon receipt of a complete headers block.

With H3 you have neither the transfer-encoding header nor the ES bit on
the frame to indicate that presence/absence. The only indication that
matches the H2 ES is the QUIC FIN that also signals the end of stream,
albeit at a lower level. That's why I think we've slowly deviated from
something very explict (H1) to something subtly explicit (H2) then
something ambiguous (H3).

> The QUIC stream, which in this case hasn't ended, so you aren't sure if
> content or trailers might follow.

Exactly.

> >    A sender which knows that no data follows the headers block SHOULD
> >    signal the end of the message by setting the FIN bit along with the
> >    HEADERS frame. A receiver that processes a HEADERS frame without
> >    seeing a FIN bit MAY expect more data to follow regardless of the
> >    HTTP method used.
> 
> This is entirely sensible advise to give, but it isn't always possible due to
> how different pieces of software work.  There are some rather fundamental
> design choices involved that can make it hard to guarantee that these two
> things leave at the same time.  As you say, most people can deliver on this,
> but not all.

I'm well aware. But it's easier to respect when you have the rule upfront
than when nobody told you there was a problem with this. Just like we've
all had to adjust certain painful things in our H2 implementations over
time, violating multiple layers, that resulted in new text being written
into RFC9113, there will be some adjustments needed to be done to various
implementations to fix some trouble detected in field.

> > Sure but while you can often do this when you're a server, you practically
> > can't when you're a gateway, because that would require to adjust the
> > behavior per-URI. 
> 
> Yeah, a gateway is in a bad position here because they don't really speak for
> the resource and so - without extra information about resources, which could
> be made for all resources on a server, but probably won't be - they have to
> sit on their hands, buffer requests, and pay the ridiculous cost of doing
> that.

Paying the cost of making two ends understand each other is the daily
job of a gateway :-)  Regardless it's also the one that takes all the
dirty stuff in the face and it needs to be robust by design. My concern
here precisely is that waiting will both make it less robust *and* will
possibly not work with some clients which forget to send their FIN.

> But if you take the fact that you have a clear signal that the headers are
> done, you can - even as a gateway - make some decisions.  It might not be
> 100% safe, but I can't see any origin servers complaining if you started
> processing from that point, for GET and HEAD requests at least.

Sadly that's not even true :-(  We've seen recently, I think it was
Elastic Search that takes JSON requests sent as the body of a GET
request. So now that we managed to better define the presence/absence
of a body in a request, we're back trying to guess it with a certain
probability based on a method, and I'd definitely not encourage
implementations to start to guess again.

Cheers,
Willy