Re: Stream limits draft posted

[Ben Schwartz <[e-mail]>]

If we truly believe that "fixed H2 only" is a behavior we want to endorse for both clients and servers, a new ALPN seems appropriate.  However, that possibility comes with a variety of headaches, and I think we should try to avoid it.

Consider a DoH server that only supports "h2.1" and "h3".  (DoH explicitly rules out HTTP/1.1.)  A (standards-compliant) DoH client that only supports "h2" would fail to connect.

Conversely, imagine a far future when "broken H2" is mostly forgotten, and clients start to drop support for "h2".  It's not enough to update your servers to support "h2.1" (which could have happened automatically via software upgrade): you also need to update your HTTPS records (which is likely manual).

Defining a new ALPN seems likely to generate a large amount of operational toil and compatibility stress.  I think we would be better off avoiding a compatibility break, by (1) defining an in-band optional upgrade like ALPS and (2) documenting HTTP/2 server behaviors required to make the baseline protocol sufficiently safe.  (A recommendation for the HAProxy rule [1] might be all we really need.)

--Ben Schwartz

[1] https://lists.w3.org/Archives/Public/ietf-http-wg/2023OctDec/0027.html
________________________________
From: David Benjamin <[e-mail]>
Sent: Tuesday, November 14, 2023 4:30 PM
To: Ben Schwartz <[e-mail]>
Cc: Ilari Liusvaara <[e-mail]>; [e-mail] <[e-mail]>
Subject: Re: Stream limits draft posted

Can you elaborate on the compatibility and configuration issues? h2. 1-only clients is a good example of why the weird implicit ALPS thing isn't ideal. It allows a server to say "I'm willing to do fixed-h2 but not broken-h2",
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender

ZjQcmQRYFpfptBannerEnd
Can you elaborate on the compatibility and configuration issues? h2.1-only clients is a good example of why the weird implicit ALPS thing isn't ideal. It allows a server to say "I'm willing to do fixed-h2 but not broken-h2", but it doesn't allow a client to say this. Less of a serious concern than the server, but it's something we give up by losing the orthogonality.

I also think this allergy towards minting ALPNs is bad for the evolvability of our protocols. There are plenty of problems where minting a new ALPN is the right solution. If we have systems that cannot handle it, that's a problem and we should exercise them better and fix this. Otherwise we'll have ossified yet another extension point.

On Mon, Nov 13, 2023 at 11:30 AM Ben Schwartz <[e-mail]<mailto:[e-mail]>> wrote:
If ALPS is in use, I think a new ALPN ID is unnecessary.  ALPS implicitly defines a new profile of each offered ALPN.  We can define a SETTINGS_USE_MAX_STREAMS setting for H2, and note that client support for this setting (and any other settings that we believe should be in a "modern" client baseline) is mandatory when offering ALPS for "h2".

This does move toward ALPS influencing the server's ALPN selection.  Perhaps that is less elegant than perfect orthogonality, but I think it's better than minting new ALPN IDs.  In particular, a new ALPN ID creates various compatibility and configuration headaches related to "h2.1-only" clients and servers, and HTTPS records.

--Ben Schwartz
________________________________
From: David Benjamin <[e-mail]<mailto:[e-mail]>>
Sent: Sunday, November 12, 2023 6:55 PM
To: Ilari Liusvaara <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>
Subject: Re: Stream limits draft posted

Even over TLS 1. 3, I would expect the vast, vast majority of HTTP/2 servers do not send SETTINGS at half-RTT. When I probed some list of top sites in 2020, I could not find a single one. This should not be surprising once you dig into it. Half-RTT
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender

ZjQcmQRYFpfptBannerEnd
Even over TLS 1.3, I would expect the vast, vast majority of HTTP/2 servers do not send SETTINGS at half-RTT. When I probed some list of top sites in 2020, I could not find a single one.

This should not be surprising once you dig into it. Half-RTT data on a 1-RTT handshake is semantically weird, in a way that half-RTT on a 0-RTT handshake is not. It really does not fit well into TLS-over-TCP's existing interfaces at all.On the client side, taking a 1-RTT hit against practically all existing HTTP/2 servers seems clearly an unreasonable performance cost. Ultimately, it is a change in protocol semantics to go from "sending SETTINGS at 1-RTT is just fine" to "SETTINGS must be sent at half-RTT to avoid latency problems". On the server side, I don't believe it's possible to send it with OpenSSL servers. BoringSSL too, which was an intentional choice on our part.

I wrote this a couple years ago, which discusses this and other problems with trying to solve this shape of problem with half-RTT:
https://www.ietf.org/archive/id/draft-davidben-tls-alps-half-rtt-00.html<https://www.ietf.org/archive/id/draft-davidben-tls-alps-half-rtt-00.html>

I haven't been following this issue as closely, but skimming the draft, it seems to me three questions here:
1. How to ensure clients know to trigger the new mode early enough.
2. How to allow clients to open streams in the new mode without waiting for an RTT, in both 1-RTT and 0-RTT handshakes.
3. How to distinguish clients that do and don't implement the new scheme so that, either under load or perhaps unconditionally in the future, servers can avoid negotiating h2 with older clients.

I suspect any solution for (2) will either look like early server->client communication (be it ALPS or a Rube Goldberg of half-RTT, 0-RTT session ticket state, and applications injecting themselves into the 0-RTT accept/reject decision), or, as Kazuho suggested, just some hard-coded initial default value. Both the ALPS and half-RTT plans will require TLS software changes, so I suspect the hard-coded initial value is most viable, provided we can find a small enough default to satisfy DoS concerns, but large enough to admit one RTT worth of stream creations.

But then we don't solve (1) for free. For that, I think the new ALPN is the way to go. As this is essentially a protocol bugfix, rather than an optional extension, a version bump to "h2.1" seems appropriate.  ALPN happens early enough, and is already correctly integrated with 0-RTT, so the client will know, before sending data, to use the new mode. Moreover, the server knows whether the client will behave before picking the protocol, so we get (3) as well, which the half-RTT and ALPS options would not give us. (ALPS puts the client message in the second client flight so that it is encrypted. That means the server only knows "client supports ALPS" at protocol selection time. It also was designed assuming it wouldn't impact protocol selection.)

The hardcoded default is a little unsatisfying compared to early server->client, but we could do both: if we get a server->client initial value in ALPS, use that. Otherwise, use the hard-coded default. (This isn't an option with half-RTT because the client needs to know when to stop waiting, when talking to a server that can't send it.)

(To fully explore the problem space, a new ALPN does resolve the client performance problems of the half-RTT approach: with a new ALPN we can say "you must send SETTINGS in half-RTT in h2.1". But all the other problems, such as those in the link and requiring TLS library changes, remain. I do not think half-RTT is viable here.)

On Sat, Nov 11, 2023 at 3:37 AM Ilari Liusvaara <[e-mail]<mailto:[e-mail]>> wrote:
On Sat, Nov 11, 2023 at 08:50:51AM +0100, Martin Thomson wrote:
> On Fri, Nov 10, 2023, at 14:16, Stefan Eissing wrote:
> > +1
> >
> > I think clients will not wait for the MAX_STREAMS from the server. I
> > assume they want to send right away instead of waiting for a while. The
> > server might support it or not. How long are they willing to wait for
> > this?
>
> In the meeting, it was suggested that we require clients to wait for
> SETTINGS before opening streams.  I think that is a good idea and
> entirely possible, without significant performance penalties.
>
> In TLS 1.3, the server can send SETTINGS immediately, which means that
> the client only needs to read a few more bytes before it can start
> sending.
>
> The minor pain here, as David Benjamin will likely remind us, is that
> it is a little awkward sending data from the server before seeing the
> client Finished.  So we'd probably want to experiment here.

I have written a HTTP reverse proxy, which used to send SETTINGS
immediately in TLS 1.3[1]. Unfortunately, that caused some clients to
break[2], manifesting as fatal unexpected_message TLS alert immediately
after handshake completed. So I changed the code to always wait for
finished from client[3].


[1] If there was no client certificate request, for shortcomings of
the interface between TLS and HTTP layers.

[2] No information about what clients broke. However, mentions by
one user might indicate that some "anti-virus" products were
involved.

[3] There might still be option to re-enable the old behavior.




-Ilari