Requesting change to Accept-Encoding [was Re: Compression Dictionaries draft 04]

["Roy T. Fielding" <[e-mail]>]

> On May 20, 2024, at 7:49 AM, Patrick Meenan <[e-mail]> wrote:
> 
> FYI, draft 04 of the compression dictionaries spec was just released: https://datatracker.ietf.org/doc/draft-ietf-httpbis-compression-dictionary/
> 
> The changes since draft 03 are:
> 
> - Added link relation ("compression-dictionary")
> - Moved the dictionary hash into the response payload and removed the "Compression-Dictionary" response header
> - Changed content-encodings to "dcb" and "dcz"
> - Specified the compression window settings for Zstandard and Brotli
> 
> We're converging on a final spec and the Chrome origin trials to-date have been extremely positive.
> 
> There is one remaining discussion topic that I'm aware of which is around the compression algorithms that we spec and if we should collapse to a single algorithm and manage fragmentation at the spec level: https://github.com/httpwg/http-extensions/issues/2756
> 
> If you're aware of any other issues or have concerns with something else in the spec, please let us know. Otherwise we're hoping to come to an agreement on the compression algorithms and then move towards last call and RFC.


A server can send Use-As-Dictionary when applicable to any response,
regardless of Accept-Encoding or user agent. This allows the original
200 response to be delivered without unnecessary Vary, and by edge
caches that might have no knowledge of dictionaries. That's great.

However, I would like to change the protocol in 6.1 (Accept-Encoding)
to require the following:

 o When (and only when) a client has a stored dictionary that
   matches a request, it can send Available-Dictionary and
   Accept-Encoding with its dictionary-compression encodings.

 o When a client does not have a stored dictionary that matches
   the request, or chooses not to use one for the request,
   the client MUST NOT send its dictionary-compression
   algorithms in Accept-Encoding.

IOW, I would require that clients not advertise their implemented
dictionary-compression encodings in Accept-Encoding for requests
that do not desire a dictionary-compression response, since such
unnecessary variance in HTTP can have a critical impact on caching.
This is consistent with the semantics of Accept-Encoding.

This change would allow dictionary-based compression to be deployed
for existing resources and existing CDNs without causing a
factorial expansion of Vary. IOW, it allows dictionary-compression
to be deployed without impacting the cacheability of resources
for non-dictionary-using clients.

The one negative is that it lessens the ability to detect when
dictionary-compression is usable before a dictionary is provided.
I am pretty sure we don't need that.

A 200 response that has been dictionary-compressed would, of course,
still be marked with "Vary: Accept-Encoding, Available-Dictionary"
and the chosen encoding given in "Content-Encoding". This would
not be affected by the above change.

The possibility of multiple cached responses for a given response,
each with its own Vary from its original request, is already defined
by HTTP caching. I doubt that everyone implements it correctly,
but that's not unusual. There's no other correct choice given how
origin servers select negotiable content.

Cheers,

....Roy T. Fielding, Senior Principal Scientist, Adobe