Compression dictionary issues for broader discussion

[Patrick Meenan <[e-mail]>]

We're making good progress on the compression dictionary encoding (thank
you to everyone for your help). There are 5 issues in-flight right now that
I wanted to bring to the attention of the wider group:


1. Include the dictionary hash in the response stream:
https://github.com/httpwg/http-extensions/issues/2770

Currently the hash used to compress a resource is communicated through the
Content-Dictionary response header which is potentially fragile. There is
discussion about creating a stream encoding that has a header before the
raw brotli/Zstandard stream with the dictionary hash (and potentially a
magic signature).


2. Pick one compression algorithm:
https://github.com/httpwg/http-extensions/issues/2756

There is some concern that shipping both brotli and Zstandard will fragment
the ecosystem and require all clients to implement both algorithms (or
servers to support both if some clients do not).


3. Zstandard compression window:
https://github.com/httpwg/http-extensions/issues/2754

Zstandard can only use the dictionary until the input passes the size of
the compression window (brotli has it available independent of the
compression window). There is discussion about what an appropriate window
size to use would be to allow clients to control their memory use while
still allowing for large delta-compression cases. Current discussion is
around setting it to the larger of 8 MB and 1.25x the dictionary size.


4. Link relation: https://github.com/httpwg/http-extensions/issues/2772

We need to add the definition for the link relation for compression
dictionaries (i.e. the HTML case where a separate dictionary should be
fetched for future requests). The current Chrome origin trial uses
"dictionary" but that is a bit too generic to use up for this use case.
"compression-dictionary" is appropriate but might be a bit long to include
in a header.


5. Hash format in HTTP Headers:
https://github.com/httpwg/http-extensions/issues/2781

There is some discussion of how the dictionary hashes should be represented
in HTTP headers. The main tension is between developer-friendliness of the
original hex encoding and the shorter (and more correct) sf-binary
representation.