Re: [hybi] preliminary WebSockets compression experiments

[Greg Wilkins <[e-mail]>]

Mike Belshe wrote:
> On Mon, Apr 26, 2010 at 7:05 AM, Greg Wilkins <[e-mail]
> <mailto:[e-mail]>> wrote:
> 
>     Mike,
> 
>     some slightly off topic thinking out loud on my idea that websockets
>     should be able to eventually replace the framing layer of SPDY....
> 
>     The problem that SPDY poses for efficient compression implemented in the
>     framing layer is that it's frames may contain both headers and
>     a content body.
> 
> 
> The most efficient compression will always come from the entity which
> knows the most about what it is compressing.   Compressing headers
> separately from other content compresses more efficiently than if you
> mix compression of headers and data.  (Imagine a gzip buffer with 32KB
> of window where you first send 500B of headers and then 50KB of data
> through it - by the time your next chunk of headers come through, all
> state from the prior headers has been lost.  And the data that you sent
> through might have been compressed anyway!)  
> 
> I welcome you to test out this theory - I've done so and I know the
> answer :-)   As you can see, header-specific compression is a feature.

I'm not disputing this.   Yet it is still useful to have compression
available when the source of the data does not compress it - otherwise
this would be an argument for HTTP not supporting compression.

If we are to have a multi-layered protocol then it would be
desirable to avoid every layer having it's own compression mechanism.

So my point was that since we appear to recognize that compression
is valuable to have in the framing layer (to handle apps that
do not compress), it would be great if we could design that
with enough flexibility so that SPDY over websocket would
not need to implement it's own compression.    Ideally the
framing compression would be flexible enough so that SDPY
would be able to specify a content-specific compressor
for header frames.

Now the complexity of layers delegating compression to
the framing layer may well be equal to the complexity of
each layer implementing it's own framing.  But I do think
it worth while evaluating if this is true or not.



> Roberto tried to convey why optional compression is a bad thing
> sometimes. 

Again, I'm not really disputing the validity of Roberto's example.
I understand that there are are intermediaries that act less than
sensibly.

However, I'm not sure that is sufficient reason to mandate a fixed
compression algorithm.    To do so, we would raise the complexity
of the simplest client (and while I agree that is not by much, there
is still a definite sentiment that we should keep it really simple).

Also, if we are to support future compression algorithms (or
content-specific ones), then we must support negotiated.  In
which case, there will be null compressors and lazy
intermediaries will just negotiate a noop compressor.

What Roberto's example does show, is that there in a genuine
desire of intermediaries (even ones baked into the client
computer) to inspect and add value to passing traffic.

cheers