Re: [hybi] permessage-deflate performance tuning statistics

[Adam Rice <[e-mail]>]

Very interesting results, thank you!

On 16 October 2013 05:50, Peter Thorson <[e-mail]> wrote:
>
> For cases without context takeover it would get more complicated, do you
> retain context between fragments? If so that negates all of the memory
> benefits of no-context-takeover… This needs to be clarified in the
> permessage-compress draft spec.
>

Context *is* retained between frames. This is the meaning of "permessage"
in "permessage-compression". The wording in the spec could maybe be
clearer. I was confused about this myself for a long time.

The consequence is that you are not guaranteed to need less compression
contexts than connections, unless you know something about the traffic you
are handling. Specifically, if you know all your messages will be smaller
than the size of the compression context, then you know you can save memory
by only keeping one compression context around. So a browser or a proxy
cannot be certain to benefit from no-content-takeover, but a server can
usefully use it on applications where it makes sense.


> It would definitely be worth experimenting with a connection that
> multiplexes small messages with large ones, say for the example case where
> you are delivering new documents over the WebSocket connection as we'll as
> small updates. Getting some tests with binary/compressed data intermixed
> would also be useful it sounds. Does anyone have any sample data for such a
> connection?
>
>
An interesting property of valid UTF-8 data is that given a sufficiently
large quantity, it is always compressible. Binary data, on the other hand,
is quite likely to have had compression applied to it already and be
incompressible by a generic algorithm like deflate. So a WebSocket with a
mixture of text and binary messages is challenging to compress effectively
and efficiently.

Thanks,
Adam Rice