Re: [hybi] Payload only compression extension, again

[Takeshi Yoshino <[e-mail]>]

Hi,

On Sat, Jul 23, 2011 at 07:43, Greg Wilkins <[e-mail]> wrote:

> On 23 July 2011 00:38, Takeshi Yoshino <[e-mail]> wrote:
> > I've implemented this proposal on pywebsocket.
> pywebsocket's deflate-stream
> > implementation flushes compressed data for each frame using Z_SYNC_FLUSH.
> > Compression level is Z_DEFAULT_COMPRESSION. Default Huffman tree is used.
> No
> > dictionary is attached.
>
> Great - I'll make sure we have an implementation in  the next release of
> Jetty.
>
>
> - output of deflate-stream are basically bigger than
> > deflate-application-data since Z_SYNC_FLUSH wastes 4 octets for each
> flush
> > operation
>
> I think you mean deflate-application-data frames are bigger....


yes... bad typo


>  > - for short messages, it just increases size because of ~2 octets for
> two
> > block headers and end of block symbol
>
> Does this not suggest that we should use a reserve bit to indicate if
> the frame is compressed or not?  This will allow uncompressed frames
> (either because of small size or incompressible content) to be sent.
>

Yeah. Taking 1 RSV bit, we can avoid inflation of the size for this kind of
content.

The problem is, as you and John were discussing once, how to give a good
criteria to determine if a certain content of a frame is worth compress or
not. Even if frame N is incompressible, it may be referred to deflate frame
N+1. If we don't get the frame N go through compressor, N+1 cannot refer
it. So, criteria like |compressed| < |original| don't work well. Criteria
like |original| < (some pre-specified size) are also not easy to
define. It's almost clear that for communicating 1 byte key type event,
deflate shouldn't be applied. Thinking of two byte, three byte, ... it's not
trivial. I think we cannot gather real data to define such criteria
sufficient to be put in the spec soon.

Servers can get some advice about this from application level, but user
agents don't have any way. May we ask API side to add some interface to
allow JavaScript to give some hint? Something like

attribute boolean compressOutgoing;

or, more generally

attribute DOMString extensionConfig;

to dynamically instruct the user agent.

It's also an option that we introduce COMP bit in wire spec but initially
have no API exposed for JavaScript to control that.

----

Based on granularity choices are grouped like:

- Connection-wise control: deflate-application-data or no compression
extension
- Direction-wise control: need to have two separate extension
deflate-application-data-server-to-client
deflate-application-data-client-to-server
- Message-wise control: need to have API to on/off compression

Thanks,
Takeshi