Re: [hybi] Call for Consensus: draft-tyoshino-hybi-websocket-perframe-deflate as WG Item

[John Tamplin <[e-mail]>]

On Thu, Mar 1, 2012 at 7:58 PM, Brian <[e-mail]> wrote:

> I like option "d."  One framing byte of overhead is practically nothing
> and enables lots of opportunities for tuning and control.  If the byte of
> overhead is too much for some people, perhaps there could be a
> configuration parameter during the handshake to allow the user to opt into
> across-the-board compression settings that would eliminate the byte and
> just apply to every frame.  That would allow users with
> consistently compressible data to eliminate the one byte overhead.
>
> Alternatively, we could define a control frame to configure the
> compression extension that could turn the per-frame control byte on or off.
>

The options of knowing what to compress are:
1) compress everything on a connection, configured at handshake time.  We
know compressing incompressible data is
a loss, so we would need to expose a way in the JS API to indicate whether
to use compression or not.  Applications that mix compressible and
incompressible data would use multiple connections (made less costly via
multiplexing).

2) compress only compressible data in a frame, leaving incompressible data
in uncompressed frames.  The choice of how to determine compressibility:
  2a) the API says whether it is compressible
  2b) the API gives a type of data being supplied, and an implementation
chooses whether to compress it based on the type
  2c) heuristic choices, such as "compress all text frames, leave binary
frames uncompressed" (though earlier experiments with GwtQuake over
WebSockets showed that even its binary data substantially benefited from
compression)
  2d) try and compress the data, and send it compressed only if it reduces
the size

Another question is about sharing compression state -- if we compress each
frame individually, we lose the benefit of redundancy with other frames.
 For example, the WS binary frames in GwtQuake saw little improvement at
all if each frame was compressed individually, but there was a lot of
redundancy across frames.  To take advantage of this, we need a compression
scheme that maintains a compression dictionary for the stream (which might
complicated 2d above depending on the compression library available).

As Greg said, if we aren't going to use a reserved bit for per-frame
compression, where we hope to reduce the payload to a small number of bytes
already so an extra byte to indicate compression may be expensive, what
exactly will we use it for?

My opinion is we should allocate a reserved bit for per-frame compression,
create an extension to indicate that is to be used for compression and a
way to select the compression algorithm, and define deflate with no state
shared across frames as a required compression algorithm to be supported if
no other algorithm is negotiated.  We can then define more effective
compression algorithms that provide ways of sharing compression state
across frames, for example, though we still have to work out the above
questions though.

-- 
John A. Tamplin
Software Engineer (GWT), Google