Re: [hybi] preliminary WebSockets compression experiments

[Roberto Peon <[e-mail]>]

Here is another thing to note.
If compression is optional and/or negotiated, it is likely that it will
either not be implemented, or it will be turned off by various end-user
devices that wish to inflate benchmark numbers.

SPDY requires compression support over the headers of certain control
frames. To date, we've not seen a case where this has resulted in a larger
request than raw, and it does provide for at least the availability of the
gzip functionality at the endpoint.

For Websockets, if we use gzip, the decision to do so should be made
*solely* by the sender of that data. There should probably be no
negotiation, else we'll end up with liars and cheats again.
-=R

On Fri, Apr 23, 2010 at 3:11 PM, John Tamplin <[e-mail]> wrote:

> On Fri, Apr 23, 2010 at 5:48 PM, Mike Belshe <[e-mail]> wrote:
>
>> *Memory Consumption:*
>> BTW - zlib, using the configuration you specified, might use a surprising
>> amount of RAM (to the tune of ~250KB!).  I suspect you want to decrease the
>> window size. Brian Olson already ran this experiment a while back with SPDY
>> and his results are here.  It would be interesting to see if you reach the
>> same conclusion.
>>
>> groups.google.com/group/spdy-dev/browse_thread/thread/dfaf498542fac792?pli=1
>>
>> As a result of Brian's work, SPDY decreased the windowBits from 15 to 11.
>>  This reduces the memory footprint substantially, while only leading to
>> modest reduction in compression ratios.
>>
>
> Thanks for the pointer -- I will try that out.
>
>
>> *On Generically Compressing A Stream:*
>> While we wanted to have SPDY provide required compression, in the end we
>> find it likely a bad idea.  Compressing the stream is never as good as
>> compressing the content.  For example, if the content can compress the
>> image, there is no point in having the stream redundantly (and likely
>> less-effectively) compress it.  Since most developers will get the content
>> compressed correctly, that is the right layer to do so; rather than having
>> the channel blindly compress what is already done.  We might revisit this
>> approach if new data arises, though.
>>
>> SSL has taken a similar approach.  When the protocol was designed, the
>> authors had the foresight to include compression negotiation in addition to
>> cypher negotiation as part of the handshake.  They did this because
>> compression should be applied before encryption (compressing encrypted data
>> is never good).  But, what do browsers today negotiate?  The major browsers
>> all advertise the empty-set of supported compression algorithms (meaning
>> don't do compression).  Why?  because compressing the stream blindly when
>> the application content is already compressed is inefficient.
>>
>> Anyway - those are two protocols which are opting not to do stream-based
>> compression and instead requiring the content provider to make the choice
>> (like HTTP does).  I think this matches your conclusion that you shouldn't
>> just compress everything.
>>
>
> I agree with the basic premise that the application can do a better job
> compressing the data than some generic algorithm -- my compression
> background is chess endgame databases where application-specific compression
> algorithms are much better than generic ones.  However, I don't know about
> trying to implement that in JS along with a corresponding server-side
> implementation.
>
> For HTTP, most servers are configured to compress things automatically if
> the client supports it, typically with a blacklist of file types not to
> compress (granted, you can pre-compress static content and serve that, and
> some servers may be intentionally configured to not compress other data).
>  Clearly using gzip compression on HTTP traffic is generally a good thing,
> and especially in the case of mobile or other low-bandwidth networks.
>
> As the GWT Quake example shows, even data that you wouldn't think is
> compressible (and isn't if you don't maintain compression state) compresses
> quite well.  With the safety valve of allowing uncompressed frames even when
> compression has been negotiated, I don't see how it can be a bad thing.  The
> server-side API might allow a way for the particular service on the other
> end of the socket to deny compression if it knows its data can't benefit,
> but that is really beyond the scope of the wire format.
>
> It would be feasible to provide an API for JS to call for compressing data
> and sending it itself, but there are a couple of problems:
>  - JS currently doesn't have any efficient way to manipulate binary data
>  - in the common case, it just adds extra work for the developer and extra
> code to download
>
> I think that might be a good idea once the binary data problem is solved,
> but I think a default case of allowing reasonable compression on the stream
> automatically is a good idea.
>
> --
> John A. Tamplin
> Software Engineer (GWT), Google
>
> _______________________________________________
> hybi mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/hybi
>
>