Re: [hybi] deflate-stream and masking

["Andy Green (林安廸)" <[e-mail]>]

On 06/20/2011 09:21 AM, Somebody in the thread at some point said:
> On 20 June 2011 18:11, "Andy Green (林安廸)"<[e-mail]>  wrote:
>> On 06/20/2011 07:33 AM, Somebody in the thread at some point said:
>>
>> Hi -
>>
>>> As an unmasked WS stream, it was 50675 bytes, and as a masked stream
>>> is was 52623 bytes.
>>> I then compressed both these streams with gzip and got 13306 bytes for
>>> unmasked and 51704 bytes for the masked!!!!
>>>
>>> So for this very typical example, masking was sufficiently random to
>>> completely negate the benefits of compression.
>>
>> Isn't this just saying that it's dumb to mask-then-compress?
>>
>> You could just compress-then-mask and get the 13Kbyte result directly and
>> "safely".
>
> That is exactly what I'm saying!
>
> We should use a deflate-frame extension that does compress-then-mask,
> rather than a deflate-stream "extension" that does mask-then-compress.

Actually I have to agree with you although I started out intending to 
just let it lie.

For good reasons we changed masking to be payload-only, and added an "I 
am masked" bit, and subordinated the mask to after the header.  All good 
stuff.  So now masking action is strongly defined inside a frame context 
and won't detach easily to be applied to deflate-stream.

Then to fix this the choices are either:

  1) add a feature to deflate-stream that it is always masked using a 
new mask scheme altogether and defeat frame masking if we know the thing 
has deflate-stream on it

  2) just operate deflate-stream on payloads before any per-frame 
masking gets applied

1) sounds like re-inventing the scary masking wheel.

The only little issue with 2) is that deflate-stream operates at bit 
level, you'll have to spill the bits up to a byte every frame payload.

So unless I missed something somewhere, to fix deflate-stream masking 
bloat it does seem to want delfate-stream action to move to only 
applying on payloads, and before any masking is done there.

-Andy