Re: [hybi] deflate-stream and masking

"Andy Green (林安廸)" <> Mon, 20 June 2011 08:33 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 55BD111E80D6 for <>; Mon, 20 Jun 2011 01:33:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, MIME_8BIT_HEADER=0.3]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id IHovAC0NlXFK for <>; Mon, 20 Jun 2011 01:33:49 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id CA39111E8080 for <>; Mon, 20 Jun 2011 01:33:48 -0700 (PDT)
Message-ID: <>
Date: Mon, 20 Jun 2011 09:33:47 +0100
From: =?UTF-8?B?IkFuZHkgR3JlZW4gKOael+WuieW7uCki?= <>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20110531 Fedora/3.1.10-2.fc16 Thunderbird/3.1.10
MIME-Version: 1.0
To: Greg Wilkins <>
References: <> <> <>
In-Reply-To: <>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Cc: Hybi <>
Subject: Re: [hybi] deflate-stream and masking
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Server-Initiated HTTP <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 20 Jun 2011 08:33:49 -0000

On 06/20/2011 09:21 AM, Somebody in the thread at some point said:
> On 20 June 2011 18:11, "Andy Green (林安廸)"<>  wrote:
>> On 06/20/2011 07:33 AM, Somebody in the thread at some point said:
>> Hi -
>>> As an unmasked WS stream, it was 50675 bytes, and as a masked stream
>>> is was 52623 bytes.
>>> I then compressed both these streams with gzip and got 13306 bytes for
>>> unmasked and 51704 bytes for the masked!!!!
>>> So for this very typical example, masking was sufficiently random to
>>> completely negate the benefits of compression.
>> Isn't this just saying that it's dumb to mask-then-compress?
>> You could just compress-then-mask and get the 13Kbyte result directly and
>> "safely".
> That is exactly what I'm saying!
> We should use a deflate-frame extension that does compress-then-mask,
> rather than a deflate-stream "extension" that does mask-then-compress.

Actually I have to agree with you although I started out intending to 
just let it lie.

For good reasons we changed masking to be payload-only, and added an "I 
am masked" bit, and subordinated the mask to after the header.  All good 
stuff.  So now masking action is strongly defined inside a frame context 
and won't detach easily to be applied to deflate-stream.

Then to fix this the choices are either:

  1) add a feature to deflate-stream that it is always masked using a 
new mask scheme altogether and defeat frame masking if we know the thing 
has deflate-stream on it

  2) just operate deflate-stream on payloads before any per-frame 
masking gets applied

1) sounds like re-inventing the scary masking wheel.

The only little issue with 2) is that deflate-stream operates at bit 
level, you'll have to spill the bits up to a byte every frame payload.

So unless I missed something somewhere, to fix deflate-stream masking 
bloat it does seem to want delfate-stream action to move to only 
applying on payloads, and before any masking is done there.