Re: [hybi] deflate-stream and masking

"Andy Green (林安廸)" <andy@warmcat.com> Mon, 20 June 2011 08:33 UTC

Return-Path: <andy@warmcat.com>
X-Original-To: hybi@ietfa.amsl.com
Delivered-To: hybi@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 55BD111E80D6 for <hybi@ietfa.amsl.com>; Mon, 20 Jun 2011 01:33:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, MIME_8BIT_HEADER=0.3]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IHovAC0NlXFK for <hybi@ietfa.amsl.com>; Mon, 20 Jun 2011 01:33:49 -0700 (PDT)
Received: from warmcat.com (warmcat.com [87.106.134.80]) by ietfa.amsl.com (Postfix) with ESMTP id CA39111E8080 for <hybi@ietf.org>; Mon, 20 Jun 2011 01:33:48 -0700 (PDT)
Message-ID: <4DFF05EB.4070201@warmcat.com>
Date: Mon, 20 Jun 2011 09:33:47 +0100
From: =?UTF-8?B?IkFuZHkgR3JlZW4gKOael+WuieW7uCki?= <andy@warmcat.com>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110531 Fedora/3.1.10-2.fc16 Thunderbird/3.1.10
MIME-Version: 1.0
To: Greg Wilkins <gregw@intalio.com>
References: <BANLkTi=UVMAd1nER6mRBe7zoD29CSbCkGA@mail.gmail.com> <4DFF00C1.8090809@warmcat.com> <BANLkTinN=ae0v0D8F78stCWRNvrLL3xYfQ@mail.gmail.com>
In-Reply-To: <BANLkTinN=ae0v0D8F78stCWRNvrLL3xYfQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Cc: Hybi <hybi@ietf.org>
Subject: Re: [hybi] deflate-stream and masking
X-BeenThere: hybi@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Server-Initiated HTTP <hybi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/hybi>, <mailto:hybi-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/hybi>
List-Post: <mailto:hybi@ietf.org>
List-Help: <mailto:hybi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Jun 2011 08:33:49 -0000

On 06/20/2011 09:21 AM, Somebody in the thread at some point said:
> On 20 June 2011 18:11, "Andy Green (林安廸)"<andy@warmcat.com>  wrote:
>> On 06/20/2011 07:33 AM, Somebody in the thread at some point said:
>>
>> Hi -
>>
>>> As an unmasked WS stream, it was 50675 bytes, and as a masked stream
>>> is was 52623 bytes.
>>> I then compressed both these streams with gzip and got 13306 bytes for
>>> unmasked and 51704 bytes for the masked!!!!
>>>
>>> So for this very typical example, masking was sufficiently random to
>>> completely negate the benefits of compression.
>>
>> Isn't this just saying that it's dumb to mask-then-compress?
>>
>> You could just compress-then-mask and get the 13Kbyte result directly and
>> "safely".
>
> That is exactly what I'm saying!
>
> We should use a deflate-frame extension that does compress-then-mask,
> rather than a deflate-stream "extension" that does mask-then-compress.

Actually I have to agree with you although I started out intending to 
just let it lie.

For good reasons we changed masking to be payload-only, and added an "I 
am masked" bit, and subordinated the mask to after the header.  All good 
stuff.  So now masking action is strongly defined inside a frame context 
and won't detach easily to be applied to deflate-stream.

Then to fix this the choices are either:

  1) add a feature to deflate-stream that it is always masked using a 
new mask scheme altogether and defeat frame masking if we know the thing 
has deflate-stream on it

  2) just operate deflate-stream on payloads before any per-frame 
masking gets applied

1) sounds like re-inventing the scary masking wheel.

The only little issue with 2) is that deflate-stream operates at bit 
level, you'll have to spill the bits up to a byte every frame payload.

So unless I missed something somewhere, to fix deflate-stream masking 
bloat it does seem to want delfate-stream action to move to only 
applying on payloads, and before any masking is done there.

-Andy