Re: [hybi] deflate-stream and masking

"Bob Gezelter" <gezelter@rlgsc.com> Mon, 20 June 2011 14:09 UTC

Return-Path: <gezelter@rlgsc.com>
X-Original-To: hybi@ietfa.amsl.com
Delivered-To: hybi@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 38F4F11E8192 for <hybi@ietfa.amsl.com>; Mon, 20 Jun 2011 07:09:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.74
X-Spam-Level:
X-Spam-Status: No, score=-0.74 tagged_above=-999 required=5 tests=[BAYES_20=-0.74]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4So6X0XzIvwS for <hybi@ietfa.amsl.com>; Mon, 20 Jun 2011 07:09:23 -0700 (PDT)
Received: from smtpoutwbe03.prod.mesa1.secureserver.net (smtpoutwbe03.prod.mesa1.secureserver.net [208.109.78.114]) by ietfa.amsl.com (Postfix) with SMTP id 7467211E8191 for <hybi@ietf.org>; Mon, 20 Jun 2011 07:09:23 -0700 (PDT)
Received: (qmail 9534 invoked from network); 20 Jun 2011 14:09:18 -0000
Received: from unknown (HELO localhost) (72.167.218.135) by smtpoutwbe03.prod.mesa1.secureserver.net with SMTP; 20 Jun 2011 14:09:17 -0000
Received: (qmail 21046 invoked by uid 99); 20 Jun 2011 14:09:16 -0000
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="utf-8"
X-Originating-IP: 162.83.147.83
User-Agent: Web-Based Email 5.5.05
Message-Id: <20110620070915.ef1fc80126c74c6c202a919c41c7bb0b.f9a4892276.wbe@email03.secureserver.net>
From: "Bob Gezelter" <gezelter@rlgsc.com>
To: hybi@ietf.org
Date: Mon, 20 Jun 2011 07:09:15 -0700
Mime-Version: 1.0
Subject: Re: [hybi] deflate-stream and masking
X-BeenThere: hybi@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Server-Initiated HTTP <hybi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/hybi>, <mailto:hybi-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/hybi>
List-Post: <mailto:hybi@ietf.org>
List-Help: <mailto:hybi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Jun 2011 14:09:24 -0000

As I have noted before, Greg's observation about the interaction of
masking and compression is undoubtedly correct.

To be effective, compression must be done when the steam of data is most
strongly correlated. In short, this should be BEFORE multiplexing,
encryption, and masking. Data on different multiplexed streams cannot be
assumed to be correlated, and correlation is needed for effective
compression. 

Encrypting or masking data to create randomness similarly negates the
effectiveness of compression. 

When dealing with correlated data (e.g., the contents of a printable
file, or even a PostScript file), I have seen compression factors of
100:1. If the data had first been encrypted, or randomly masked, the
ratio would have been likely 2:1 (what is often achieved by disk or tape
compression facilities), or even less than 1:1 (worst case).

Compression should be done within each sub-stream (assuming
multiplexing, which is not in the current specification), and in any
event, before encryption and masking.

- Bob Gezelter, http://www.rlgsc.com