Re: Choosing a header compression algorithm

James M Snell <jasnell@gmail.com> Thu, 21 March 2013 19:51 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C070921F8C9D for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 21 Mar 2013 12:51:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.331
X-Spam-Level:
X-Spam-Status: No, score=-9.331 tagged_above=-999 required=5 tests=[AWL=1.267, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hynej7Pzzvrk for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 21 Mar 2013 12:51:58 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id BFBD521F853A for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 21 Mar 2013 12:51:58 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1UIlWA-0001hF-QW for ietf-http-wg-dist@listhub.w3.org; Thu, 21 Mar 2013 19:51:18 +0000
Resent-Date: Thu, 21 Mar 2013 19:51:18 +0000
Resent-Message-Id: <E1UIlWA-0001hF-QW@frink.w3.org>
Received: from lisa.w3.org ([128.30.52.41]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <jasnell@gmail.com>) id 1UIlVv-0001e8-BK for ietf-http-wg@listhub.w3.org; Thu, 21 Mar 2013 19:51:03 +0000
Received: from mail-oa0-f46.google.com ([209.85.219.46]) by lisa.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <jasnell@gmail.com>) id 1UIlVt-000255-Nv for ietf-http-wg@w3.org; Thu, 21 Mar 2013 19:51:03 +0000
Received: by mail-oa0-f46.google.com with SMTP id k1so3549817oag.19 for <ietf-http-wg@w3.org>; Thu, 21 Mar 2013 12:50:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=vzHEiVUMNm3lShGDrDR6ofCZMgHsZ60Ul9TH8hLJals=; b=RGSjQs5hxDfkp4n4GAkr17tFIoSYx8Iwg+CxUHDM1WsONIVO+4U/SQtXbkQTcerclD vlx3WCus2nFIscaZgnBicn9F65/R6syH/8nYbmhulo4nGqBaDym1yIQ2iYQJj8edYZE/ uR6sJ+nPvHiK4mY4KpSlJHD2KrdpzFbda49n6iRoWEocIdznVM0qlxnKlLQx9ArMGuaC ZHlL45VRfWKKfTapRc1muCQwFgHmP0N5QBRH+ncQs2z7ZLepLUQsotowAbayWDGnXPZo UdTIASuN2Ee3oY81E+PkmagzNI+jfQ1WaVBaWeacN9CPVvnQzOFOUh5EzI8TYhwU3VXX VKyA==
MIME-Version: 1.0
X-Received: by 10.60.22.34 with SMTP id a2mr7627886oef.97.1363895435802; Thu, 21 Mar 2013 12:50:35 -0700 (PDT)
Received: by 10.60.23.193 with HTTP; Thu, 21 Mar 2013 12:50:35 -0700 (PDT)
Received: by 10.60.23.193 with HTTP; Thu, 21 Mar 2013 12:50:35 -0700 (PDT)
In-Reply-To: <514B29FE.6010501@cs.tcd.ie>
References: <254AABEE-22B9-418E-81B0-2729902C4413@mnot.net> <514B29FE.6010501@cs.tcd.ie>
Date: Thu, 21 Mar 2013 12:50:35 -0700
Message-ID: <CABP7Rbcy_ur8T2z+WrHOD9-dELisN=KRxMJJxmwcrvkNqCr8ow@mail.gmail.com>
From: James M Snell <jasnell@gmail.com>
To: Stephen Farrell <stephen.farrell@cs.tcd.ie>
Cc: ietf-http-wg@w3.org, Mark Nottingham <mnot@mnot.net>
Content-Type: multipart/alternative; boundary="e89a8fb205e4f6424804d874a60e"
Received-SPF: pass client-ip=209.85.219.46; envelope-from=jasnell@gmail.com; helo=mail-oa0-f46.google.com
X-W3C-Hub-Spam-Status: No, score=-3.5
X-W3C-Hub-Spam-Report: AWL=-2.686, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001
X-W3C-Scan-Sig: lisa.w3.org 1UIlVt-000255-Nv 9f3b1abbed5574f428d4abd32d4e9c6a
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Choosing a header compression algorithm
Archived-At: <http://www.w3.org/mid/CABP7Rbcy_ur8T2z+WrHOD9-dELisN=KRxMJJxmwcrvkNqCr8ow@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/17102
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

On Mar 21, 2013 8:42 AM, "Stephen Farrell" <stephen.farrell@cs.tcd.ie>
wrote:
>
[snip]
>
> The kind of canonicalisation requirement might be that
> you need to ensure one can define a reasonable c14n
> function such that c14n(X)=c14n(uncompress(compress(X)).
>

I think this is going to be the best we can do really. There is no way the
compression algorithms discussed thus far can preserve c14n. At best,
you'll have to reapply c14n after decompression and reconstitution of the
header set.

- james

> Mark's item 3 below triggered this, I guess one could
> argue that it might be a requirement for compression
> that it not break higher level canonicalisation which
> isn't quite the same as being able to reconstitute the
> semantics. (For example, with timestamps that specify
> a zero TZ offset, or list ordering maybe.)
>
> Ta,
> S.
>
> PS: Apologies if this is all obvious when one reads
> the algorithm descriptions, which I've not;-)
>
>
> On 03/21/2013 07:11 AM, Mark Nottingham wrote:
> > Previously, we've talked about starting with just a delta-encoding
approach for our first implementation draft. In Orlando, we focused
primarily on two proposals:
> >
> > * Delta2
> >   Draft:
http://tools.ietf.org/html/draft-rpeon-httpbis-header-compression-03
> >   Python Implementation:
https://github.com/http2/compression-test/tree/master/compressor/delta2
> >
> > * HeaderDiff
> >   Draft: http://tools.ietf.org/html/draft-ruellan-headerdiff-00
> >   Python Implementation:
https://github.com/http2/compression-test/tree/master/compressor/headerdiff
> >
> > As I understand it, Herve et al want to work on making HeaderDiff more
resistant to CRIME, and hopefully we'll see the results of that in the very
near future.
> >
> > In the meantime, I'd like everyone to become familiar with both drafts
and the characteristics of their implementations, so that we can have an
informed discussion of them.
> >
> > I'd like to see a few things happen while we do this:
> >
> > 1) We need to do apples-to-apples comparison of these compressors to
see how they behave under a range of constraints (especially, memory).
> >
> > 2) I'd like us to verify that they are respecting those constraints,
and that they're implemented in an equivalent way (this is likely to be
manual).
> >
> > 3) It would be very good to have a test suite that verifies that they
correctly reconstitute the semantically significant parts of the headers;
in particular, large/unusual values, ordering where appropriate, etc. Our
current header corpus undoubtedly has holes in this regard.
> >
> > If you make any progress along these lines (dare I ask for
volunteers?), pleas share with the list.
> >
> > Looking at our issues list, this is one of the major items preventing
us from getting to a first implementation draft, so I'd like to chose a way
forward soon -- especially since we're choosing a starting point, and the
approach we take can evolve or, if necessary, be replaced.
> >
> > Regards,
> >
> > --
> > Mark Nottingham   http://www.mnot.net/
> >
> >
> >
> >
> >
>