Re: bohe implementation for compression tests
Roberto Peon <grmocg@gmail.com> Thu, 10 January 2013 22:25 UTC
Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 586CC21F8514 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 10 Jan 2013 14:25:46 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.615
X-Spam-Level:
X-Spam-Status: No, score=-9.615 tagged_above=-999 required=5 tests=[AWL=0.983, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pand68gPilZZ for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 10 Jan 2013 14:25:45 -0800 (PST)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id 143EB21F8510 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 10 Jan 2013 14:25:44 -0800 (PST)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1TtQXb-0000gL-Ro for ietf-http-wg-dist@listhub.w3.org; Thu, 10 Jan 2013 22:24:03 +0000
Resent-Date: Thu, 10 Jan 2013 22:24:03 +0000
Resent-Message-Id: <E1TtQXb-0000gL-Ro@frink.w3.org>
Received: from maggie.w3.org ([128.30.52.39]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <grmocg@gmail.com>) id 1TtQXV-0000fb-HT for ietf-http-wg@listhub.w3.org; Thu, 10 Jan 2013 22:23:57 +0000
Received: from mail-la0-f42.google.com ([209.85.215.42]) by maggie.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <grmocg@gmail.com>) id 1TtQXT-0001ew-9P for ietf-http-wg@w3.org; Thu, 10 Jan 2013 22:23:57 +0000
Received: by mail-la0-f42.google.com with SMTP id fe20so1181213lab.15 for <ietf-http-wg@w3.org>; Thu, 10 Jan 2013 14:23:28 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=E867z9aIR0USTBPD3L51ygi4As0UN2agcwRiowVl1Mo=; b=RcAWOd4KCOR9+3bTjOWM92Ef72//qKtqgjdvX5ESs62WZboXw+lN9Hw82PUueiLvRG ZbGnl45j0nJ3Asw+T4YYCgJC5qs4hzhjcAUzdLmfMGG43I4TDqZdnvQeNUyNEBuvYSOt Sk8buuGO43268mhAoI7QIwp6S/m7WQOF4k8+7ufxBusQCg2wjlFrWWm1y6cmqFiij/sG FROrZKIefbpgFu6C+GiW48qyeLhoctyp5Xmi982OZINzVbPchm2K+1YWw6tz84NM+Wde ZnEmaivvALuzvlMjWaHhhGOucHs9HeDUBlkbVp7gDj16BikPwf+sGYpbppnGrVrfvE18 6p/w==
MIME-Version: 1.0
Received: by 10.112.11.33 with SMTP id n1mr30850221lbb.18.1357856608660; Thu, 10 Jan 2013 14:23:28 -0800 (PST)
Received: by 10.112.81.5 with HTTP; Thu, 10 Jan 2013 14:23:28 -0800 (PST)
In-Reply-To: <CABP7Rbe-B89vVm8=OnHtAG0Y3G2UOysX+DKaTQ3+rAKBJBJyKA@mail.gmail.com>
References: <CABP7Rbe-B89vVm8=OnHtAG0Y3G2UOysX+DKaTQ3+rAKBJBJyKA@mail.gmail.com>
Date: Thu, 10 Jan 2013 14:23:28 -0800
Message-ID: <CAP+FsNeV-U+xTzTk1tKFF8Z1EkJRPkFZU95J2=TE9sWXKNSQ0Q@mail.gmail.com>
From: Roberto Peon <grmocg@gmail.com>
To: James M Snell <jasnell@gmail.com>
Cc: "ietf-http-wg@w3.org" <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary="e0cb4efe2a30d0ad8004d2f6a0bc"
Received-SPF: pass client-ip=209.85.215.42; envelope-from=grmocg@gmail.com; helo=mail-la0-f42.google.com
X-W3C-Hub-Spam-Status: No, score=-3.5
X-W3C-Hub-Spam-Report: AWL=-2.690, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001
X-W3C-Scan-Sig: maggie.w3.org 1TtQXT-0001ew-9P 2b3bcb9840f7abfa45f35374e69ad35b
X-Original-To: ietf-http-wg@w3.org
Subject: Re: bohe implementation for compression tests
Archived-At: <http://www.w3.org/mid/CAP+FsNeV-U+xTzTk1tKFF8Z1EkJRPkFZU95J2=TE9sWXKNSQ0Q@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/15829
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
Very cool! When I wrote the original form of the test framework, I used gzip'd "raw" HTTP as a benchmark, but only that since any stream-based compressor is unsafe in the presence of CRIME and similar attacks. Also, the way the encoding works for the delta-compressor if we added an ephemereal "clone" statement, it'd have roughly the same semantics as BOHE, where the 'default dictionary' defines the pre-agreed-upon codes. Would you like me to add opcode type and add an option to delta so you can play with that too? -=R On Thu, Jan 10, 2013 at 11:08 AM, James M Snell <jasnell@gmail.com> wrote: > I have an initial bohe implementation for the compression tests... it's > very preliminary and uses the same gzip compression as the current spdy3. > I'm going to be playing around with the delta compression mechanism as well > and see how much of an impact that has. Initial results are very promising > but I haven't done much debugging yet. Just wanted folks to know that this > work was underway... > > https://github.com/jasnell/compression-test/tree/master/compressor/bohe > > Some test runs.... > > ./compare_compressors.py -c bohe -c spdy3 -c delta > ../http_samples/mnot/amazon.com.har > 732 req messages processed > compressed | ratio min max std > req bohe 26,122 | 0.13 0.04 0.70 0.08 > req delta 33,955 | 0.17 0.02 0.71 0.09 > req http1 195,386 | 1.00 1.00 1.00 0.00 > req spdy3 27,238 | 0.14 0.04 0.71 0.08 > > 732 res messages processed > compressed | ratio min max std > res bohe 39,628 | 0.25 0.04 0.66 0.07 > res delta 44,499 | 0.28 0.02 0.65 0.09 > res http1 159,968 | 1.00 1.00 1.00 0.00 > res spdy3 41,325 | 0.26 0.04 0.67 0.08 > > > ./compare_compressors.py -c bohe -c spdy3 -c delta > ../http_samples/mnot/craigslist.org.har > 66 req messages processed > compressed | ratio min max std > req bohe 1,948 | 0.15 0.06 0.73 0.11 > req delta 2,036 | 0.16 0.07 0.71 0.11 > req http1 12,894 | 1.00 1.00 1.00 0.00 > req spdy3 2,016 | 0.16 0.07 0.75 0.11 > > 66 res messages processed > compressed | ratio min max std > res bohe 1,786 | 0.18 0.07 0.77 0.13 > res delta 2,858 | 0.28 0.08 0.69 0.12 > res http1 10,147 | 1.00 1.00 1.00 0.00 > res spdy3 1,869 | 0.18 0.09 0.78 0.13 > > > ./compare_compressors.py -c bohe -c spdy3 -c delta > ../http_samples/mnot/flickr.com.har > 438 req messages processed > compressed | ratio min max std > req bohe 11,988 | 0.10 0.02 0.69 0.07 > req delta 26,372 | 0.22 0.01 0.71 0.14 > req http1 121,854 | 1.00 1.00 1.00 0.00 > req spdy3 12,550 | 0.10 0.02 0.71 0.07 > > 438 res messages processed > compressed | ratio min max std > res bohe 13,073 | 0.09 0.05 0.66 0.06 > res delta 25,236 | 0.18 0.02 0.70 0.11 > res http1 140,457 | 1.00 1.00 1.00 0.00 > res spdy3 14,142 | 0.10 0.05 0.66 0.06 > > > ./compare_compressors.py -c bohe -c spdy3 -c delta > ../http_samples/mnot/facebook.com.har > 234 req messages processed > compressed | ratio min max std > req bohe 6,091 | 0.15 0.06 0.78 0.07 > req delta 7,800 | 0.19 0.02 0.70 0.07 > req http1 41,980 | 1.00 1.00 1.00 0.00 > req spdy3 6,301 | 0.15 0.06 0.77 0.07 > > 234 res messages processed > compressed | ratio min max std > res bohe 9,458 | 0.23 0.07 0.68 0.07 > res delta 12,045 | 0.30 0.13 0.60 0.08 > res http1 40,252 | 1.00 1.00 1.00 0.00 > res spdy3 9,788 | 0.24 0.07 0.69 0.07 > > > > >
- bohe implementation for compression tests James M Snell
- Re: bohe implementation for compression tests Roberto Peon
- Re: bohe implementation for compression tests James M Snell
- Re: bohe implementation for compression tests James M Snell
- Re: bohe implementation for compression tests James M Snell
- Re: bohe implementation for compression tests Roberto Peon
- Re: bohe implementation for compression tests James M Snell
- Re: bohe implementation for compression tests Roberto Peon
- Re: bohe implementation for compression tests James M Snell
- Re: bohe implementation for compression tests Martin Thomson
- Re: bohe implementation for compression tests Roberto Peon
- Re: bohe implementation for compression tests Mark Nottingham
- Re: bohe implementation for compression tests Martin Thomson
- Re: bohe implementation for compression tests Roberto Peon
- Re: bohe implementation for compression tests Mark Nottingham
- Re: bohe implementation for compression tests Mark Nottingham