Re: bohe implementation for compression tests

Mark Nottingham <mnot@mnot.net> Fri, 18 January 2013 06:54 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D09EB21F8A52 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 17 Jan 2013 22:54:38 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.148
X-Spam-Level:
X-Spam-Status: No, score=-9.148 tagged_above=-999 required=5 tests=[AWL=1.451, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JbKb1xg75Cyx for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 17 Jan 2013 22:54:38 -0800 (PST)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id 1FB3B21F8A49 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 17 Jan 2013 22:54:37 -0800 (PST)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1Tw5pT-00043L-CK for ietf-http-wg-dist@listhub.w3.org; Fri, 18 Jan 2013 06:53:31 +0000
Resent-Date: Fri, 18 Jan 2013 06:53:31 +0000
Resent-Message-Id: <E1Tw5pT-00043L-CK@frink.w3.org>
Received: from maggie.w3.org ([128.30.52.39]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <mnot@mnot.net>) id 1Tw5pP-00042Q-MS for ietf-http-wg@listhub.w3.org; Fri, 18 Jan 2013 06:53:27 +0000
Received: from mxout-07.mxes.net ([216.86.168.182]) by maggie.w3.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from <mnot@mnot.net>) id 1Tw5pO-00078I-Js for ietf-http-wg@w3.org; Fri, 18 Jan 2013 06:53:27 +0000
Received: from [192.168.1.80] (unknown [118.209.240.13]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTPSA id 69D7822E256; Fri, 18 Jan 2013 01:53:03 -0500 (EST)
Content-Type: text/plain; charset="us-ascii"
Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\))
From: Mark Nottingham <mnot@mnot.net>
In-Reply-To: <CABkgnnUrs19_73Xc=E+TAhnP6AEdmuKQ5-XmDez+mJg-MONZfA@mail.gmail.com>
Date: Fri, 18 Jan 2013 17:52:59 +1100
Cc: James M Snell <jasnell@gmail.com>, "ietf-http-wg@w3.org" <ietf-http-wg@w3.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <90D04E0D-A284-4CB4-B5DF-3B7CCA0F262D@mnot.net>
References: <CABP7Rbe-B89vVm8=OnHtAG0Y3G2UOysX+DKaTQ3+rAKBJBJyKA@mail.gmail.com> <CABP7RbdSurNdLxvVV1F-9Bx4LwdhrX2btEvfA+RV=rVZ0moXNA@mail.gmail.com> <CABkgnnVdXwo5eiBMNMjONG88r8EvErxQ9_yrgdh5NFH=KQhfZQ@mail.gmail.com> <B8C9ED58-31E8-4DF0-9AE0-34E07B95F0A8@mnot.net> <CABkgnnUrs19_73Xc=E+TAhnP6AEdmuKQ5-XmDez+mJg-MONZfA@mail.gmail.com>
To: Martin Thomson <martin.thomson@gmail.com>
X-Mailer: Apple Mail (2.1499)
Received-SPF: pass client-ip=216.86.168.182; envelope-from=mnot@mnot.net; helo=mxout-07.mxes.net
X-W3C-Hub-Spam-Status: No, score=-3.3
X-W3C-Hub-Spam-Report: AWL=-3.281, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001
X-W3C-Scan-Sig: maggie.w3.org 1Tw5pO-00078I-Js cb7cd1c9322c704983a13f261655d6c7
X-Original-To: ietf-http-wg@w3.org
Subject: Re: bohe implementation for compression tests
Archived-At: <http://www.w3.org/mid/90D04E0D-A284-4CB4-B5DF-3B7CCA0F262D@mnot.net>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/15984
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

OK, I've started to record CPU time in my refactor branch:

* TOTAL: 1012 req messages
                  size  time | ratio min   max   std
   http1       830,970  0.05 | 1.00  1.00  1.00  0.00
  simple       320,883  0.05 | 0.39  0.07  0.92  0.24
   spdy3        85,492  0.06 | 0.10  0.03  0.66  0.08

* TOTAL: 1012 res messages
                  size  time | ratio min   max   std
   http1       424,075  0.04 | 1.00  1.00  1.00  0.00
  simple       176,216  0.12 | 0.42  0.11  0.95  0.12
   spdy3        80,706  0.07 | 0.19  0.04  0.68  0.09


https://github.com/http2/compression-test/tree/stream-sep


On 17/01/2013, at 12:21 PM, Martin Thomson <martin.thomson@gmail.com> wrote:

> On 16 January 2013 17:11, Mark Nottingham <mnot@mnot.net> wrote:
>> Getting there, although you may need a small truck to haul the grain of salt that will accompany it...
> 
> Even if it just means running the sample set n times using 'time', it
> would be nice to get ballpark figures.  Even if the errors are
> enormous.  For instance, a python implementation of delta is probably
> not that good to compare to an optimized gzip implementation, but it
> might still let us know that deltav3 is better than deltav2.
> 

--
Mark Nottingham   http://www.mnot.net/