Re: Header Stats

James M Snell <jasnell@gmail.com> Wed, 23 January 2013 04:36 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9912F21F87E7 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Tue, 22 Jan 2013 20:36:27 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.948
X-Spam-Level:
X-Spam-Status: No, score=-9.948 tagged_above=-999 required=5 tests=[AWL=0.650, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gdUjRyhweLpH for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Tue, 22 Jan 2013 20:36:26 -0800 (PST)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id A9C8421F87B6 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Tue, 22 Jan 2013 20:36:26 -0800 (PST)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1Txs2w-00033Z-0X for ietf-http-wg-dist@listhub.w3.org; Wed, 23 Jan 2013 04:34:46 +0000
Resent-Date: Wed, 23 Jan 2013 04:34:46 +0000
Resent-Message-Id: <E1Txs2w-00033Z-0X@frink.w3.org>
Received: from lisa.w3.org ([128.30.52.41]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <jasnell@gmail.com>) id 1Txs2o-00032p-TZ for ietf-http-wg@listhub.w3.org; Wed, 23 Jan 2013 04:34:38 +0000
Received: from mail-ie0-f170.google.com ([209.85.223.170]) by lisa.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <jasnell@gmail.com>) id 1Txs2n-00046Q-7x for ietf-http-wg@w3.org; Wed, 23 Jan 2013 04:34:38 +0000
Received: by mail-ie0-f170.google.com with SMTP id k10so13258042iea.29 for <ietf-http-wg@w3.org>; Tue, 22 Jan 2013 20:34:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:content-type; bh=HUP9XjZbL0ECYzCA8aOi6CggyLd4AgmklOgaD/IvoMk=; b=wbJXixkJS5vIYUkqYM31XjehR63ILabnKJKKi4yzjUuijyky8Hyw6OyR/MkL9MXxEq mQahf6whxhMZQNgByN/Oo9P5x11Nm4ugrRO8ebqnonxa0VqkkHWOcmj2ZZIwwNHrE/Ow AupO7HAf82p1fv7XD2OLdw6STgyhV0BiUnZkdMEvkkoadEqKZMbANpfUwNHYXtEnXkIl 1mB1d6af1fLEyTZDRO4SjBCduAUNagqCJzfICBv2hkjQTHESiGKu4hG+MOVglyalYscR 7DgOqP6qYrNBOOr8Yrvz57hFoVqTQiBoTuKo7aAFRLGwX85bJlFG3ac6UbKym/0BQMC0 AjLQ==
X-Received: by 10.50.196.227 with SMTP id ip3mr13547678igc.97.1358915651280; Tue, 22 Jan 2013 20:34:11 -0800 (PST)
MIME-Version: 1.0
Received: by 10.64.26.137 with HTTP; Tue, 22 Jan 2013 20:33:50 -0800 (PST)
In-Reply-To: <CABP7RbeHyWOPb=iDA+oPYB6QP28Ctq4qZaf2ThApinp1aRZRaA@mail.gmail.com>
References: <CABP7RbeHyWOPb=iDA+oPYB6QP28Ctq4qZaf2ThApinp1aRZRaA@mail.gmail.com>
From: James M Snell <jasnell@gmail.com>
Date: Tue, 22 Jan 2013 20:33:50 -0800
Message-ID: <CABP7RbdYsjQELcnZseCz8x_uhqyoFcd1kQaAyA2VHvdBjZ_QKA@mail.gmail.com>
To: "ietf-http-wg@w3.org" <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary="14dae934117baca0b104d3ed3457"
Received-SPF: pass client-ip=209.85.223.170; envelope-from=jasnell@gmail.com; helo=mail-ie0-f170.google.com
X-W3C-Hub-Spam-Status: No, score=-3.5
X-W3C-Hub-Spam-Report: AWL=-2.710, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001
X-W3C-Scan-Sig: lisa.w3.org 1Txs2n-00046Q-7x d59788de145b59146afa4d61a1d3bdc6
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Header Stats
Archived-At: <http://www.w3.org/mid/CABP7RbdYsjQELcnZseCz8x_uhqyoFcd1kQaAyA2VHvdBjZ_QKA@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/16125
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

Ok.. I have updated the calculations to show:

  1. General variability of header values. The lower the number, the more
redundant the value tends to be.
  2. A frequency distribution of specific values per header. This is rather
verbose but extremely informative
  3. For date and numeric header values, comparison values between the
text-value and optimized binary encoding value.
  4. A summation of the total bytes saved by using the optimized binary
encoding for dates and numeric headers.

TODO:

  - Implement experimental Set-Cookie, Cookie and Cache-Control headers to
see the difference for binary encoding
  - Properly handle null-separated value lists

If there are other interesting calculations you'd like to see, let me
know...

The updated output is here:
https://github.com/jasnell/compression-test/tree/master/counts


On Tue, Jan 22, 2013 at 1:24 PM, James M Snell <jasnell@gmail.com> wrote:

> I've started working on generating stats for individual headers within
> messages. Rather than take up too much space here on the list for the
> results, I am keeping the results in my github fork [1] of the
> compression-test code and will be posting summaries of the results
> periodically on my personal blog [2]. I will be putting together a summary
> of my findings together in time for the interim meeting next week.
> Unfortunately, however, I will not be able to attend the meeting.
>
> [1] https://github.com/jasnell/compression-test/blob/master/counts/
> [2] http://chmod777self.blogspot.com/2013/01/http-20-header-stats.html
>
> - James
>