Re: Header compression: buffer management

Amos Jeffries <squid3@treenet.co.nz> Thu, 21 March 2013 15:13 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4548D21F9132 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 21 Mar 2013 08:13:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.449
X-Spam-Level:
X-Spam-Status: No, score=-10.449 tagged_above=-999 required=5 tests=[AWL=0.150, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tgKOGEbw5JCr for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 21 Mar 2013 08:13:35 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id 9109321F9130 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 21 Mar 2013 08:13:26 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1UIhAJ-0002Qv-3I for ietf-http-wg-dist@listhub.w3.org; Thu, 21 Mar 2013 15:12:27 +0000
Resent-Date: Thu, 21 Mar 2013 15:12:27 +0000
Resent-Message-Id: <E1UIhAJ-0002Qv-3I@frink.w3.org>
Received: from lisa.w3.org ([128.30.52.41]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <squid3@treenet.co.nz>) id 1UIhA8-0002Pg-2M for ietf-http-wg@listhub.w3.org; Thu, 21 Mar 2013 15:12:16 +0000
Received: from ip-58-28-153-233.static-xdsl.xnet.co.nz ([58.28.153.233] helo=treenet.co.nz) by lisa.w3.org with esmtp (Exim 4.72) (envelope-from <squid3@treenet.co.nz>) id 1UIhA3-0001gF-Ew for ietf-http-wg@w3.org; Thu, 21 Mar 2013 15:12:16 +0000
Received: from [192.168.2.7] (103-9-43-128.flip.co.nz [103.9.43.128]) by treenet.co.nz (Postfix) with ESMTP id CAAB1E6F20 for <ietf-http-wg@w3.org>; Fri, 22 Mar 2013 03:11:48 +1200 (NZST)
Message-ID: <514B2330.2030807@treenet.co.nz>
Date: Fri, 22 Mar 2013 04:11:44 +1300
From: Amos Jeffries <squid3@treenet.co.nz>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130307 Thunderbird/17.0.4
MIME-Version: 1.0
To: ietf-http-wg@w3.org
References: <6C71876BDCCD01488E70A2399529D5E5163F39C4@ADELE.crf.canon.fr>
In-Reply-To: <6C71876BDCCD01488E70A2399529D5E5163F39C4@ADELE.crf.canon.fr>
Content-Type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-Transfer-Encoding: 7bit
Received-SPF: pass client-ip=58.28.153.233; envelope-from=squid3@treenet.co.nz; helo=treenet.co.nz
X-W3C-Hub-Spam-Status: No, score=-1.7
X-W3C-Hub-Spam-Report: AWL=-1.725, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001
X-W3C-Scan-Sig: lisa.w3.org 1UIhA3-0001gF-Ew 256bf1ca409de8a246310c464bfcc5ec
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Header compression: buffer management
Archived-At: <http://www.w3.org/mid/514B2330.2030807@treenet.co.nz>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/17098
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

On 22/03/2013 1:50 a.m., RUELLAN Herve wrote:
> In HeaderDiff we chose to let the encoder decide how the buffer is managed (regarding additions and removals). These decisions are encoded on the wire and applied by the decoder on its own buffer. We think this choice has several advantages.

The encoder *will* at some point be a malicious attacker out to cause 
the decoder problems. Placing such an encoder in charge of buffer 
management at the decoder end of the wire is a sure-fire recipe for 
trouble. For anything like correct operation under such conditions it 
requires a pre-known buffer limit and enforcement of that limit at the 
decoder. Possibly with one size in the specs and a larger buffer size 
offered by the decoder for ongoing traffic (yes I can hear the cries 
about RTT lag already).


> First, this allows to have a very simple and lightweight decoder: it only needs to decode the decisions made by the encoder and apply them. It has no need to effectively implements a LRU mechanism for its buffer. This is especially important for small devices with limited CPU.
> In addition, we think it could be of interest for an intermediary, that could keep a partial buffer containing only the entries it is interested with, ignoring the other entries.

I notice that you are not making the same claim for encoder. All HTTP 
nodes will need both algorithms to follow the request+response model. 
How does the encoder+decoder as a pair stack up for size and complexity?


> Second, this allows to adapt the buffer management to the context. While LRU is a good algorithm in the general case for deciding which entry to remove from the buffer, it may not be the best one for every specific case. More complex algorithms can probably be devised to improve the compaction (for example taking also into account the frequency of occurrence of headers), or simpler algorithms could be used to reduce the CPU cost to the detriment of compaction (this is important for small devices).
> Adaptability is also important for the future of HTTP/2.0. If we do our job correctly, HTTP/2.0 will still be used in 10 or 20 years, but we have no idea how it will be used at that time. Making HTTP/2.0 adaptable to new usages is crucial to give it a long life expectancy.
> For these reasons, we prefer to keep all the buffer management on the encoder side, allowing an implementer to choose its preferred approach.

Good reasons.

Amos