Re: [hybi] Background info: Properties of sendfile()

Jack Moffitt <jack@collecta.com> Fri, 06 August 2010 13:49 UTC

Return-Path: <metajack@gmail.com>
X-Original-To: hybi@core3.amsl.com
Delivered-To: hybi@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 554F63A69C3 for <hybi@core3.amsl.com>; Fri, 6 Aug 2010 06:49:14 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.957
X-Spam-Level:
X-Spam-Status: No, score=-1.957 tagged_above=-999 required=5 tests=[AWL=0.020, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4W+XUqvQugiV for <hybi@core3.amsl.com>; Fri, 6 Aug 2010 06:49:12 -0700 (PDT)
Received: from mail-qy0-f172.google.com (mail-qy0-f172.google.com [209.85.216.172]) by core3.amsl.com (Postfix) with ESMTP id 66F603A67EE for <hybi@ietf.org>; Fri, 6 Aug 2010 06:49:11 -0700 (PDT)
Received: by qyk1 with SMTP id 1so4108630qyk.10 for <hybi@ietf.org>; Fri, 06 Aug 2010 06:49:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=ENG53VKIR4UZt9ZGBxkm7c/lwZMQxkW0hhx9+Jp/VOI=; b=cZcoKh3FPUbHQpnSEy2MNHVtXlI6KPCHqgFAm/FlbtOgCOJfh9xiy8I1L4KwieaEEl CiaC69XGKGYRVlwaHUGAzO/p+X6w/j7RKUFvOo302eDmSiDmrCxk8A16fz3u9UMn+Xr0 jQ4TEkatOx6JzSUJdhRU2EZV2BqZ1mRz7SY4I=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=CD3RDe1SSe/sqqxXnq1CeZRdiPfph4cljIPg41BizsyGdLIf8GSPnK8Vhy1pMl8Zyz Nzuh/yxighyYkHxQDveYWW/0cviQo+e6NDNFs79DjC0JU8/CfsNggdHxh6dteM8BfRfr MeL5mnfyPwJL9dHv3IMDcuxt4hXU5iLAfSSh0=
MIME-Version: 1.0
Received: by 10.229.232.209 with SMTP id jv17mr4766736qcb.63.1281102581957; Fri, 06 Aug 2010 06:49:41 -0700 (PDT)
Sender: metajack@gmail.com
Received: by 10.229.18.147 with HTTP; Fri, 6 Aug 2010 06:49:41 -0700 (PDT)
In-Reply-To: <20100806062928.GF20057@1wt.eu>
References: <4C5B1695.6070704@gmx.de> <F8E2F702-9F74-4316-B3B2-D5A731409ABF@apple.com> <4C5B2029.90403@gmx.de> <AANLkTim1WeCRfcPxXUNQcVhb4+t_TtDQDv2bXaxOQ=bk@mail.gmail.com> <01098AD0-FBF4-4A61-B565-947C95722BAA@apple.com> <AANLkTi=qQSND5BvUP5+P=wJ7E8SG6NncGZH8U8+VYwZ0@mail.gmail.com> <20100806004907.GF27827@shareable.org> <C0FC87B7-C51C-4B36-BC16-DBDB0B00A20F@gbiv.com> <20100806012845.GI27827@shareable.org> <AANLkTimuvuj87qwQi_1Gjg47-gGrCsDCQ5TDd5zvOw1N@mail.gmail.com> <20100806062928.GF20057@1wt.eu>
Date: Fri, 06 Aug 2010 07:49:41 -0600
X-Google-Sender-Auth: NZ-jDBrE7CfJf1DpLV42SMVl-fM
Message-ID: <AANLkTimUkx8_Ssiuh4vJsgiQb2xmUzr5uAb-TgG=bjyh@mail.gmail.com>
From: Jack Moffitt <jack@collecta.com>
To: Willy Tarreau <w@1wt.eu>
Content-Type: text/plain; charset="ISO-8859-1"
Cc: "Roy T. Fielding" <fielding@gbiv.com>, "hybi@ietf.org" <hybi@ietf.org>
Subject: Re: [hybi] Background info: Properties of sendfile()
X-BeenThere: hybi@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Server-Initiated HTTP <hybi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/hybi>
List-Post: <mailto:hybi@ietf.org>
List-Help: <mailto:hybi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Aug 2010 13:49:14 -0000

> At least I've seen it. Haproxy happily forwards traffic at 10 Gbps
> with only 25% of one CPU core on a 2.66 GHz Core2Duo using splice().
> And some file sharing sites make a big use of it ! Without splice,
> it hardly achieves 10 Gbps at 100% CPU.

It's not clear how much of the benefit we'd lose with fixed size
lengths (and therefore mandatory chunking). At this point, it seems
that kind of data would be valuable.

The tradeoff *seems* to be simple intermediaries versus
sendfile()/splice() efficiency. If sendfile() is severely affected, it
may make sense to make the intermediaries life slightly more
complicated. If not, I think we keep their job simple.

jack.