Re: [hybi] frame length encoding

Patrick McManus <mcmanus@ducksong.com> Sat, 21 August 2010 23:57 UTC

Return-Path: <mcmanus@ducksong.com>
X-Original-To: hybi@core3.amsl.com
Delivered-To: hybi@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D067D3A6832 for <hybi@core3.amsl.com>; Sat, 21 Aug 2010 16:57:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.503
X-Spam-Level:
X-Spam-Status: No, score=-2.503 tagged_above=-999 required=5 tests=[AWL=0.096, BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ucLQ4X7TpHSC for <hybi@core3.amsl.com>; Sat, 21 Aug 2010 16:57:43 -0700 (PDT)
Received: from linode.ducksong.com (linode.ducksong.com [64.22.125.164]) by core3.amsl.com (Postfix) with ESMTP id E6DFA3A680B for <hybi@ietf.org>; Sat, 21 Aug 2010 16:57:42 -0700 (PDT)
Received: by linode.ducksong.com (Postfix, from userid 1000) id 7EF23102A9; Sat, 21 Aug 2010 19:58:16 -0400 (EDT)
Received: from [192.168.16.214] (cpe-67-253-92-25.maine.res.rr.com [67.253.92.25]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by linode.ducksong.com (Postfix) with ESMTPSA id 5D52C102A5; Sat, 21 Aug 2010 19:58:15 -0400 (EDT)
From: Patrick McManus <mcmanus@ducksong.com>
To: John Tamplin <jat@google.com>
In-Reply-To: <AANLkTikE8+oEbviWrPO=Pi+=CHSMnCD9KYmH8EMxMktw@mail.gmail.com>
References: <AANLkTimKbmcpgx8k0uXUWvCO=8w9pPrtV=3y4qh6363k@mail.gmail.com> <alpine.DEB.2.00.1008212037190.27211@tvnag.unkk.fr> <AANLkTinrsT+wV48nHvVW_1ChGYffkq7jisU2-PZnMyKg@mail.gmail.com> <alpine.DEB.2.00.1008212123460.27211@tvnag.unkk.fr> <20ef7ed5e135c57c1ee5a741658b9d98.squirrel@sm.webmail.pair.com> <1282423311.2014.6.camel@tng> <b09e8e740ef2b5b5ec3c85a5d9dcfd08.squirrel@sm.webmail.pair.com> <AANLkTikE8+oEbviWrPO=Pi+=CHSMnCD9KYmH8EMxMktw@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Date: Sat, 21 Aug 2010 19:58:10 -0400
Message-ID: <1282435090.2014.12.camel@tng>
Mime-Version: 1.0
X-Mailer: Evolution 2.28.3
Content-Transfer-Encoding: 7bit
Cc: Hybi <hybi@ietf.org>
Subject: Re: [hybi] frame length encoding
X-BeenThere: hybi@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Server-Initiated HTTP <hybi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/hybi>
List-Post: <mailto:hybi@ietf.org>
List-Help: <mailto:hybi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 21 Aug 2010 23:57:44 -0000

On Sat, 2010-08-21 at 16:59 -0400, John Tamplin wrote:
> On Sat, Aug 21, 2010 at 4:57 PM, Shelby Moore <shelby@coolpage.com>
> wrote:

>          Can you simply rewrite
>         one sector on disk and the memory manager remaps?
> 
> 
> I haven't tried it personally, but I would suspect you could use
> TCP_CORK, write a header, use sendfile to send a chunk of the file to
> finish the frame, uncork, repeat until done.  I don't think there
> would be any extra copying involved here, but you can't just issue the
> sendfile and go on to the next message, you need some thread that will
> queue up the next frame when it is ready.

That is correct (and I have done this personally), you need multiple
calls of the ZC API interleaved with the write of the chunk header.
There is no copying of the data through the CPU just because it requires
multiple calls. And there is no reason you couldn't implement some kind
of writev() like API that lets you interleave copied and zero-copied
data.

And yes, multiple calls are more overhead than 1 call. but at 1 per 4GB,
it really is inconsequential.