Re: [hybi] Frame size

Scott Ferguson <ferg@caucho.com> Mon, 19 April 2010 04:06 UTC

Return-Path: <ferg@caucho.com>
X-Original-To: hybi@core3.amsl.com
Delivered-To: hybi@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id F22CC3A69EF for <hybi@core3.amsl.com>; Sun, 18 Apr 2010 21:06:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.104
X-Spam-Level:
X-Spam-Status: No, score=0.104 tagged_above=-999 required=5 tests=[AWL=-0.232, BAYES_50=0.001, IP_NOT_FRIENDLY=0.334, UNPARSEABLE_RELAY=0.001]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OchJ0WYt5bMl for <hybi@core3.amsl.com>; Sun, 18 Apr 2010 21:06:27 -0700 (PDT)
Received: from smtp114.biz.mail.sp1.yahoo.com (smtp114.biz.mail.sp1.yahoo.com [69.147.92.227]) by core3.amsl.com (Postfix) with SMTP id 0B9133A6A2D for <hybi@ietf.org>; Sun, 18 Apr 2010 21:06:26 -0700 (PDT)
Received: (qmail 23896 invoked from network); 19 Apr 2010 04:06:15 -0000
Received: from scott-fergusons-macbook-pro-3.local (ferg@70.231.144.150 with plain) by smtp114.biz.mail.sp1.yahoo.com with SMTP; 18 Apr 2010 21:06:15 -0700 PDT
X-Yahoo-SMTP: L1_TBRiswBB5.MuzAo8Yf89wczFo0A2C
X-YMail-OSG: qD5YnzsVM1nu8jc5I9KkpkhuDmX4JlC_g0r7hgfx3magYjy864roDBFcUWi7t9DluU_M3gx9McVvAn3ZbjOvOVHJKwoK2WoQg_F4RgX6_JY7qJFq9yPfL0s.96lbAKrZz0bsSD0VtACEX45yTsGlyb1cne31T0FbkXwuBP1MINxEdkbBLINxq3dVxXQeMjV6LZqIuKC4zlzoYZ94zxMkabLxypU2OxDDNIES1X4zFP_oZgypjP2cgXw6gqsaaTeh9hnRCj6492XjS2bozKnMFPqpOMc7oBu7Ye7_OEqxL..dkRBlJu.v4w--
X-Yahoo-Newman-Property: ymail-3
Message-ID: <4BCBD6B6.7010802@caucho.com>
Date: Sun, 18 Apr 2010 21:06:14 -0700
From: Scott Ferguson <ferg@caucho.com>
User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228)
MIME-Version: 1.0
To: Mike Belshe <mike@belshe.com>
References: <8B0A9FCBB9832F43971E38010638454F03E3F313ED@SISPE7MB1.commscope.com> <v2m5c902b9e1004160043i7b5ccc79y2346e1b2b2c55cf5@mail.gmail.com> <s2qad99d8ce1004160053w436a29b1idae0c66737b3760a@mail.gmail.com> <4BC85A31.6060605@webtide.com> <t2iad99d8ce1004160949yb1ba9582l3b626c19dacf8d9@mail.gmail.com> <4BC96DA1.3000706@webtide.com> <u2m2a10ed241004181635qd0554193v36da94ecd7284d31@mail.gmail.com> <l2o2a10ed241004181637hdfab97d5r68f6845be49e8ad8@mail.gmail.com> <20100419005102.GC18876@shareable.org> <g2n2a10ed241004182005n9d8a5f02o29702620ae6205f4@mail.gmail.com>
In-Reply-To: <g2n2a10ed241004182005n9d8a5f02o29702620ae6205f4@mail.gmail.com>
Content-Type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-Transfer-Encoding: 7bit
Cc: Hybi <hybi@ietf.org>
Subject: Re: [hybi] Frame size
X-BeenThere: hybi@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Server-Initiated HTTP <hybi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/hybi>
List-Post: <mailto:hybi@ietf.org>
List-Help: <mailto:hybi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 19 Apr 2010 04:06:28 -0000

Mike Belshe wrote:
>
>
>      - to leave room for transport control messages inserted by the
>        transport (such as graceful close and transport error indicators),
>        so that applications don't have implement them
>
>
> The protocol has implicit chunks - frames.  Having a second level of 
> chunking has little value here.  Or at least I haven't heard the use case.
Same as HTTP/1.1 chunking: when the length of dynamically produced 
content cannot be calculated beforehand.

Unless the frames can be assembled into messages, it's not implicit 
chunking.
>
> Applications that want to use chunking can do so already by just using 
> smaller frames.
I don't see how that makes any sense.

Here's a basic, typical application. Suppose messages are XMPP packets 
(ours are binary HMTP, but the idea's the same). The message is 
identical to the entire packet.  The sender don't know how big the 
packet will be until serialization completes, but receivers may need to 
know the message/packet end before parsing.

For sanity and performance, we use a fixed-size output buffer for each 
packet. Each frame is a fragment of the XMPP packet and the message only 
ends with the final frame. We don't know the length of the entire packet 
until the entire thing is serialized. So the protocol needs multiple 
frames forming a message.

This is a very basic pattern. There's nothing unusual about it and I 
don't see how smaller frames addresses the issue, because it's the 
message boundary that's important. If you're not marking message 
boundaries, you may as well eliminate the frames entirely and just have 
raw TCP.

16 vs 32 vs variable-length integer encoding isn't hugely important. 
Chunking is important.
> Overall, I'm talking about simplicity.  A 32bit fixed length is simple 
> and sufficient for purposes today and tomorrow.
HTTP's fixed Content-Length is not sufficient for the same reasons.

-- Scott