Re: [quicwg/base-drafts] DATA frame encoding is inefficient for long dynamically generated bodies (#1885)

Kazuho Oku <notifications@github.com> Fri, 19 October 2018 22:19 UTC

Return-Path: <noreply@github.com>
X-Original-To: quic-issues@ietfa.amsl.com
Delivered-To: quic-issues@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2A948130E71 for <quic-issues@ietfa.amsl.com>; Fri, 19 Oct 2018 15:19:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.064
X-Spam-Level:
X-Spam-Status: No, score=-8.064 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.064, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=github.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Jn6guF0SDtvD for <quic-issues@ietfa.amsl.com>; Fri, 19 Oct 2018 15:19:14 -0700 (PDT)
Received: from out-13.smtp.github.com (out-13.smtp.github.com [192.30.254.196]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2A893128C65 for <quic-issues@ietf.org>; Fri, 19 Oct 2018 15:19:14 -0700 (PDT)
Date: Fri, 19 Oct 2018 15:19:13 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=github.com; s=pf2014; t=1539987553; bh=ieqYjN+Bm12be8rJpH/O2OYWCURcRkvRAxifFSD9Kbk=; h=Date:From:Reply-To:To:Cc:In-Reply-To:References:Subject:List-ID: List-Archive:List-Post:List-Unsubscribe:From; b=BIgi/qJ7dEyTNxc4hCeBGEeQKd+aZtY7tVe8WfVC7900HYE06AJBnYtoosuhuekv+ Tco6hZDL5XCWPAKHn/ag69C5j9fxY0bn9BWjmg3rKk4QgnMAHJllTuhvnMcV0onQyb QcLpeYqtMW7/XRZ0wsrkS68dHpLH3BfZjodLH6Kw=
From: Kazuho Oku <notifications@github.com>
Reply-To: quicwg/base-drafts <reply+0166e4abe8445341e3e33984beed50558c1bb9481d16b5e492cf0000000117e21a6192a169ce16286866@reply.github.com>
To: quicwg/base-drafts <base-drafts@noreply.github.com>
Cc: Subscribed <subscribed@noreply.github.com>
Message-ID: <quicwg/base-drafts/issues/1885/431515701@github.com>
In-Reply-To: <quicwg/base-drafts/issues/1885@github.com>
References: <quicwg/base-drafts/issues/1885@github.com>
Subject: Re: [quicwg/base-drafts] DATA frame encoding is inefficient for long dynamically generated bodies (#1885)
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="--==_mimepart_5bca586183dcf_641d3f85b80d45bc2257e3"; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: list
X-GitHub-Sender: kazuho
X-GitHub-Recipient: quic-issues
X-GitHub-Reason: subscribed
X-Auto-Response-Suppress: All
X-GitHub-Recipient-Address: quic-issues@ietf.org
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic-issues/0vYA0LjoVJb5lxTOyCErQh2WMhA>
X-BeenThere: quic-issues@ietf.org
X-Mailman-Version: 2.1.29
List-Id: Notification list for GitHub issues related to the QUIC WG <quic-issues.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic-issues>, <mailto:quic-issues-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic-issues/>
List-Post: <mailto:quic-issues@ietf.org>
List-Help: <mailto:quic-issues-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic-issues>, <mailto:quic-issues-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 19 Oct 2018 22:19:16 -0000

@RyanAtGoogle
> how would this proposal prevent an implementation from capping the maximum size of a frame?

It doesn't. However, introducing the concept of non-DATA frames terminated by EOS has a impact on complexity. We handle HQ frames in the following way:
* We keep the frame header in the receive buffer until all the header is delivered. Then,
* if it is a DATA frame, we pop the frame header from the receive buffer, and continue reading the data out from the receive buffer until we read the exact amount of data specified by the frame header
* if it is not a DATA frame, we check that the size of the frame is below 16KB. If it is above that, we issue a connection close. If it is below that, we keep the octets in the receive buffer until the entire frame arrives. Once the frame arrives, we process it in-place, then remove it from the receive buffer.

Having non-DATA frames terminated by EOS means that we need add a different path for the last bullet. I'm sure I can implement that, but my preference goes to avoiding it.

I can understand the motivation to use EOS as a terminator for DATA frames. It makes sense because you might emit a large number of DATA frames for each HTTP response, and because DATA is typically the last frame type to be sent using a stream. Neither of the two points apply to other frame types, which means that the optimization is far less useful for them, even though I understand the elegance of permitting that for all frame types.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1885#issuecomment-431515701