Re: Proposal: drop QPACK encoder stream framing
Ian Swett <ianswett@google.com> Fri, 08 June 2018 18:41 UTC
Return-Path: <ianswett@google.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1A35B131002 for <quic@ietfa.amsl.com>; Fri, 8 Jun 2018 11:41:54 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -17.51
X-Spam-Level:
X-Spam-Status: No, score=-17.51 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Kw750e-kVUjO for <quic@ietfa.amsl.com>; Fri, 8 Jun 2018 11:41:51 -0700 (PDT)
Received: from mail-yb0-x232.google.com (mail-yb0-x232.google.com [IPv6:2607:f8b0:4002:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9F079130F75 for <quic@ietf.org>; Fri, 8 Jun 2018 11:41:51 -0700 (PDT)
Received: by mail-yb0-x232.google.com with SMTP id f79-v6so4716781ybg.2 for <quic@ietf.org>; Fri, 08 Jun 2018 11:41:51 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=l385/Dri+0TniRXMS1z9oBo8cXSCoemVTy6zgj0qfto=; b=QH0tHpzPzWQqpbHICGJcctVAn3h5dGw3Rkt8Ck4jI7P4awfLfZbbJnDx70hat4p/c5 rDNpHmMLrJ+m5PZVJ5KNO2Nki0YgYkcP+om4Txl35zQowAQQzeeDwPRreYwKLASgfIF+ U3RuPvUyqhiWO1YEiLwUVgHbWNZxUKEXTcbfbgHD7TjKg80fcoFNw1ib2VywmKiTqJ28 jwt7rqJ11obXqf8tDp3citLzRVbKZ/8DY4lNa67cEnHGyKxj61u+rC/ku9lxuDC+MB1l 7p9Dg9AU0su98KWfXV1/L4ON+G14gBtqpD1uZGTDKGmnF9ozvJ9GFE3bLGKAoqa+XM3a q5Wg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=l385/Dri+0TniRXMS1z9oBo8cXSCoemVTy6zgj0qfto=; b=nr9Xva87j0lGgc7ayCNwWDdbDfBg6dtTGesg9lJCsunekWQRgjDYSjlyCZiRjNX2K6 WCWunezXyxhyeTNBdkZpjxDeiGUwpM/nNLFRLEwvd1RkoMGLr1QLntosI3VwY44yD14K ZMCTPAYGkqykgkwm36v1mvfY03tcQZDBB2irhLrnwMTfxZKp43altdbrNFr90Do+Wt0h U9HVYSWp/YogOmvFs2rlfSRikwBmKIpDeSSLfRyZF44vQ1qv3/C/Fkr4bpVKfakqgtLu USnQH+xMAIo94FEjk6GoaHGGorfe07GbTj+SXpy5WkHTzf4jTeOQyQR9KYebVhKJsqZV tlIQ==
X-Gm-Message-State: APt69E2jncpmkxhOwgFfM4Srq/kSHAG108wT29Jyca53O00vVj6hC5cA jbWr4MTqa6kPMdR12+iBhDet4KJrjCFUalEUqkrbK1ws
X-Google-Smtp-Source: ADUXVKJgEVlyexSXjEHsczzfyyUOjaUYF7Ay7VTIfI2R4AmLxjmGqBHB0ZQ7vWjHdAoCjRgn01fe2Rgw/TloHUgQJUU=
X-Received: by 2002:a25:cb96:: with SMTP id b144-v6mr4332524ybg.376.1528483310468; Fri, 08 Jun 2018 11:41:50 -0700 (PDT)
MIME-Version: 1.0
References: <20180607151112.GA28823@ubuntu-dmitri> <CABkgnnXbfkVbq6vCvud100h6wr3+9ir3iO7dD6-qaykOmK3JBQ@mail.gmail.com> <CANatvzzzXF8WCXgDCahTa_7JWZ1c1i-9rU_c8c9QWk5_O5u4Ow@mail.gmail.com>
In-Reply-To: <CANatvzzzXF8WCXgDCahTa_7JWZ1c1i-9rU_c8c9QWk5_O5u4Ow@mail.gmail.com>
From: Ian Swett <ianswett@google.com>
Date: Fri, 08 Jun 2018 14:40:59 -0400
Message-ID: <CAKcm_gOUT-QBmjt5rYwHrhu=A+PY6Sse7b5y=x+jnJ-SXqF5BA@mail.gmail.com>
Subject: Re: Proposal: drop QPACK encoder stream framing
To: Kazuho Oku <kazuhooku@gmail.com>
Cc: Martin Thomson <martin.thomson@gmail.com>, IETF QUIC WG <quic@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000c4b435056e25c046"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/Hsj1pVUMsJdm6S1od4q8KDB6uDA>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.26
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Jun 2018 18:41:55 -0000
I'm not as much a QPACK expert as others on this thread, but I do agree that it would be simpler without the length in this case, as well as saving a few bytes. On Fri, Jun 8, 2018 at 7:19 AM Kazuho Oku <kazuhooku@gmail.com> wrote: > 2018-06-08 13:12 GMT+03:00 Martin Thomson <martin.thomson@gmail.com>: > > I think that this is a reasonable request and I'd support this. It > > saves an encoder from having to buffer and it even saves a few octets. > > The complexity increase (the increased need to delay acknowledgments) > > is manageable and even encourages good practice - the current design > > encourages a little bit of laziness because you can acknowledge at the > > end of each block and have a reasonable expectation of it being > > approximately correct... most of the time. > > +1 > > Previously I had thought that having prefix has the possibility of > simplifying the encoder. But I think I was wrong. > > If we have framing, we need to do either of the following: > > 1a. buffer the entire frame, and then iterate though the instructions, > while checking for every instruction that it does not go beyond the > end of the frame > 1b. iterate though the instructions as they arrive, while checking > that the entire instruction is available and that the instruction does > not go beyond the end of the frame > > If we remove framing, the decoder can be implemented as: > > 2. iterate through the instructions as they arrive, while checking > that the entire instruction is available > > As you can see, removing the framing simplifies the decoder design. > Because it reduces one layer of variable-length encoding. > > > On Thu, Jun 7, 2018 at 5:11 PM Dmitri Tikhonov > > <dtikhonov@litespeedtech.com> wrote: > >> > >> Abstract > >> -------- > >> > >> The encoder block specified in Section 3.3 of draft-ietf-quic-qpack-00 > >> is unnecessary. For reasons of efficiency, I propose that the framing > >> be removed: let the encoder instructions be written and interpreted as > >> a stream instead. > >> > >> Background > >> ---------- > >> > >> I brought this up during the first day (June 6) of the Kista Interim. > >> Alan explained that there are two reasons for framing the instructions > >> in this way on the encoder stream: > >> > >> 1. Historically, some (or most) HPACK decoders have struggled with > >> input that breaks in the middle of an instruction, as they do > >> not maintain state. > >> > >> 2. The blocks can be used as a clue to send Table State Synchronize > >> instructions on the decoder stream (Section 3.4.1). > >> > >> The downside to having length-prefixed blocks is that it makes it > >> difficult to optimize encoding by writing to the outgoing packets > >> directly. One has to buffer the block, write the block length first, > >> and then copy the block. > >> > >> Additionally, there is a potential for a deadlock (issue #1420). Recall > >> that QPACK owes its current form to the fact that the WG decided to > >> prevent deadlocks at the compression protocol level, and not via > advisory > >> notes to the implementers. > >> > >> Reasons Behind the Block > >> ------------------------ > >> > >> First, let's look at the historical HPACK chokers. > >> > >> The HPACK decoder only processes HEADER blocks. There is only one > stream > >> and thus only one HEADER block can be processed at a time. > >> > >> The QPACK decoder has to process two inputs: a) the encoder stream > >> and b) HEADER blocks. There is only one encoder stream. This stream > >> modifies the decoder state: the dynamic table. The HEADER blocks do not > >> modify the state. > >> > >> The HEADER blocks already come with a prefix length (HQ framing), and so > >> QPACK implementations can choose to wait until the whole HEADER block is > >> available. It is processing of the encoder stream that is under > >> consideration. > >> > >> With regards to the Table Synch signaling, the draft states that > >> > >> " A decoder MAY > >> " coalesce multiple synchronization updates into a single update. > >> > >> This effectively means that the encoder already should not expect to get > >> a one-to-one mapping between its Encoder Blocks and Table Synchs. > >> > >> Working with the Encoder Stream > >> ------------------------------- > >> > >> The dynamic table instruction stream can be thought of as its own state. > >> Already, the QPACK decoder must be able to pause reading from streams > when > >> a blocked HEADERS block arrives. Similar techniques can be used to > resume > >> reading from the encoder stream. It should not be difficult to resume > >> decoding a single instruction! While breaking with the HPACK precedent > >> (the hold-my-hand-I-need-buffer-size chokers), it keeps in line with the > >> Seattle hum that the new compression mechanism is free from its HPACK > >> shackles. > >> > >> Conclusion > >> ---------- > >> > >> Prefixing arbitrary sets of encoder instructions with a length denies > >> zero-copy optimization opportunities. At the cost of some slight > >> complexity increase at the decoder, this framing mechanism can be > >> dropped. The proposed change is limited to just this -- technically > >> superfluous -- piece. > >> > >> - Dmitri. > >> > >> P.S. An added bonus is some saved framing bytes on the encoder stream. > >> I believe the "compression performance above everything else" is > >> also the Working Group's consensus (Melbourne). > >> > > > > > > -- > Kazuho Oku > >
- Re: Proposal: drop QPACK encoder stream framing Ian Swett
- Re: Proposal: drop QPACK encoder stream framing Roberto Peon
- Re: Proposal: drop QPACK encoder stream framing Roberto Peon
- Proposal: drop QPACK encoder stream framing Dmitri Tikhonov
- Re: Proposal: drop QPACK encoder stream framing Spencer Dawkins at IETF
- Re: Proposal: drop QPACK encoder stream framing Spencer Dawkins at IETF
- Re: Proposal: drop QPACK encoder stream framing Dmitri Tikhonov
- Re: Proposal: drop QPACK encoder stream framing Kazuho Oku
- Re: Proposal: drop QPACK encoder stream framing Martin Thomson
- Re: Proposal: drop QPACK encoder stream framing Ian Swett
- Re: Proposal: drop QPACK encoder stream framing Alan Frindell
- Re: Proposal: drop QPACK encoder stream framing Kazuho Oku
- Re: Proposal: drop QPACK encoder stream framing Alan Frindell
- Re: Proposal: drop QPACK encoder stream framing Kazuho Oku
- Re: Proposal: drop QPACK encoder stream framing Dmitri Tikhonov
- RE: Proposal: drop QPACK encoder stream framing Mike Bishop
- Re: Proposal: drop QPACK encoder stream framing Alan Frindell
- Re: Proposal: drop QPACK encoder stream framing Dmitri Tikhonov
- Re: Proposal: drop QPACK encoder stream framing Dmitri Tikhonov