Re: Proposal: drop QPACK encoder stream framing

[Dmitri Tikhonov <[e-mail]>]

On Fri, Jun 08, 2018 at 08:49:37PM +0000, Alan Frindell wrote:
> Consider the most complicated instruction, which is Inserting a pure
> literal (all fields have variable length)
> 
> | Name Length | Name | Value Length | Value |
> 
> When you know the length of the instruction block in advance, you know
> that an incomplete instruction is a decoding error which you can check
> as you decode, and fail immediately on underflow.

The example above contradicts the argument that follows.  It is evident
that we have a length followed by an instruction block (the Huffman-encoded
string).  A QPACK implementation is still free to buffer the instruction
blocks, in which case all the perceived advantages of framing you present
below are applicable.

Parsing of the length is, of course, more complicated than requiring a
known number of bytes to be available to read in a fixed-length integer.
Nevertheless, as Kazuho points out, there are practical limits whereby
most implementations will abide.

> Here, you either need to do a trial decode of the variable lengths
> to see if you have enough bytes, or check while decoding, and decide
> whether you want to save the state for later or discard it and reparse
> from the beginning of the instruction.  I'm not even sure reparsing
> is workable since it might open you up to doing N^2 work -- at least
> you need to be careful how you implement it.  It's worth calling out
> that discovering the length of a QPACK varint is more complicated than
> a QUIC varint.

An efficient implementation will not just blindly reparse over and over
as each byte of the stream data becomes available.  With a buffering
implementation, the need to reparse disappears if one is willing to
allocate the buffer large enough.  This is exactly the approach you
advocate to use with the current instruction block framing.

The proposal to drop framing is to allow an optimization (eliminate a
copy) at the encoder, not the decoder.  The decoder is still free to
use the same programming techniques as before.

> It’s not impossible to solve of course.  “It’s just software”
> as Dmitri likes to say.  It’s just one piece of complexity I was
> hoping to avoid

And I hope I have demonstrated that the complexity increase is
minimal.  Soon, I will demonstrate the same using C code.

  - Dmitri.