Re: Proposal: drop QPACK encoder stream framing

[Kazuho Oku <[e-mail]>]

2018-06-08 13:12 GMT+03:00 Martin Thomson <[e-mail]>:
> I think that this is a reasonable request and I'd support this.  It
> saves an encoder from having to buffer and it even saves a few octets.
> The complexity increase (the increased need to delay acknowledgments)
> is manageable and even encourages good practice - the current design
> encourages a little bit of laziness because you can acknowledge at the
> end of each block and have a reasonable expectation of it being
> approximately correct... most of the time.

+1

Previously I had thought that having prefix has the possibility of
simplifying the encoder. But I think I was wrong.

If we have framing, we need to do either of the following:

1a. buffer the entire frame, and then iterate though the instructions,
while checking for every instruction that it does not go beyond the
end of the frame
1b. iterate though the instructions as they arrive, while checking
that the entire instruction is available and that the instruction does
not go beyond the end of the frame

If we remove framing, the decoder can be implemented as:

2. iterate through the instructions as they arrive, while checking
that the entire instruction is available

As you can see, removing the framing simplifies the decoder design.
Because it reduces one layer of variable-length encoding.

> On Thu, Jun 7, 2018 at 5:11 PM Dmitri Tikhonov
> <[e-mail]> wrote:
>>
>> Abstract
>> --------
>>
>> The encoder block specified in Section 3.3 of draft-ietf-quic-qpack-00
>> is unnecessary.  For reasons of efficiency, I propose that the framing
>> be removed: let the encoder instructions be written and interpreted as
>> a stream instead.
>>
>> Background
>> ----------
>>
>> I brought this up during the first day (June 6) of the Kista Interim.
>> Alan explained that there are two reasons for framing the instructions
>> in this way on the encoder stream:
>>
>>     1. Historically, some (or most) HPACK decoders have struggled with
>>        input that breaks in the middle of an instruction, as they do
>>        not maintain state.
>>
>>     2. The blocks can be used as a clue to send Table State Synchronize
>>        instructions on the decoder stream (Section 3.4.1).
>>
>> The downside to having length-prefixed blocks is that it makes it
>> difficult to optimize encoding by writing to the outgoing packets
>> directly.  One has to buffer the block, write the block length first,
>> and then copy the block.
>>
>> Additionally, there is a potential for a deadlock (issue #1420).  Recall
>> that QPACK owes its current form to the fact that the WG decided to
>> prevent deadlocks at the compression protocol level, and not via advisory
>> notes to the implementers.
>>
>> Reasons Behind the Block
>> ------------------------
>>
>> First, let's look at the historical HPACK chokers.
>>
>> The HPACK decoder only processes HEADER blocks.  There is only one stream
>> and thus only one HEADER block can be processed at a time.
>>
>> The QPACK decoder has to process two inputs: a) the encoder stream
>> and b) HEADER blocks.  There is only one encoder stream.  This stream
>> modifies the decoder state: the dynamic table.  The HEADER blocks do not
>> modify the state.
>>
>> The HEADER blocks already come with a prefix length (HQ framing), and so
>> QPACK implementations can choose to wait until the whole HEADER block is
>> available.  It is processing of the encoder stream that is under
>> consideration.
>>
>> With regards to the Table Synch signaling, the draft states that
>>
>>  "                                                         A decoder MAY
>>  " coalesce multiple synchronization updates into a single update.
>>
>> This effectively means that the encoder already should not expect to get
>> a one-to-one mapping between its Encoder Blocks and Table Synchs.
>>
>> Working with the Encoder Stream
>> -------------------------------
>>
>> The dynamic table instruction stream can be thought of as its own state.
>> Already, the QPACK decoder must be able to pause reading from streams when
>> a blocked HEADERS block arrives.  Similar techniques can be used to resume
>> reading from the encoder stream.  It should not be difficult to resume
>> decoding a single instruction!  While breaking with the HPACK precedent
>> (the hold-my-hand-I-need-buffer-size chokers), it keeps in line with the
>> Seattle hum that the new compression mechanism is free from its HPACK
>> shackles.
>>
>> Conclusion
>> ----------
>>
>> Prefixing arbitrary sets of encoder instructions with a length denies
>> zero-copy optimization opportunities.  At the cost of some slight
>> complexity increase at the decoder, this framing mechanism can be
>> dropped.  The proposed change is limited to just this -- technically
>> superfluous -- piece.
>>
>>   - Dmitri.
>>
>> P.S.  An added bonus is some saved framing bytes on the encoder stream.
>>       I believe the "compression performance above everything else" is
>>       also the Working Group's consensus (Melbourne).
>>
>



-- 
Kazuho Oku