Re: Header Serialization Discussion

[James M Snell <[e-mail]>]

The compression ratio for requests will be a bit lower, yes, but I'm
purposefully trading off simplified, more predictable and more
controlled state management and higher throughput. I'm looking now to
see how best to support the shared prefix stuff using the typed codecs
to get more compression out of it. The compression ratio for responses
tends to be significantly better than delta2 and headerdiff in
testing, largely due to the use of the typed codecs.

Here are the test numbers I'm currently getting running against the
amazon.har  (dhe = "delta header encoding".. which is the new impl
based on what I described).

Requests:
               size  time | ratio min   max   std
     http1  200,876  0.01 | 1.00  1.00  1.00  0.00
       dhe   71,352  0.08 | 0.36  0.00  0.71  0.15
    delta2   35,850  0.37 | 0.18  0.03  0.67  0.12
headerdiff   38,673  0.23 | 0.19  0.03  0.88  0.16

Responses:
               size  time | ratio min   max   std
     http1  160,435  0.06 | 1.00  1.00  1.00  0.00
       dhe   37,132  0.02 | 0.23  0.00  0.66  0.11
    delta2   42,764  0.43 | 0.27  0.02  0.65  0.10
headerdiff   46,321  0.69 | 0.29  0.04  0.84  0.13


- James

On Tue, Apr 16, 2013 at 9:06 AM, RUELLAN Herve
<[e-mail]> wrote:
>
>
>> -----Original Message-----
>> From: James M Snell [mailto:[e-mail]]
>> Sent: lundi 15 avril 2013 17:55
>> To: RUELLAN Herve
>> Cc: [e-mail]
>> Subject: Re: Header Serialization Discussion
>>
>> [snip]
>>
>> > My main concern with your proposal is about doing the LRU on both the
>> encoder and the decoder.
>> > I'd really like to keep the decoder as simple as possible, and so keeping all
>> the buffer management on the encoder side is a real win for me.
>> > In addition, the asymmetry means that the encoder is free to do whatever
>> buffer management it wants. LRU is a very good default buffer management
>> scheme, however I think there are cases where some clever scheme could
>> beat it.
>>
>> Well, it's not so much an LRU cache as a "least recently written"
>> queue. The buffer essentially consists of 128 memory slots. These are
>> assigned in order and rotate, with used slots deallocated and reassigned as
>> the buffer fills past it's limit. The encoder, then, needs to be selective about
>> just what it decides to assign to the buffer. So long as an implementation
>> follows the proper assignment order, the specific implementation does not
>> matter.
>>
>
> I'm afraid that using a "least recently written" queue would have a bad impact on compaction performances. There are some headers that don't change over a whole session (e.g. the user agent), or only take a few values over the whole session (e.g. accept). With a rotating scheme, these headers would have to be periodically re-added to the buffer.
>
> Hervé.