Re: [hybi] Framing take IV [why fragment]

[John Tamplin <[e-mail]>]

On Thu, Aug 5, 2010 at 10:07 AM, Dave Cridland <[e-mail]> wrote:

> On Wed Aug  4 15:33:13 2010, Patrick McManus wrote:
>
>> On Wed, 2010-08-04 at 00:53 +0000, Ian Hickson wrote:
>> >  I could see an argument for supporting
>> > fragmentation in the case of multiplexing, but without that it doesn't
>> > seem to actually gain us anything.
>>
>> In the absence of fragmentation the sender must have the entire message
>> available in order to prefix the total length in the header - so it
>> engages in store and forward instead of streaming behavior.
>>
>>
>>  I think some of these are bad arguments.


Imagine we have frame-based compression (I know this still up for debate,
though I think a good case has been made that we need to be able to compress
some payloads and not others).  Then the code is going to look something
like this:

empty output buffer
while there is input data
    read the next buffer full (if not already in a byte array/string)
    while the input buffer is not empty
      compress the input buffer into the output buffer
      if output buffer is full, send a non-final frame with the output
buffer and clear it
      advance the input buffer pointer
    end while
end while
send a final frame with the output buffer

If we don't have chunking, we must buffer the entire compressed output in
memory, which means allocating and reallocating an arbitrarily large buffer
rather than sending each output buffer's full of data.

No, because the receiver cnnot process portions of messages - at least with
> our current API. And you can achieve this behaviour by sending multiple
> messages anyway.


For per-frame compression, you certainly can decompress each chunk as you
receive it.  Yes, the JS API currently does not have a way to pass partial
messages, and perhaps that needs to get fixed when binary support is added
since it seems more likely you will want to be able to react to partial
messages as each part is received.

-- 
John A. Tamplin
Software Engineer (GWT), Google