Re: [hybi] Framing take IV

[Maciej Stachowiak <[e-mail]>]

On Aug 3, 2010, at 6:29 PM, Greg Wilkins wrote:

> 
> 
> On 4 August 2010 11:20, Maciej Stachowiak <[e-mail]> wrote:
> 
> On Aug 3, 2010, at 6:07 PM, Greg Wilkins wrote:
> 
> 
> Hum volume is not an adequate response to a reasoned argument. If people think fragmentation is simpler than a variable size length field, they should explain why, rather than just citing the number of people who agree.
> 
> 
> I did briefly explain.  I cited the hum volume not as an argument, but as an indication that we have already had the arguments explained.

If good arguments were presented at the face-to-face, they should be presented on the mailing list, since the IETF process is that consensus cannot be declared without confirming on the mailing list.

> 
> happy to explain again in a little bit more detail:

Thanks for the explanation, here are some counterpoints:
> variable length length fields is more complex than a fixed length length field.
Is it more complex than fragment reassembly? I find that extremely unlikely based on my experience as a programmer.
> sending large content in a single frame is difficult if the content is being generated dynamically or it's length is otherwise not known.
You are correct, and that is a good reason for optional chunking. However, sending large content as chunks instead of a single frame is harder when it already exists as a data item of known length (e.g. a large file). Extremely large objects (over a gig, say) tend not to be dynamically generated. If you have to send a gig of data, chunking it into 64k frames is likely to be extremely inefficient.
> a small fixed sized length allows simple implementations to have fixed sized buffers (eg 64k)
A variable size length allows this too - you can just read up to 64k at a time. Whether with a single large frame or multiple fragments, you can choose yourself whether to process 64k at a time or reassemble at the end.
> fragmentation prepares the ground for future extensions for multiplexing and flow control.
I'm not assuming we will need such extensions until we have data to back it up.
> fragmentation on its own is not that complex
I disagree. It's more complex for senders of fixed-length content, and for receivers in nearly all cases.

On the other hand:

* If the receiver would like to read all contents into a single large memory buffer up to some fairly large size (say, a gigabyte), then it's a significant disadvantage, and likely performance hit, to get it in chunked form. The length is not known up front, therefore it is impossible to pre-allocate a buffer of the right size. To read a 256k message if chunks are 64k max, you have to perform at least 4 memory reallocations; with a single frame it could be one. Browser-hosted clients will likely have to provide a 256k message in memory to JavaScript, so it's not an option to have a 64k buffer and process the content in chunks or spool it to disk.

Thus, a single large message supports both of the following strategies:

A) Read a chunk at a time into a smallish fixed-size buffer.
B) Pre-allocate a buffer large enough for all the data and read it all in.

Mandatory fragmentation for large messages only supports strategy (A). Strategy (B) is likely preferable in the case where the browser is the receiver.

> But if you have a counter proposal that you think might satisfy the hummers, please make it.  The current protocol certainly does not.

Without knowing the rationale of any given hummer, I cannot predict what they would find satisfactory. I hope I have explained enough of my reasoning that others can guess whether a proposal would seem good me. Mandatory chunking for large messages does not sound good. 

Regards,
Maciej