Re: [hybi] hum #3: Message

[Julian Reschke <[e-mail]>]

On 05.08.2010 22:21, Maciej Stachowiak wrote:
>
> On Aug 5, 2010, at 12:52 PM, Julian Reschke wrote:
>
>> On 05.08.2010 21:31, Ian Hickson wrote:
>>> On Thu, 5 Aug 2010, Maciej Stachowiak wrote:
>>>>
>>>> That being said, I think you're misunderstanding Ian's point. He
>>>> expressed concern about the overhead for small messages to having a
>>>> large fixed length, not the overhead to large messages of a chunked
>>>> encoding.
>>>
>>> I was trying to express both.
>>>
>>> Having to split a 2GB file into a bazilion pieces and write each one out
>>> individually with frame headers, compared to just writing the whole thing
>>> at once, seems inefficient far beyond simply the concern about the
>>> on-the-wire overhead.
>>
>> Hum, where exactly do you see the additional overhead? It's not like you need to split it into inidividual *files* - you just interleave the data with the frame headers while writing to the stream...
>
> Sending a file from disk to the network without processing it in any way is more efficient than cutting it into slices and interleaving with headers. The IP layer is already going to cut it into slices, in a way that almost certainly won't line up with WebSocket-level fragment boundaries.

There's no way to "send a file over the network without processing". You 
need to read it into memory, and then write it to the network. Of course 
adding frame headers *is* overhead, but it's really identical or close 
to the overhead on the wire, nothing more.

> It's also more efficient for the receiver if they know the length up front. That way they can determine whether to allocate a large enough buffer, or just start spooling to persistent storage immediately.

Absolutely. When the length is known (see video streaming and on-the-fly 
compression). Thus, it might be good to be able to announce the eppected 
size upfront.

Best regards, Julian