Re: [hybi] With deflate-stream, Close frame doesn't work as an end of data marker

[Andy Green <[e-mail]>]

On 03/09/2011 12:06 PM, Somebody in the thread at some point said:

Hi -

>     Having implemented this now, I don't see the problem using
>     Z_PARTIAL_FLUSH as the standard seems to recommend normally, and
>     then Z_FULL_FLUSH when sending the close packet, and disabling any
>     further packet issue on that connection.  So I don't think Takeshi's
>     issue is more than an implementation problem, I'd welcome being
>     educated if I missed any point.
>
>
> Maybe, exhaustive (to be strict, recv with big buffer) recv call in your
> code drains all the deflated data arrived (everything including Huffman
> codes for some preceding frames, ones for close frame, 3 bit header
> BFINAL=0 BTYPE=00 and 00 00 ff ff) from the TCP stack.

Yes on both rx / inflate and tx / deflate case libwebsockets will always 
loop and go back to in/deflate() for the rest if the callback handling 
that indicates the zlib call filled its spill buffer (suggesting that it 
very likely has more to spill).  So if there are multiple buffers full 
of stuff to flush in both rx or tx path they will always all get sent on 
the wire or sent to the user callback as uncompressed rx data before 
moving on.

You can see the loop here for the tx path

http://git.warmcat.com/cgi-bin/cgit/libwebsockets/tree/lib/parsers.c#n1378

After issuing specifically the CLOSE opcode packet like that, the 
connection state is changed so nothing more can be emitted on the 
socket, and this works reliably to issue the CLOSE to the far end for 
sure without further zlib housekeeping traffic or any other traffic 
going out.

> Yes, since it's rare that octets for a single WebSocket frame are put in
> separate TCP packets and delivered to application layer separately
> (requiring separate recv-call), this is not a big problem. But with
> in-frame compression, it becomes perfect.

I think you must handle extreme compression cases anyway to keep an eye 
on latency if nothing else, consider he sends 200K of 0x00 and the 
compressor run-length encodes it, you receive a handful of compressed 
code bytes but must sit looping through 200K of decompressed content 
that represents one message.  Or you pass in 1K of high quality random 
and 3K of codes comes out of the compressor.

Like I say I think keeping framing out of compression is okay, just 
deflate-stream babbling after close isn't any reason for it AFAICS.

-Andy