Re: [hybi] With deflate-stream, Close frame doesn't work as an end of data marker

[John Tamplin <[e-mail]>]

On Thu, Mar 10, 2011 at 7:34 AM, Takeshi Yoshino <[e-mail]>wrote:

> OK. So we can add an implementation note like this.
>
> Senders using this extension MUST apply RFC 1951 DEFLATE to all the bytes
>> of the application data portion of data frames. Senders MAY include multiple
>> blocks in a frame. Senders MAY use blocks of any type as defined in RFC
>> 1951. Senders MUST use the algorithm described in Section 2.1 of RFC1979 to
>> align compressed data to byte boundary. Senders MAY keep using the same LZ77
>> sliding window for multiple frames.
>
>
> If you're using zlib for applying DEFLATE compression, specify Z_SYNC_FLUSH
> when you flush the last octets of application data for each frame (in case
> the last block is non-compressed block (BTYPE=00), append one empty
> compressed block (BTYPE=01)), and then strip the last 4 octets from the
> compressed octets.
>
> Receivers MUST append 0x00 0x00 0xff 0xff to the application data portion
>> and decompress it using DEFLATE to decode the original application data.
>> Receivers MUST keep using the same LZ77 sliding window for all the frames of
>> the same WebSocket connection. Receivers MUST assume the senders use 32768
>> byte LZ77 sliding window.
>
>
> If you're using zlib for applying DEFLATE compression, for each received
> frame, concatenate 4 octets 0x00 0x00 0xff 0xff and the application data,
> and then supply the resulting bytes to zlib and use the uncompressed data.
>

zlib is just one implementation, and I don't think the spec should be
choosing one implementation instead of the algorithm.  Therefore, we
shouldn't be using zlib-specific terminology in the spec.

I don't object to compressing only the payload (though it does make
implementations more complicated, rather than just pushing a
compressor/decompressor onto the stream having to switch back and forth),
but couldn't the issue we are trying to solve here be done by just saying
that if deflate-stream is negotiated and a close frame is read, you cannot
initiate close processing until the final compressed block is read
completely?

In any case, I think deflate-stream is only a stop-gap solution to get some
form of compression in the spec, also serving as an example extension.  I
think realistically we are going to have to have the ability to avoid
compressing incompressible data in the stream.  Initially, where WebSockets
implementations only support plain text, this won't be much of a problem,
but eventually clients will be mixing binary data with their text and we
would prefer to be able to use compression where it helps and not use it
where it hurts.

-- 
John A. Tamplin
Software Engineer (GWT), Google