Re: [hybi] With deflate-stream, Close frame doesn't work as an end of data marker

[Takeshi Yoshino <[e-mail]>]

On Thu, Mar 10, 2011 at 21:16, Andy Green <[e-mail]> wrote:

> On <03%2F10%2F2011>03/10/2011 12:12 PM, Somebody in the thread at some
> point said:
>
> Hi -
>
>     Also one note on Yoshino's language to describe the compression
>>    actions it mixes Z_ nomenclature that zlib uses and hex packets on
>>    the wire, if this hex trailer that he mentions is a byproduct of
>>    Z_FINISH or whatever it might be clearer to an implementer that he
>>    can generate it like that and forget mentioning magic hex.
>>
>>
>> Basically I agree your point here that we should use only deflate
>> terminology in definition, and append implementation note to it using
>> zlib nomenclature, while avoiding mentioning magic hex as you say.
>>
>> In my proposal
>> (http://www.ietf.org/mail-archive/web/hybi/current/msg06642.html the
>> bottom of the message), I used one magic hex "0x00 0x00 0xff 0xff" to
>> include optimization on deflate adopted by PPP and explain what
>> implementor have to do quickly. That can be changed to ... something
>> like "0 byte length block with BTYPE=00 excluding headers", but it's not
>> helpful in this context IMO. This is originally a part of byproduct of
>> zlib's de-facto standard way to flush deflated stream, but operation
>> introduced by RFC 1979 like cutting, removing on sender, re-appending
>> the bytes on receiver is out of scope of RFC1951. Maybe, just referring
>> RFC1979 in the explanation of receiver's action would be fine?
>>
>
> Well my issue with it was I don't know what code to write as described, it
> sounds like I should fill up a four byte bufer with constants and write() it
> out.
>
> I guess all the actions fall into simple zlib calls with the right Z_
> flags, eg, it really means you must call the last guy with Z_FINISH. However
> else it's described it'll be helpful to implementers to give examples where
> it isn't clear in Z_ nomenclature as well.
>
> -Andy
>

OK. So we can add an implementation note like this.

Senders using this extension MUST apply RFC 1951 DEFLATE to all the bytes of
> the application data portion of data frames. Senders MAY include multiple
> blocks in a frame. Senders MAY use blocks of any type as defined in RFC
> 1951. Senders MUST use the algorithm described in Section 2.1 of RFC1979 to
> align compressed data to byte boundary. Senders MAY keep using the same LZ77
> sliding window for multiple frames.


If you're using zlib for applying DEFLATE compression, specify Z_SYNC_FLUSH
when you flush the last octets of application data for each frame (in case
the last block is non-compressed block (BTYPE=00), append one empty
compressed block (BTYPE=01)), and then strip the last 4 octets from the
compressed octets.

Receivers MUST append 0x00 0x00 0xff 0xff to the application data portion
> and decompress it using DEFLATE to decode the original application data.
> Receivers MUST keep using the same LZ77 sliding window for all the frames of
> the same WebSocket connection. Receivers MUST assume the senders use 32768
> byte LZ77 sliding window.


If you're using zlib for applying DEFLATE compression, for each received
frame, concatenate 4 octets 0x00 0x00 0xff 0xff and the application data,
and then supply the resulting bytes to zlib and use the uncompressed data.

----

There might be some function prepared on zlib that performs these procedure,
but I don't know that (i'm not so familiar with the original C lang zlib).
And, e.g. zlib wrapper of Python which I'm using to write WebSocket server
doesn't expose such method at all. I thought simple
uncompress/compress/flush methods are at least available on most of
platforms and built this text.