Re: [hybi] With deflate-stream, Close frame doesn't work as an end of data marker

[Takeshi Yoshino <[e-mail]>]

Hi,

On Fri, Feb 25, 2011 at 14:19, <[e-mail]> wrote:

>
> Hi,
>
> If we could apply deflate-stream only to payload, that would solve
> the problem. I was looking for reasons why entire stream has to be
> compressed, and found only one here:
>
> http://www.ietf.org/mail-archive/web/hybi/current/msg04117.html
>
> From layering aspects, I would prefer any compression taking place
> on top of framing layer from the other way around:
>
>
Yeah, Patrick initially proposed compression using that approach
http://www.ietf.org/mail-archive/web/hybi/current/msg04071.html

The options we can take are basically
a) compress only application data not keeping compression state
b) compress only application data while keeping compression state between
multiple frame
c) compress entire stream

John, Dave objected a) since it's degrading compression rate. I agree that.
So, b) and c) are left. Patrick wrote up c) as deflate-stream extension.

The name deflate-stream was given for c) (see
http://www.ietf.org/mail-archive/web/hybi/current/msg04129.html). I think
it's OK to leave deflate-stream extension as is. I'm also fine with this
built-in in the initial version of the WebSocket protocol spec. What we
should do now is try to define some "better-deflate" which convinces people.
These threads is in a kind of suspended state. We should use them or create
a new thread for resuming discussion on some "better-deflate" there.

- Some said we need ability to disable compression on frame with
incompressible contents
- Coordination with mux is also problem
- If frame header is compressible/incompressible is still a question.
etc...


>    +-------------+
>    | compression |
>    +-------------+
>    +-------------+
>    | WS framing  |
>    +-------------+
>    +-------------+
>    |     TCP     |
>    +-------------+
>
> where, current draft -05 sugggests:
>
>    +-------------+
>    | WS framing  |
>    +-------------+
>    +-------------+
>    | compression |
>    +-------------+
>    +-------------+
>    |     TCP     |
>    +-------------+
>
> Justifications for the former layering would be:
>
> o  Framing is necessary because of the underlying transport
>    layer is in fact 'streaming', or TCP. (TCP and WS-framing
>    are more tightly coupled)
>

For endpoints/intermediaries which don't recognize deflate-stream, yes, it's
problematic.

Except for this, technically, it's ok since frame boundary can be
immediately told to the other peer by flush-ing e.g. Z_SYNC_FLUSH of zlib.


> o  WS-layer needs to buffer entire message (passed by app/JS)
>    anyway.
> o  Text and binary do not interleave fragments, hence there
>    needs only one context for compression / decompression in
>    memory.
>

People seemed to be more concerned about compression efficiency at that time
than the cost of compression state sharing. This is a point to revisit.


> o  Overhead (frame headers and control frames) are much smaller
>    than payload of text/binary frames and negligible as to
>    compression ratio.
>

Not sure.


> o  Close frame appears at the end of stream with no traling
>    bytes. (Can remove requirement for TCP graceful shutdown).
>
>
Yes, it's not a problem if we adopt payload only compression.


> Please let me know if I am missing something...
>
> Best.
> - Yutaka
>
>