Re: [hybi] With deflate-stream, Close frame doesn't work as an end of data marker

[[e-mail]]

Hi Takeshi, my comments below...


> o  Framing is necessary because of the underlying transport 
>    layer is in fact 'streaming', or TCP. (TCP and WS-framing 
>    are more tightly coupled) 
> 
> For endpoints/intermediaries which don't recognize deflate-stream, 
> yes, it's problematic.

What if we'd like to introduce a new control frame that is intended
to communicate with intermediaries (imagine, trace-route mechanism, 
etc.) in the future, then the 'compression' layer gets in the way 
and all intermediaries have to decompress stream all the time. 

This is clearly another consequence of having compression under the 
framing layer. We should avoid this, IMO.


> o  WS-layer needs to buffer entire message (passed by app/JS) 
>    anyway. 
> o  Text and binary do not interleave fragments, hence there 
>    needs only one context for compression / decompression in 
>    memory. 
> 
> People seemed to be more concerned about compression efficiency at 
> that time than the cost of compression state sharing. This is a 
> point to revisit.

Yeah... as now we see downside of the current direction.

>  
> o  Overhead (frame headers and control frames) are much smaller 
>    than payload of text/binary frames and negligible as to 
>    compression ratio. 
> 
> Not sure.

We may need more quantitative evidence here, though I agree 
larger scope (~32KB in case of gzip) for compression by
references for recurring texts would make compression ratio 
better. I also imagine that recurring texts may appear more in
the same message (or channel) than in the other channel. In mux'd
case for example, I am not sure how the current approach would
help.


> o  Close frame appears at the end of stream with no traling 
>    bytes. (Can remove requirement for TCP graceful shutdown). 

I think this benefit is significant. We do not want to worry
about another trailing bytes in the future again when a new 
compression algorithm needs to be introduced.

Per message compression has another benefit of shrinking the 
size of buffer for storing application messages in memory.


- Yutaka