Re: [hybi] Flow control quota

[Martin Sustrik <[e-mail]>]

Hi Arman,

I am not following the hybi mailing list thoroughly, so I am not 
familiar with the technical details... However, it seems that the 
problem here is that it's possible to use any combination of extensions.

This pattern doesn't work well. The problem is that with the growing 
number of extensions the number of possible combinations and 
(mis)interactions between them is growing exponentially, so it's 
basically impossible to deliver a fully functional stack.

The approach that standard Internet protocols take is separating the 
functionality into layers where each layer can be filled by only a 
single protocol at a time. E.g. if you are using TCP at L4 for a 
particular connection you can't use SCTP at the same time.

I don't understand the details of WS, so I don't want to make any strong 
proposals, but maybe separating the extensions into several well-defined 
layers where each layer can be occupied only by a single extension 
instance may help.

Martin

On 01/06/12 16:15, Arman Djusupov wrote:
> Unfortunately making extensions mutually aware of each other at the
> implementation level does not prevent problems from being encountered by
> a mux intermediary that receives a frame with an RSV bit set. An
> intermediary is not permitted to fragment a frame with an RSV bit set
> especially when the extension is unknown. If such a frame is larger than
> the available quota then the intermediary would have to wait until the
> quota becomes larger than the frame size before it can send it. This
> will result in the frame never being sent if its size is larger than the
> flow control window.
>
> A practical example of this problem is that a mux intermediary might end
> up being unable to forward compressed frames larger than the flow
> control window, unless it first decompresses them and refragments them.
>
> I am trying to figure out a general purpose solution that would allow us
> to deal with non-fragmentable frames, even if the extension that is
> applied to them is unknown. So far I have two ideas:
>
> 1.
>
> Allow the sender to send a frame with an RSV bit set even if it is
> bigger that the available quota. In such a case the quota becomes
> negative and from that moment on the sending side MUST wait until the
> quota becomes positive before sending the subsequent frame. This way the
> flow control would work and the sender would be able to send a frame
> that is larger than the available quota but cannot be fragmented.
>
> Flow control is not intended to limit the maximum frame size, but rather
> to prevent a logical channel from exclusively using a physical
> connection. This solution would satisfy this requirement. A logical
> channel would wait as long as necessary until the quota becomes
> positive. This would let other channels use the physical connection.
> However, this solution would not prevent a sender from forcing the
> receiver to buffer an amount of bytes that is a larger than the amount
> it had agreed to buffer. But the amount of extra buffering required
> would not exceed the size of a single frame so a mux intermediary would
> at least be able to forward frames of reasonable size.
>
> 2.
>
> An alternative solution is to introduce a WebSocket fragmentation
> extension. Using this extension, multiplexing intermediaries would
> advertise the maximum frame size limit for non-mux clients. Non-mux
> clients that want their connections to be multiplexed by an intermediary
> would have to stick to the frame size limits required by that
> intermediary. By supporting this extension WebSocket client/browser
> vendors will be able to make their clients mux-intermediary-friendly.
> This would help not only in non-mux-client-to-mux-intermediary scenarios
> (and flow control in general) but would also permit us to advertise a
> frame size limit in any scenario where such limits are present. This was
> discussed by the WG but didn’t made it into RFC 6455, the idea at that
> time was that it could go in a separate extension. So currently we have
> section 10.4 that requires implementations to protect themselves against
> exceeding frame size limits. A fragmentation extension would enable us
> to advertise these limits.
>
> 10.4. Implementation-Specific Limits
>
> Implementations that have implementation- and/or platform-specific
>
> limitations regarding the frame size or total message size after
>
> reassembly from multiple frames MUST protect themselves against
>
> exceeding those limits. (For example, a malicious endpoint can try
>
> to exhaust its peer's memory or mount a denial-of-service attack by
>
> sending either a single big frame (e.g., of size 2**60) or by sending
>
> a long stream of small frames that are a part of a fragmented
>
> message.) Such an implementation SHOULD impose a limit on frame
>
> sizes and the total message size after reassembly from multiple
>
> frames.
>
> With best regards,
>
> Arman
>
> *From:*Takeshi Yoshino [mailto:[e-mail]]
> *Sent:* Friday, June 01, 2012 3:33 PM
> *To:* Martin Sustrik
> *Cc:* Arman Djusupov; [e-mail]
> *Subject:* Re: [hybi] Flow control quota
>
> On Fri, Jun 1, 2012 at 5:43 PM, Martin Sustrik <[e-mail]
> <mailto:[e-mail]>> wrote:
>
> That seems to suggest there's some kind of mis-layering happening here.
> Frames are used for flow control. Does it make sense to use them for
> compression as well? Shouldn't the compression rather happen on top of
> mutliplexing layer?
>
>
> The situation is similar to compressing the payload in TCP packets. It
> is feasible, but in corner cases it can clash with MTU limits, cause
> re-fragmentation etc. The clean solution would be to layer compression
> on top of TCP layer.
>
> Mux want to control fragmentation while per-frame compression locks down
> fragmentation. Yes, it's not healthy to use of per-frame compression
> before mux.
>
> Shall we discourage this layering (preframe-compress then mux)?
>