Re: [hybi] Flow control quota

["Arman Djusupov" <[e-mail]>]

According to the WebSocket specification extensions are layered and should
be applied in a specific order. In this particular case the mux
specification provides two ways of applying the per frame compression
extension: compression can be applied either before or after the mux
extension. In the first case the frame being multiplexed is already
compressed and should not be fragmented. This allows intermediaries to
de-multiplex frames without decompressing them. In the second case the mux
frame is compressed along with the mux header, so it cannot be
de-multiplexed unless it is first decompressed. Supporting both options is
beneficial.

In any case the problem preventing  unfragmentable frames from being relayed
over flow-controlled logical connections should be resolved. The per frame
compression is not the only case when it might be impossible to fragment a
frame. A mux intermediary should either be able to control the size of the
frame that the sending side produces or should be able to fragment them.

Yes, the number of extensions may grow and some of them might be mutually
exclusive, that's why WebSocket supports negotiation of the extensions stack
so that each side can validate whether it can apply the requested set of
extensions in the given order. But I firmly believe that mux and per frame
compression are not mutually exclusive extensions.

With best regards,
Arman

-----Original Message-----
From: Martin Sustrik [mailto:[e-mail]] 
Sent: Monday, June 04, 2012 3:48 PM
To: Arman Djusupov
Cc: 'Takeshi Yoshino'; [e-mail]
Subject: Re: [hybi] Flow control quota

Hi Arman,

I am not following the hybi mailing list thoroughly, so I am not familiar
with the technical details... However, it seems that the problem here is
that it's possible to use any combination of extensions.

This pattern doesn't work well. The problem is that with the growing number
of extensions the number of possible combinations and (mis)interactions
between them is growing exponentially, so it's basically impossible to
deliver a fully functional stack.

The approach that standard Internet protocols take is separating the
functionality into layers where each layer can be filled by only a single
protocol at a time. E.g. if you are using TCP at L4 for a particular
connection you can't use SCTP at the same time.

I don't understand the details of WS, so I don't want to make any strong
proposals, but maybe separating the extensions into several well-defined
layers where each layer can be occupied only by a single extension instance
may help.

Martin

On 01/06/12 16:15, Arman Djusupov wrote:
> Unfortunately making extensions mutually aware of each other at the 
> implementation level does not prevent problems from being encountered 
> by a mux intermediary that receives a frame with an RSV bit set. An 
> intermediary is not permitted to fragment a frame with an RSV bit set 
> especially when the extension is unknown. If such a frame is larger 
> than the available quota then the intermediary would have to wait 
> until the quota becomes larger than the frame size before it can send 
> it. This will result in the frame never being sent if its size is 
> larger than the flow control window.
>
> A practical example of this problem is that a mux intermediary might 
> end up being unable to forward compressed frames larger than the flow 
> control window, unless it first decompresses them and refragments them.
>
> I am trying to figure out a general purpose solution that would allow 
> us to deal with non-fragmentable frames, even if the extension that is 
> applied to them is unknown. So far I have two ideas:
>
> 1.
>
> Allow the sender to send a frame with an RSV bit set even if it is 
> bigger that the available quota. In such a case the quota becomes 
> negative and from that moment on the sending side MUST wait until the 
> quota becomes positive before sending the subsequent frame. This way 
> the flow control would work and the sender would be able to send a 
> frame that is larger than the available quota but cannot be fragmented.
>
> Flow control is not intended to limit the maximum frame size, but 
> rather to prevent a logical channel from exclusively using a physical 
> connection. This solution would satisfy this requirement. A logical 
> channel would wait as long as necessary until the quota becomes 
> positive. This would let other channels use the physical connection.
> However, this solution would not prevent a sender from forcing the 
> receiver to buffer an amount of bytes that is a larger than the amount 
> it had agreed to buffer. But the amount of extra buffering required 
> would not exceed the size of a single frame so a mux intermediary 
> would at least be able to forward frames of reasonable size.
>
> 2.
>
> An alternative solution is to introduce a WebSocket fragmentation 
> extension. Using this extension, multiplexing intermediaries would 
> advertise the maximum frame size limit for non-mux clients. Non-mux 
> clients that want their connections to be multiplexed by an 
> intermediary would have to stick to the frame size limits required by 
> that intermediary. By supporting this extension WebSocket 
> client/browser vendors will be able to make their clients
mux-intermediary-friendly.
> This would help not only in non-mux-client-to-mux-intermediary 
> scenarios (and flow control in general) but would also permit us to 
> advertise a frame size limit in any scenario where such limits are 
> present. This was discussed by the WG but didn't made it into RFC 
> 6455, the idea at that time was that it could go in a separate 
> extension. So currently we have section 10.4 that requires 
> implementations to protect themselves against exceeding frame size 
> limits. A fragmentation extension would enable us to advertise these
limits.
>
> 10.4. Implementation-Specific Limits
>
> Implementations that have implementation- and/or platform-specific
>
> limitations regarding the frame size or total message size after
>
> reassembly from multiple frames MUST protect themselves against
>
> exceeding those limits. (For example, a malicious endpoint can try
>
> to exhaust its peer's memory or mount a denial-of-service attack by
>
> sending either a single big frame (e.g., of size 2**60) or by sending
>
> a long stream of small frames that are a part of a fragmented
>
> message.) Such an implementation SHOULD impose a limit on frame
>
> sizes and the total message size after reassembly from multiple
>
> frames.
>
> With best regards,
>
> Arman
>
> *From:*Takeshi Yoshino [mailto:[e-mail]]
> *Sent:* Friday, June 01, 2012 3:33 PM
> *To:* Martin Sustrik
> *Cc:* Arman Djusupov; [e-mail]
> *Subject:* Re: [hybi] Flow control quota
>
> On Fri, Jun 1, 2012 at 5:43 PM, Martin Sustrik <[e-mail] 
> <mailto:[e-mail]>> wrote:
>
> That seems to suggest there's some kind of mis-layering happening here.
> Frames are used for flow control. Does it make sense to use them for 
> compression as well? Shouldn't the compression rather happen on top of 
> mutliplexing layer?
>
>
> The situation is similar to compressing the payload in TCP packets. It 
> is feasible, but in corner cases it can clash with MTU limits, cause 
> re-fragmentation etc. The clean solution would be to layer compression 
> on top of TCP layer.
>
> Mux want to control fragmentation while per-frame compression locks 
> down fragmentation. Yes, it's not healthy to use of per-frame 
> compression before mux.
>
> Shall we discourage this layering (preframe-compress then mux)?
>