Re: [hybi] Is it important to know frame length at the start of frame? (was: Re: Discontinuation of mux ...)

Joakim Erdfelt <joakim@intalio.com> Wed, 12 February 2014 16:20 UTC

Return-Path: <joakim@intalio.com>
X-Original-To: hybi@ietfa.amsl.com
Delivered-To: hybi@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 343AF1A040E for <hybi@ietfa.amsl.com>; Wed, 12 Feb 2014 08:20:02 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.977
X-Spam-Level:
X-Spam-Status: No, score=-1.977 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MPN-4ftcPoTH for <hybi@ietfa.amsl.com>; Wed, 12 Feb 2014 08:19:57 -0800 (PST)
Received: from mail-ea0-f182.google.com (mail-ea0-f182.google.com [209.85.215.182]) by ietfa.amsl.com (Postfix) with ESMTP id B0F661A0465 for <hybi@ietf.org>; Wed, 12 Feb 2014 08:19:56 -0800 (PST)
Received: by mail-ea0-f182.google.com with SMTP id r15so4547516ead.27 for <hybi@ietf.org>; Wed, 12 Feb 2014 08:19:55 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=znXBQF/eY2n9tfI9oxlql7b5i2aN4uLEkKF5bg3KssA=; b=AH6bbC8gSAS0IUzU0R6Xm2ygocYsBDi7fg9ARVkEUtWsffv+uzZF6uV5PfNmaf3zly bq98W9fV7nWfgybBlA6Angr8xC1OLkstuhXpdHn2B5TR6YsABHFdz5ocug6+7YZZfkHx 7fonH1Vs1rkNP1TkdVEITQytmqUzfUCL44nZ8QYG0euQisakiVV49GILMEQSPCB3p7gd EUsTZ7YM7ky58Y4ctH4SybXVD5W17ul9tp8S9zbg3+xlJvtyE78cftoIXdptFJaTROOd RPhVWDNVfODRgL+olXXlZU1i/BRZzKvKinOoiqOmHpbHCUTBLD86MxZGFw+g44L1h1cU 6Kgg==
X-Gm-Message-State: ALoCoQmCVjqyr3eomaFCfgSXfNCMQvGuzMDACO0U8TeUkvYY+0/G/w45bXu4LxcYeCCdheg3iu0V
MIME-Version: 1.0
X-Received: by 10.14.119.197 with SMTP id n45mr2011528eeh.93.1392221995413; Wed, 12 Feb 2014 08:19:55 -0800 (PST)
Received: by 10.14.152.202 with HTTP; Wed, 12 Feb 2014 08:19:55 -0800 (PST)
In-Reply-To: <CAG4zZZCr4aTfVpw2coX2g0qw++4kdgNCFVze6tHKZ+fJNqb0aQ@mail.gmail.com>
References: <CAH9hSJbf_ABT7ECL9eS=_ADrncX8qBtxZv=uLcdu9_6GUv23Uw@mail.gmail.com> <CACuKZqEcA1Pv8RpWfmThMjTzi2BbVMMKXqujs6BxVfxRPZJ9NQ@mail.gmail.com> <CAG4zZZCr4aTfVpw2coX2g0qw++4kdgNCFVze6tHKZ+fJNqb0aQ@mail.gmail.com>
Date: Wed, 12 Feb 2014 09:19:55 -0700
Message-ID: <CAG4zZZA4WR50Ea2e36XEgTgpJURwiMFn+DXeYuxHFfZatYzDOQ@mail.gmail.com>
From: Joakim Erdfelt <joakim@intalio.com>
To: Zhong Yu <zhong.j.yu@gmail.com>
Content-Type: multipart/alternative; boundary="001a1132f7f87c49be04f237f136"
Cc: "hybi@ietf.org" <hybi@ietf.org>
Subject: Re: [hybi] Is it important to know frame length at the start of frame? (was: Re: Discontinuation of mux ...)
X-BeenThere: hybi@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Server-Initiated HTTP <hybi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/hybi>, <mailto:hybi-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/hybi/>
List-Post: <mailto:hybi@ietf.org>
List-Help: <mailto:hybi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Feb 2014 16:20:02 -0000

Eliminating the size from the frame also eliminates the ability to
fast-fail on excessive messages.

Scenario (with current RFC6455 / WS+HTTP/1.1):
  Sending side starts to write out a single message of 100MB.
  Receiving side, using a whole message delivery based API is configured
for max messages of 20MB.
  Receiving side sees the 100MB frame size and automatically closes the
connection with 1009 and doesn't bother reading past the current network
buffer.

Scenario (without frame sizes):
  Sending side starts to write out a single message of 100MB.
  Receiving side, using a whole message delivery based API is configured
for max messages of 20MB.
  Receiving side reads till the 20MB mark, then fails the connection with
1009.  Wasting time, memory, and resources for no good reason.

I really don't see how a discussion of a protocol without sizes is even
going on.
I mean, even the IP layer and TCP layer have sizes for their frames.



--
Joakim Erdfelt <joakim@intalio.com>
webtide.com <http://www.webtide.com/> - intalio.com/jetty
Expert advice, services and support from from the Jetty & CometD experts
eclipse.org/jetty - cometd.org


On Wed, Feb 12, 2014 at 9:13 AM, Joakim Erdfelt <joakim@intalio.com> wrote:

> Scenario:
>   Sending side uses a message stream based API to write out 1 websocket
> message.
>   The sending side's API is configured/implemented to buffer only 16k at a
> time, writing out a frame of 16k for each buffer filled.
>   The sending side writes out a 600k message.
>   The receiving side is using a different API, one that is based on
> complete messages.
>   The receiving side is configured to not accept messages over 100k.
>   The receiving side's API/implementation tosses a close code 1009 to the
> sending side, per API.
>   The receiving side's endpoint receives an error and close notification
> indicating a protocol closure due to excessive message size from sending
> side.
>
>
>
> --
> Joakim Erdfelt <joakim@intalio.com>
> webtide.com <http://www.webtide.com/> - intalio.com/jetty
> Expert advice, services and support from from the Jetty & CometD experts
> eclipse.org/jetty - cometd.org
>
>
> On Wed, Feb 12, 2014 at 9:00 AM, Zhong Yu <zhong.j.yu@gmail.com> wrote:
>
>> Why not send the message in one frame? The frame length is then the
>> message length.
>>
>> One argument for fragmentation is to insert control frames while
>> transmitting a long message. However that reasoning is not very
>> convincing:
>>
>> PONG - as long as we are writing data, any data, the peer should see
>> that the connection is alive. It's unreasonable for the peer to think
>> that our end is unresponsive while it is receiving data from us.
>>
>> PING - not necessary while we are writing data. If peer is
>> unresponsive, we'll learn that from TCP layer.
>>
>> CLOSE - closing the connection in the middle of a message is abnormal
>> anyway, it's probably not important to do the close handshake.
>>
>> Zhong Yu
>>
>>
>> On Fri, Feb 7, 2014 at 1:30 AM, Takeshi Yoshino <tyoshino@google.com>
>> wrote:
>> > I remember that in early days of the standardization, somebody was
>> proposing
>> > (*) that we should have a message length field in the first fragment to
>> > allow for allocating buffer for full message at the beginning.
>> >
>> > On Thu, Jan 16, 2014 at 3:27 PM, "Martin J. Dürst" <
>> duerst@it.aoyama.ac.jp>
>> > wrote:
>> >>
>> >> On 2014/01/16 15:05, Yutaka Hirano wrote:
>> >>>
>> >>> Hi Joakim,
>> >>>
>> >>> Thanks for the comment.
>> >>>
>> >>> Knowing the size of a frame at the start of the frame, traditional
>> >>>>
>> >>>> websocket behavior, is desired, as it allows us to fast fail for the
>> >>>> 1009
>> >>>> (message too big) use case.
>> >>>
>> >>>
>> >>> Do you know how effective it is?
>> >>>
>> >>> Having no prior knowledge of frame size is undesirable to us.
>> >>>
>> >>> On the other hand, the frame sender have more freedom because they
>> don't
>> >>> have to know the size of the frame in front.
>> >>
>> >>
>> >> My understanding of why messages are split into frames was that this
>> would
>> >> allow to start a message without knowing its length. But if you don't
>> know
>> >> the size of a frame before sending it, then the right solution is to
>> make it
>> >> smaller. Frames are just arbitrary pieces of a message.
>> >
>> >
>> > We should be clear if we want to make it able to
>> > (a) know the length of WebSocket frame on receiving the header of each
>> > fragment
>> > (b) know the length of WebSocket message on receiving the first fragment
>> >
>> > As the proposal (*) has been rejected and not in the RFC6455 header,
>> (b) is
>> > not true even in RFC6455. What Joakim said should be to keep (a) which
>> is
>> > true now in RFC6455.
>> >
>> > But if a message is fragmented into smaller frames, we cannot reject
>> (and
>> > send 1009) the message on receiving the first frame. "Whether we can
>> > fast-reject a big message on receiving some non-final frame" depends on
>> how
>> > the peer fragments a message.
>> >
>> > In some plans discussed in the main thread, it's planned to drop frame
>> > length info for efficiency. But something else such as the length field
>> of
>> > HTTP/2.0 DATA frame tells how much data the fragment (when multiple
>> DATAs
>> > convey a WebSocket frame, each DATA would be called fragments of a
>> frame)
>> > contains. So, we won't be able to know the length of fragments
>> > (original/encapsulated WebSocket frames) on receiving each of them, but
>> > still be able to know the length of sub-fragments (e.g. HTTP/2.0 data
>> > frames).
>> >
>> > I think this change is acceptable and shouldn't be considered as
>> breaking of
>> > RFC6455 semantics. Even in the mux extension I was editing
>> > (http://tools.ietf.org/html/draft-ietf-hybi-websocket-multiplexing-11),
>> I
>> > made the same change. The length of the encapsulated frame is unknown
>> until
>> > receiving all the fragments of the encapsulating message. No one was
>> > objecting to that when we were discussing the spec, IIRC.
>> >
>> >
>> > _______________________________________________
>> > hybi mailing list
>> > hybi@ietf.org
>> > https://www.ietf.org/mailman/listinfo/hybi
>> >
>>
>> _______________________________________________
>> hybi mailing list
>> hybi@ietf.org
>> https://www.ietf.org/mailman/listinfo/hybi
>>
>
>