Re: WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP)

Takeshi Yoshino <tyoshino@google.com> Sun, 30 October 2016 12:32 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F80C1294B1 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Sun, 30 Oct 2016 05:32:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.997
X-Spam-Level:
X-Spam-Status: No, score=-7.997 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_SORBS_SPAM=0.5, RP_MATCHES_RCVD=-1.497, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bkrUG_8w-tk1 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Sun, 30 Oct 2016 05:32:44 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E7ED8127076 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Sun, 30 Oct 2016 05:32:43 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.80) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1c0pDv-0002Iy-7j for ietf-http-wg-dist@listhub.w3.org; Sun, 30 Oct 2016 12:28:27 +0000
Resent-Date: Sun, 30 Oct 2016 12:28:27 +0000
Resent-Message-Id: <E1c0pDv-0002Iy-7j@frink.w3.org>
Received: from mimas.w3.org ([128.30.52.79]) by frink.w3.org with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <tyoshino@google.com>) id 1c0pDq-0002ID-4g for ietf-http-wg@listhub.w3.org; Sun, 30 Oct 2016 12:28:22 +0000
Received: from mail-it0-f50.google.com ([209.85.214.50]) by mimas.w3.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <tyoshino@google.com>) id 1c0pDj-0002eB-S6 for ietf-http-wg@w3.org; Sun, 30 Oct 2016 12:28:16 +0000
Received: by mail-it0-f50.google.com with SMTP id u205so56945901itc.0 for <ietf-http-wg@w3.org>; Sun, 30 Oct 2016 05:27:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=81wJGd0ItcxZ6vvBzvsRuY+oMuAxlYy0YepO58RObSY=; b=PXuxdQbJcy6OdsbCWp2mpQPx45392kslvGFW632Ph05cXKfH3dx+Cq6G6HjSJsaHJ3 rGhpcu63YODbdxiK2YbAi/GlGXQGjX/0bHmYv9Q0a6STns9xlMnnE2kRkqKrxPFM9wT4 S84xB9GgiacggloFGH0Y/kcxDHQyOsicQrnuIyxKzNlyC5QfO1z3TA9HD2iKAg/EDGcs wpRTG9oHyRB9CumOkJQdGypSOfjBPdyUcvhg28LXmkKCf/JVuudwMlJk0ljPv4nqL/g5 UCAFzpgtcJ/IlDUOPhee2WMdrMxg6FKZHKilCrCqzvOgqHG/cE3bZPHu+chA3kmNWf7v QVkA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=81wJGd0ItcxZ6vvBzvsRuY+oMuAxlYy0YepO58RObSY=; b=KQNXibGOULzZtMOjQ0y2PVc39STQNExKyhxT8BdEmk8NPMdnUCwQvCbB4daW4JrlN1 5TRNGE54Z5qtbTKOpoiCmHj1m4fmlPMg+b7elfst2haW2p625ZuqYLtXl/Z4QNzayqm7 +obRmSphgfRoie6cyWb2x07OBg0ZtMRm5MZTZoAfkfCgkrEf3M1iNYVggMDdo69s+h7x w+tUEvafZPgYgmXDZpBf0sdzodJqmgWQTYAdNcurY7+k0qtKgg1E9/xIYabjssxZ8DDm 6/saPIcb4PeIwbfdItM/dkfX6TYV9TYg1wuNVs2lxHM7LeYLjCmXcQ7y0DGuDCqH4mSl T7aA==
X-Gm-Message-State: ABUngvfnITMPvN6WdXIW2P90mZDmM1JmPprpnVUg4l3Jm7dfUvgQTVOx9tkMKrUoQFLVR29Zjg4f1ZUJJmQIW3gG
X-Received: by 10.107.148.4 with SMTP id w4mr17198853iod.135.1477830469103; Sun, 30 Oct 2016 05:27:49 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.79.113.68 with HTTP; Sun, 30 Oct 2016 05:27:28 -0700 (PDT)
In-Reply-To: <CAP8-Fqk9SQJOuKWQmf5cRm9z2ja9wWUeG9xmivhiJf5O57Uryw@mail.gmail.com>
References: <CAH9hSJZdBJ02+Z6o=aanZ=5PN=9VwyL1ZcX2jct-6f_FFivLGA@mail.gmail.com> <0f79ddf6-c455-c41a-f269-a1bdcef05b14@ninenines.eu> <CAH9hSJb2R9gv2vNqoyTjbMV4hZTYdpX2PoAoYgWUT1UuigLHRA@mail.gmail.com> <5541be74-afcc-6aef-404e-63acb2f608eb@ninenines.eu> <CAH9hSJarsNFqX1tAL7BZmZQwUrEQs1X3wtrAPuMyz8s-k_7WRg@mail.gmail.com> <43998e7b-9227-7562-b2c6-c08134065e22@ninenines.eu> <CAD3-0rPRPzVvYb6_Z4wDZp73L5Kyb7LmE0P5j4A-2VSRwT7FMw@mail.gmail.com> <CAH9hSJb=mWdHP8xcBis8-jhWgQTfN-cgQXVV3eCyT4U8JYQHZA@mail.gmail.com> <CAP8-FqnLaRvyQgXXkoNQPKcyMhv-O3RN67CMw5L_-1iQ9c6mhw@mail.gmail.com> <CAH9hSJYpsPW4S9n2LaaLTYYKB7wR3Sod2=fny2CZoUR7A0bSJA@mail.gmail.com> <CAP8-FqkOX1Sq6_=Sgb++QRiDWKEiOxAJ13kzMSr9heu-Ek3QmA@mail.gmail.com> <508f7085-b6b9-572e-7b0f-26cafc94dd44@ninenines.eu> <CAH9hSJZcGui08=DivN9vynKejvNFy+RYtRDYDnd6U6gxyX3UgQ@mail.gmail.com> <CAH9hSJZZCVMpQrpEV_JTceEmf2Y2aC_kJNXJmLW=LPebG+JR7g@mail.gmail.com> <CAP8-Fqk9SQJOuKWQmf5cRm9z2ja9wWUeG9xmivhiJf5O57Uryw@mail.gmail.com>
From: Takeshi Yoshino <tyoshino@google.com>
Date: Sun, 30 Oct 2016 21:27:28 +0900
Message-ID: <CAH9hSJZTVKx-8vg2xcqr_g4Bg+hc1ufvPZ2hZ+F=dXeVOdSu_Q@mail.gmail.com>
To: Costin Manolache <costin@gmail.com>
Cc: =?UTF-8?B?TG/Dr2MgSG9ndWlu?= <essen@ninenines.eu>, Wenbo Zhu <wenboz@google.com>, "ietf-http-wg@w3.org Group" <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary=001a113fe122267cf3054014386b
Received-SPF: pass client-ip=209.85.214.50; envelope-from=tyoshino@google.com; helo=mail-it0-f50.google.com
X-W3C-Hub-Spam-Status: No, score=-7.4
X-W3C-Hub-Spam-Report: AWL=1.507, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, RP_MATCHES_RCVD=-2.656, SPF_PASS=-0.001, W3C_AA=-1, W3C_DB=-1, W3C_IRA=-1, W3C_WL=-1
X-W3C-Scan-Sig: mimas.w3.org 1c0pDj-0002eB-S6 aeb85160d12931dd327d86cc98a05017
X-Original-To: ietf-http-wg@w3.org
Subject: Re: WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP)
Archived-At: <http://www.w3.org/mid/CAH9hSJZTVKx-8vg2xcqr_g4Bg+hc1ufvPZ2hZ+F=dXeVOdSu_Q@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/32736
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

Thanks, Van, Costin.

On Sun, Oct 30, 2016 at 2:43 AM, Costin Manolache <costin@gmail.com> wrote:

> Good point - websocket is widely deployed, including IoT - and the header
> is pretty easy to handle anyways.
> +1.
>
> One question: is this intended to be handled by browsers, and exposed
> using the W3C websocket API ?
> Will a regular app be able to make WiSH requests and parse the stream by
> itself, without browser
> interference ? And if yes, any advice on how it interact with CORS ?
>

The first step would be using Streams based upload/download via the Fetch
API + protocol processing in JS.

The next step could be either introduction of an optimized native
implementation of WiSH parser/framer in the form of the TransformStream
which can be used as follows:

const responsePromise = fetch(url, init);
responsePromise.then(response => {
  const wishStream = response.body().pipeThrough(wishTransformStream);
  function readAndProcessMessage() {
    const readPromise = wishStream.read();
    readPromise.then(result => {
      if (result.done) {
        // End of stream.
        return;
      }

      const message = result.value;
      // Process the message
      // E.g. access message.opcode for opcode, message.body for the body
data
      readAndProcessMessage();
    });
  }
  readAndProcessMessage();
});

and provide a polyfill that presents this as the WebSocket API, and (or
skip the step and) go further i.e. native implementation for everything if
it turns out optimization is critical.

We need to discuss this also in W3C/WHATWG.

Regarding CORS, if the request includes non CORS-safelisted headers,
fetch() based JS polyfills will be basically subject to the CORS preflight
requirement. We could try to exempt some of well defined headers if any for
CORS like WebSocket handshake's headers and server-sent event's
Last-Event-Id are exempted. Regarding the proposed subprotocol negotiation
in the form of combination of the Accept header and the Content-Type
header, the Accept header is one of the CORS-safelisted headers, so it's
not a problem. The Content-Type header is considered to be
non-CORS-safelisted if it's value is none of the CORS-safelisted media
types. So, WiSH media type would trigger the preflight unless we exclude it.

Origin policy https://wicg.github.io/origin-policy/ might also help.


>
> Costin
>
> On Fri, Oct 28, 2016 at 12:06 PM Takeshi Yoshino <tyoshino@google.com>
> wrote:
>
>> Sorry for being ambivalent.
>>
>> We can of course revisit each design decision we made for RFC 6455
>> framing and search for the optimal again. But as:
>> - one of the main philosophies behind WiSH is compatibility with
>> WebSocket in terms of both spec and implementation
>> - the WebSocket is widely deployed and therefore we have a lot of
>> implementations in various languages/platform
>> - most browsers already have logic for the framing
>> - the framing is not considered to be so big pain
>> inheriting the WebSocket framing almost as-is is just good enough.
>> Basically, I'm leaning toward this plan.
>>
>> Takeshi
>>
>> On Sat, Oct 29, 2016 at 3:12 AM, Takeshi Yoshino <tyoshino@google.com>
>> wrote:
>>
>> On Sat, Oct 29, 2016 at 2:55 AM, Loïc Hoguin <essen@ninenines.eu> wrote:
>>
>> On 10/28/2016 08:41 PM, Costin Manolache wrote:
>>
>> Current overhead is 2 bytes if frame is up to 125 bytes long - which I
>> think it's not very common,
>> 4 bytes for up to 64k, and 10 bytes for anything larger.
>> IMHO adding one byte - i.e. making it fixed 5-byte, with first as is,
>> and next 4 fixed length would
>> be easiest to parse.
>>
>>
>> Is making it easy (or easier) to parse even a concern anymore?
>>
>> Considering the number of agents and servers already supporting
>> Websocket, the numerous libraries for nearly all languages and the great
>> autobahntestsuite project validating it all, reusing the existing code is a
>> very sensible solution.
>>
>>
>> Yeah, I've been having similar feeling regarding cost for parser/encoder
>> implementation though I might be biased.
>>
>>
>> There are obviously too many options to encode and each has benefits -
>> my only concern was
>> that the choice of 1, 2, 8 bytes for length may not match common sizes.
>>
>> ( in webpush frames will be <4k ).
>>
>>
>> --
>> Loïc Hoguin
>> https://ninenines.eu
>>
>>
>>
>>