Re: WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP)

Van Catha <vans554@gmail.com> Thu, 24 November 2016 18:43 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 87F981296B6 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 24 Nov 2016 10:43:26 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.497
X-Spam-Level:
X-Spam-Status: No, score=-8.497 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-1.497, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id X3l77Qh55clY for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 24 Nov 2016 10:43:24 -0800 (PST)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1C54E12963C for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 24 Nov 2016 10:43:23 -0800 (PST)
Received: from lists by frink.w3.org with local (Exim 4.80) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1c9yvy-0002bC-PO for ietf-http-wg-dist@listhub.w3.org; Thu, 24 Nov 2016 18:39:46 +0000
Resent-Date: Thu, 24 Nov 2016 18:39:46 +0000
Resent-Message-Id: <E1c9yvy-0002bC-PO@frink.w3.org>
Received: from mimas.w3.org ([128.30.52.79]) by frink.w3.org with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <vans554@gmail.com>) id 1c9yvq-0002ZG-6B for ietf-http-wg@listhub.w3.org; Thu, 24 Nov 2016 18:39:38 +0000
Received: from mail-qk0-f176.google.com ([209.85.220.176]) by mimas.w3.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <vans554@gmail.com>) id 1c9yvj-0005Np-75 for ietf-http-wg@w3.org; Thu, 24 Nov 2016 18:39:32 +0000
Received: by mail-qk0-f176.google.com with SMTP id n204so57294040qke.2 for <ietf-http-wg@w3.org>; Thu, 24 Nov 2016 10:39:10 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=RImnhbb1Sbk2EKK6cQq/b7VJJGLYhzJVYykhCWwarEY=; b=jJQvsL4mwcITXQBmfu1lSA7YGzi0PTL50KUdXXqC9hDOUN5D9GEFSC/H0126fEL+ku NqTS2zADaTID1R7+M+5f1XXlsgIHWMg1G8s0cLgQRXBzd+SDqnT3WYmEgEV0H4t2vMn9 mTQvzIElaxd1Le1tgJj6Rk53PpNO4r2N5CO6V8CVlrFELE4SlMAy3DFDWyVad9Vai575 fs3GXwfTYJNhXzoGCncjvxg1KXa2EYkjYUVjSuTqm7AIhaUyIhtaq8clnbYNIgIx1Rp4 kT7FZBQx7LI8Gs5IfcEkfRFfCb2crmnciNniRNxQjz3Ew0UyZw7E4JB+hOpsPnxv+Y+h sLVg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=RImnhbb1Sbk2EKK6cQq/b7VJJGLYhzJVYykhCWwarEY=; b=LnK2i0RFp8Tvl84uayGYjGCKwN1HSMH4fhmorUw+BDJGNTVZCeVqLGbxtIJMxMeJXH iIEC3VMAAY2wEPwJhZ0ZqSw5vshcO0oL3n8YBuLOFLf4rERN6wTbKiZoDm9pJH27cSQu rKKr8/NJh2ygLRlbeIzlyKn8S5/B6cbwyYdW7an2KshlqfVpvkrcKCBPtkOX/FbI/ary O2MgonQHRzkJTVqU0C5weThbaYu9DntN7peP/4dCj/IDZJgvW8u5+6S+sl3uo1ty1Az8 ThA8tDCDPBmhSU2d0tKpsclZSaqvc5sPfO9PxmyTnVsZYVoMfMpSl4t9QJQNjtAggyXI ELnQ==
X-Gm-Message-State: AKaTC03H5Zl+FDryhUP/vy7SqMJjXk072PzmcJ4J17dm7z8x19C40i9t9FmeSjtwSAhRm9LKD+TtkgVqHhFwZw==
X-Received: by 10.55.92.199 with SMTP id q190mr3164396qkb.303.1480012744615; Thu, 24 Nov 2016 10:39:04 -0800 (PST)
MIME-Version: 1.0
Received: by 10.55.209.8 with HTTP; Thu, 24 Nov 2016 10:39:04 -0800 (PST)
In-Reply-To: <CAH9hSJbNk83FT0WqB1tHJvEfaU5CMoAaKRdvy8NTb4zgEUdzBw@mail.gmail.com>
References: <CAH9hSJZdBJ02+Z6o=aanZ=5PN=9VwyL1ZcX2jct-6f_FFivLGA@mail.gmail.com> <0f79ddf6-c455-c41a-f269-a1bdcef05b14@ninenines.eu> <CAH9hSJb2R9gv2vNqoyTjbMV4hZTYdpX2PoAoYgWUT1UuigLHRA@mail.gmail.com> <5541be74-afcc-6aef-404e-63acb2f608eb@ninenines.eu> <CAH9hSJarsNFqX1tAL7BZmZQwUrEQs1X3wtrAPuMyz8s-k_7WRg@mail.gmail.com> <43998e7b-9227-7562-b2c6-c08134065e22@ninenines.eu> <CAD3-0rPRPzVvYb6_Z4wDZp73L5Kyb7LmE0P5j4A-2VSRwT7FMw@mail.gmail.com> <CAH9hSJb=mWdHP8xcBis8-jhWgQTfN-cgQXVV3eCyT4U8JYQHZA@mail.gmail.com> <CAP8-FqnLaRvyQgXXkoNQPKcyMhv-O3RN67CMw5L_-1iQ9c6mhw@mail.gmail.com> <CAH9hSJYpsPW4S9n2LaaLTYYKB7wR3Sod2=fny2CZoUR7A0bSJA@mail.gmail.com> <CAP8-FqkOX1Sq6_=Sgb++QRiDWKEiOxAJ13kzMSr9heu-Ek3QmA@mail.gmail.com> <508f7085-b6b9-572e-7b0f-26cafc94dd44@ninenines.eu> <CAH9hSJZcGui08=DivN9vynKejvNFy+RYtRDYDnd6U6gxyX3UgQ@mail.gmail.com> <CAH9hSJZZCVMpQrpEV_JTceEmf2Y2aC_kJNXJmLW=LPebG+JR7g@mail.gmail.com> <CAP8-Fqk9SQJOuKWQmf5cRm9z2ja9wWUeG9xmivhiJf5O57Uryw@mail.gmail.com> <CAH9hSJZTVKx-8vg2xcqr_g4Bg+hc1ufvPZ2hZ+F=dXeVOdSu_Q@mail.gmail.com> <CAP8-Fqm=OVaOJ1imySM41_OuNu0D12Jby59dOpgqz-Bg4M+YOQ@mail.gmail.com> <CAD3-0rM35uXJnwfGay-1s9uw=-P71EubOkxFdKF=gjoXub8YXw@mail.gmail.com> <CAH9hSJZB0SyFiqLqLjd9R-T11yTa12Ekb-H8hYwfc6FeOjD2xQ@mail.gmail.com> <CAP8-FqmU+uBas5zH8oQHkt0zh18YrBm-O-umGPGMkLAjShw1Gw@mail.gmail.com> <CAH9hSJa10DLSozTpXjETyFX0bVYqfRbRFJnmFQNRGeSuZVKWPQ@mail.gmail.com> <CAG-EYChszHdWhp=o+fdOW+pAN90t61MExzsLnteM3tmf9=N0Yw@mail.gmail.com> <CAH9hSJbNk83FT0WqB1tHJvEfaU5CMoAaKRdvy8NTb4zgEUdzBw@mail.gmail.com>
From: Van Catha <vans554@gmail.com>
Date: Thu, 24 Nov 2016 13:39:04 -0500
Message-ID: <CAG-EYCjwptZcsHeDKwyRBhLTREEC4zxXxtTZvNLe2m1ei2r55g@mail.gmail.com>
To: Takeshi Yoshino <tyoshino@google.com>
Cc: "ietf-http-wg@w3.org Group" <ietf-http-wg@w3.org>, Wenbo Zhu <wenboz@google.com>, Martin Thomson <martin.thomson@gmail.com>
Content-Type: multipart/alternative; boundary="001a114e6506e7eb7505421051fb"
Received-SPF: pass client-ip=209.85.220.176; envelope-from=vans554@gmail.com; helo=mail-qk0-f176.google.com
X-W3C-Hub-Spam-Status: No, score=-5.0
X-W3C-Hub-Spam-Report: AWL=-1.240, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: mimas.w3.org 1c9yvj-0005Np-75 64a850da9e1c4b83fdc0ae2e49d88c39
X-Original-To: ietf-http-wg@w3.org
Subject: Re: WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP)
Archived-At: <http://www.w3.org/mid/CAG-EYCjwptZcsHeDKwyRBhLTREEC4zxXxtTZvNLe2m1ei2r55g@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/32998
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

Thanks for clarification. Unfortunate that so little attention was paid to
this.  Looks like some of us will be on HTTP1.1 for a long time.

On Mon, Nov 21, 2016 at 11:14 PM, Takeshi Yoshino <tyoshino@google.com>
wrote:

> Ah, no. Martin just warned us that we might face the same issue that SSE
> faced.
>
> Mark's suggestion is a separate thing. The co-chairs (Mark and Patrick)
> said that this (WiSH) doesn't seems to be a topic that should be discussed
> in the HTTP WG given the charter of the WG, I think.
>
> On Sun, Nov 20, 2016 at 12:26 PM, Van Catha <vans554@gmail.com> wrote:
>
>> I do not understand what this means.  Is the suggestion to ignore WiSH
>> for now in favor of SSE?
>>
>> On Fri, Nov 18, 2016 at 1:55 AM, Takeshi Yoshino <tyoshino@google.com>
>> wrote:
>>
>>> I'd like to share the feedback on WiSH from IETF 97.
>>>
>>> ----
>>>
>>> Due to limited time, I got just one on-site comment from Martin about
>>> comparison with Server-sent event (EventSource).
>>>
>>> As mentioned in the I-D, yes, this is kinda full-duplex SSE with the WS
>>> framing, and it might suffer from unexpected buffering by intermediaries if
>>> any as Martin said.
>>>
>>> WiSH should work well for deployment with TLS only (possibly with some
>>> non-TLS part beyond server side front-end but are under control of the
>>> service providers). Given the wide trend of encouraging TLS and browser
>>> vendors' implementation status of H2, I think we should prioritize layering
>>> simplicity than taking care of gain of WiSH/H2/TCP + transparent proxy
>>> (with unexpected buffering) case. For H2-less TLS-less environment, we can
>>> just use the WebSocket protocol.
>>>
>>> There can still be some risk of MITM (trusted) proxy and unexpected
>>> buffering with AntiVirus/Firewall for deployment with TLS, but other
>>> WebSocket/H2 mapping proposals also have issues of possible blocking,
>>> buffering, etc. WebSocket/TCP's handshake success rate for non-TLS port 80
>>> was also not so good when it started getting deployed, and got improved
>>> gradually. I think the problems will get resolved once WiSH is accepted
>>> widely, and I believe the total pain and cost would be smaller.
>>>
>>> ----
>>>
>>> Mark suggested that we should find some other right place than HTTP WG.
>>> I'll discuss this with Mark. Maybe we'll consult the DISPATCH WG.
>>>
>>> ----
>>>
>>> Thanks everyone for the feedback.
>>>
>>> Takeshi
>>>
>>> On Thu, Nov 3, 2016 at 3:20 AM, Costin Manolache <costin@gmail.com>
>>> wrote:
>>>
>>>> Good timing -  http://httpwg.org/http-exte
>>>> nsions/encryption-preview.html is addressing my concerns for
>>>> webpush ( and general 'encrypted content' ), we're still discussing
>>>> some details, but for this use
>>>> case metadata won't be needed.
>>>>
>>>> Costin
>>>>
>>>>
>>>> On Tue, Nov 1, 2016 at 10:34 PM Takeshi Yoshino <tyoshino@google.com>
>>>> wrote:
>>>>
>>>>> On Mon, Oct 31, 2016 at 5:57 AM, Wenbo Zhu <wenboz@google.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Oct 30, 2016 at 10:25 AM, Costin Manolache <costin@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Thanks for the answer and pointers. From earlier responses, it seems
>>>>> possible to use GET
>>>>> or a non-web-stream request to would avoid the extra cost of the
>>>>> pre-flight.
>>>>>
>>>>>
>>>>>
>>>>> Yeah, at least the Content-Type in the HTTP request gets eliminated.
>>>>>
>>>>>
>>>>> One more question/issue: in some cases it would be good to send some
>>>>> metadata (headers) along with binary frames. For example in webpush
>>>>> the content is an encrypted
>>>>> blob, and needs headers for the key/salt. I would assume a lot of
>>>>> other 'binary' messages would
>>>>> benefit if simple metadata could be sent along. Would it be possible
>>>>> to use one of the reserved
>>>>> bits for 'has metadata' and add some encoded headers ? I know in
>>>>> websocket they are intended
>>>>> for 'extensions', but 'headers' seems a very common use case.
>>>>>
>>>>> Q about webpush: is the metadata different for each binary message?
>>>>>
>>>>> We discussed about metadata and how to use one of RSV bits etc. For
>>>>> the current version, let's make sure the WS compatibility is fully
>>>>> addressed (with minimum wire encoding like WiSH)
>>>>>
>>>>>
>>>>> Agreed. Let's discuss the metadata needs separately. I agree it's
>>>>> important.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Having the binary frame use some MIME encoding to pass both text
>>>>> headers and the binary blob
>>>>> is possible - but has complexity and overhead.
>>>>>
>>>>> OTOH, if the binary blob relies on text headers (metata) to be useful,
>>>>> then you probably need define a dedicated MIME encoding.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Costin
>>>>>
>>>>> On Sun, Oct 30, 2016 at 5:27 AM Takeshi Yoshino <tyoshino@google.com>
>>>>> wrote:
>>>>>
>>>>> Thanks, Van, Costin.
>>>>>
>>>>> On Sun, Oct 30, 2016 at 2:43 AM, Costin Manolache <costin@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Good point - websocket is widely deployed, including IoT - and the
>>>>> header is pretty easy to handle anyways.
>>>>> +1.
>>>>>
>>>>> One question: is this intended to be handled by browsers, and exposed
>>>>> using the W3C websocket API ?
>>>>> Will a regular app be able to make WiSH requests and parse the stream
>>>>> by itself, without browser
>>>>> interference ? And if yes, any advice on how it interact with CORS ?
>>>>>
>>>>>
>>>>> The first step would be using Streams based upload/download via the
>>>>> Fetch API + protocol processing in JS.
>>>>>
>>>>> The next step could be either introduction of an optimized native
>>>>> implementation of WiSH parser/framer in the form of the TransformStream
>>>>> which can be used as follows:
>>>>>
>>>>> const responsePromise = fetch(url, init);
>>>>> responsePromise.then(response => {
>>>>>   const wishStream = response.body().pipeThrough(wishTransformStream);
>>>>>   function readAndProcessMessage() {
>>>>>     const readPromise = wishStream.read();
>>>>>     readPromise.then(result => {
>>>>>       if (result.done) {
>>>>>         // End of stream.
>>>>>         return;
>>>>>       }
>>>>>
>>>>>       const message = result.value;
>>>>>       // Process the message
>>>>>       // E.g. access message.opcode for opcode, message.body for the
>>>>> body data
>>>>>       readAndProcessMessage();
>>>>>     });
>>>>>   }
>>>>>   readAndProcessMessage();
>>>>> });
>>>>>
>>>>> and provide a polyfill that presents this as the WebSocket API, and
>>>>> (or skip the step and) go further i.e. native implementation for everything
>>>>> if it turns out optimization is critical.
>>>>>
>>>>> We need to discuss this also in W3C/WHATWG.
>>>>>
>>>>> Regarding CORS, if the request includes non CORS-safelisted headers,
>>>>> fetch() based JS polyfills will be basically subject to the CORS preflight
>>>>> requirement. We could try to exempt some of well defined headers if any for
>>>>> CORS like WebSocket handshake's headers and server-sent event's
>>>>> Last-Event-Id are exempted. Regarding the proposed subprotocol negotiation
>>>>> in the form of combination of the Accept header and the Content-Type
>>>>> header, the Accept header is one of the CORS-safelisted headers, so it's
>>>>> not a problem. The Content-Type header is considered to be
>>>>> non-CORS-safelisted if it's value is none of the CORS-safelisted media
>>>>> types. So, WiSH media type would trigger the preflight unless we exclude it.
>>>>>
>>>>> Origin policy https://wicg.github.io/origin-policy/ might also help.
>>>>>
>>>>>
>>>>>
>>>>> Costin
>>>>>
>>>>> On Fri, Oct 28, 2016 at 12:06 PM Takeshi Yoshino <tyoshino@google.com>
>>>>> wrote:
>>>>>
>>>>> Sorry for being ambivalent.
>>>>>
>>>>> We can of course revisit each design decision we made for RFC 6455
>>>>> framing and search for the optimal again. But as:
>>>>> - one of the main philosophies behind WiSH is compatibility with
>>>>> WebSocket in terms of both spec and implementation
>>>>> - the WebSocket is widely deployed and therefore we have a lot of
>>>>> implementations in various languages/platform
>>>>> - most browsers already have logic for the framing
>>>>> - the framing is not considered to be so big pain
>>>>> inheriting the WebSocket framing almost as-is is just good enough.
>>>>> Basically, I'm leaning toward this plan.
>>>>>
>>>>> Takeshi
>>>>>
>>>>> On Sat, Oct 29, 2016 at 3:12 AM, Takeshi Yoshino <tyoshino@google.com>
>>>>> wrote:
>>>>>
>>>>> On Sat, Oct 29, 2016 at 2:55 AM, Loïc Hoguin <essen@ninenines.eu>
>>>>> wrote:
>>>>>
>>>>> On 10/28/2016 08:41 PM, Costin Manolache wrote:
>>>>>
>>>>> Current overhead is 2 bytes if frame is up to 125 bytes long - which I
>>>>> think it's not very common,
>>>>> 4 bytes for up to 64k, and 10 bytes for anything larger.
>>>>> IMHO adding one byte - i.e. making it fixed 5-byte, with first as is,
>>>>> and next 4 fixed length would
>>>>> be easiest to parse.
>>>>>
>>>>>
>>>>> Is making it easy (or easier) to parse even a concern anymore?
>>>>>
>>>>> Considering the number of agents and servers already supporting
>>>>> Websocket, the numerous libraries for nearly all languages and the great
>>>>> autobahntestsuite project validating it all, reusing the existing code is a
>>>>> very sensible solution.
>>>>>
>>>>>
>>>>> Yeah, I've been having similar feeling regarding cost for
>>>>> parser/encoder implementation though I might be biased.
>>>>>
>>>>>
>>>>> There are obviously too many options to encode and each has benefits -
>>>>> my only concern was
>>>>> that the choice of 1, 2, 8 bytes for length may not match common sizes.
>>>>>
>>>>> ( in webpush frames will be <4k ).
>>>>>
>>>>>
>>>>> --
>>>>> Loïc Hoguin
>>>>> https://ninenines.eu
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>
>