Re: WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP)
Costin Manolache <costin@gmail.com> Wed, 02 November 2016 18:25 UTC
Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AC2D8129B1A for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 2 Nov 2016 11:25:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.497
X-Spam-Level:
X-Spam-Status: No, score=-8.497 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-1.497, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rRYNvpWV7e8e for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 2 Nov 2016 11:25:34 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DA107129B30 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Wed, 2 Nov 2016 11:25:23 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.80) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1c209k-0007Wf-7U for ietf-http-wg-dist@listhub.w3.org; Wed, 02 Nov 2016 18:21:00 +0000
Resent-Date: Wed, 02 Nov 2016 18:21:00 +0000
Resent-Message-Id: <E1c209k-0007Wf-7U@frink.w3.org>
Received: from titan.w3.org ([128.30.52.76]) by frink.w3.org with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <costin@gmail.com>) id 1c209e-0007Vq-4V for ietf-http-wg@listhub.w3.org; Wed, 02 Nov 2016 18:20:54 +0000
Received: from mail-yw0-f175.google.com ([209.85.161.175]) by titan.w3.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <costin@gmail.com>) id 1c209X-0001mM-Gh for ietf-http-wg@w3.org; Wed, 02 Nov 2016 18:20:48 +0000
Received: by mail-yw0-f175.google.com with SMTP id h14so17878534ywa.2 for <ietf-http-wg@w3.org>; Wed, 02 Nov 2016 11:20:27 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=vrtoI12F6sqLzio+eq7//hWRYWANT8oBWCE4TjFVzZ0=; b=TPMZT7NXqYornuiewAfyOFs0YL12ssFtxFVIyRE+FDrClfj9ITWtVUseKsqmhNlQHW vTPrd03UXuvu+54kN0GFHLQNvgVvY0Q2VMybFRHSeoDEzs1sVv4e4gozrrzBvPQkhO3+ vtxoLjLL025qA2nqbVquCERbSe2W6ESFTWORJgFm5xBiDPyfJUQlBFZvTK1VaFuvBhk/ 7PYRGFDmR9jxAjxPjbXpKakCgAD0Dh4dc0cHUBY4k3iYw7QpCrczFrF/7xn3l7b1EleG d2Y5dIyKMBaDa6cdqgSNPpykGsqPlRYjcBIMBhdMyKpdIQqo5BSUrTO/F4nO0mpbtOh9 JVOQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=vrtoI12F6sqLzio+eq7//hWRYWANT8oBWCE4TjFVzZ0=; b=UpCemwSssDQCk938ktG3aWPhdy24DoElRqu1E0o5QJWE7xHZkmQARIUp9vUnLhJlso byrmy85cwG/gMwCExZj/R8vHJvvuvvBSSSMtjZwf/CLpui8nLVwLkqKw++R+gzhy2jx+ fw2uHuAbZGITED8lNr+lgRWeW0BEovkPH6Z+R3bY0e/GOpy00Jky9QBXkKDXwzRjwpmY dvBQ7lipAeFVMxkF7RNr69nmB3jSll87SvGQxuKJplK7Di5MBrTCfDv807TW2n+MjQli mFRPwyvp0aSqsJ0Y1dXQ54Ew0zoCrthYTQ+CzWSZ1/vKUK0Dnr89Da6p6KC0tRvlhrVq vzRg==
X-Gm-Message-State: ABUngvdTlcuoegFDR07zm3zZFWQLdO/XTD5NM7gCiVydgSnYB59reRIk4+TTRCJGsEb3WEtlhdqBKQc2KMrzug==
X-Received: by 10.107.183.148 with SMTP id h142mr5623994iof.190.1478110821615; Wed, 02 Nov 2016 11:20:21 -0700 (PDT)
MIME-Version: 1.0
References: <CAH9hSJZdBJ02+Z6o=aanZ=5PN=9VwyL1ZcX2jct-6f_FFivLGA@mail.gmail.com> <0f79ddf6-c455-c41a-f269-a1bdcef05b14@ninenines.eu> <CAH9hSJb2R9gv2vNqoyTjbMV4hZTYdpX2PoAoYgWUT1UuigLHRA@mail.gmail.com> <5541be74-afcc-6aef-404e-63acb2f608eb@ninenines.eu> <CAH9hSJarsNFqX1tAL7BZmZQwUrEQs1X3wtrAPuMyz8s-k_7WRg@mail.gmail.com> <43998e7b-9227-7562-b2c6-c08134065e22@ninenines.eu> <CAD3-0rPRPzVvYb6_Z4wDZp73L5Kyb7LmE0P5j4A-2VSRwT7FMw@mail.gmail.com> <CAH9hSJb=mWdHP8xcBis8-jhWgQTfN-cgQXVV3eCyT4U8JYQHZA@mail.gmail.com> <CAP8-FqnLaRvyQgXXkoNQPKcyMhv-O3RN67CMw5L_-1iQ9c6mhw@mail.gmail.com> <CAH9hSJYpsPW4S9n2LaaLTYYKB7wR3Sod2=fny2CZoUR7A0bSJA@mail.gmail.com> <CAP8-FqkOX1Sq6_=Sgb++QRiDWKEiOxAJ13kzMSr9heu-Ek3QmA@mail.gmail.com> <508f7085-b6b9-572e-7b0f-26cafc94dd44@ninenines.eu> <CAH9hSJZcGui08=DivN9vynKejvNFy+RYtRDYDnd6U6gxyX3UgQ@mail.gmail.com> <CAH9hSJZZCVMpQrpEV_JTceEmf2Y2aC_kJNXJmLW=LPebG+JR7g@mail.gmail.com> <CAP8-Fqk9SQJOuKWQmf5cRm9z2ja9wWUeG9xmivhiJf5O57Uryw@mail.gmail.com> <CAH9hSJZTVKx-8vg2xcqr_g4Bg+hc1ufvPZ2hZ+F=dXeVOdSu_Q@mail.gmail.com> <CAP8-Fqm=OVaOJ1imySM41_OuNu0D12Jby59dOpgqz-Bg4M+YOQ@mail.gmail.com> <CAD3-0rM35uXJnwfGay-1s9uw=-P71EubOkxFdKF=gjoXub8YXw@mail.gmail.com> <CAH9hSJZB0SyFiqLqLjd9R-T11yTa12Ekb-H8hYwfc6FeOjD2xQ@mail.gmail.com>
In-Reply-To: <CAH9hSJZB0SyFiqLqLjd9R-T11yTa12Ekb-H8hYwfc6FeOjD2xQ@mail.gmail.com>
From: Costin Manolache <costin@gmail.com>
Date: Wed, 02 Nov 2016 18:20:10 +0000
Message-ID: <CAP8-FqmU+uBas5zH8oQHkt0zh18YrBm-O-umGPGMkLAjShw1Gw@mail.gmail.com>
To: Takeshi Yoshino <tyoshino@google.com>, Wenbo Zhu <wenboz@google.com>
Cc: Loïc Hoguin <essen@ninenines.eu>, "ietf-http-wg@w3.org Group" <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary="94eb2c0b8e6e762d2a0540557ea5"
Received-SPF: pass client-ip=209.85.161.175; envelope-from=costin@gmail.com; helo=mail-yw0-f175.google.com
X-W3C-Hub-Spam-Status: No, score=-2.7
X-W3C-Hub-Spam-Report: AWL=1.343, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: titan.w3.org 1c209X-0001mM-Gh adb1aa01e048d2022e7546124f09de01
X-Original-To: ietf-http-wg@w3.org
Subject: Re: WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP)
Archived-At: <http://www.w3.org/mid/CAP8-FqmU+uBas5zH8oQHkt0zh18YrBm-O-umGPGMkLAjShw1Gw@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/32815
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
Good timing - http://httpwg.org/http-extensions/encryption-preview.html is addressing my concerns for webpush ( and general 'encrypted content' ), we're still discussing some details, but for this use case metadata won't be needed. Costin On Tue, Nov 1, 2016 at 10:34 PM Takeshi Yoshino <tyoshino@google.com> wrote: > On Mon, Oct 31, 2016 at 5:57 AM, Wenbo Zhu <wenboz@google.com> wrote: > > > > On Sun, Oct 30, 2016 at 10:25 AM, Costin Manolache <costin@gmail.com> > wrote: > > Thanks for the answer and pointers. From earlier responses, it seems > possible to use GET > or a non-web-stream request to would avoid the extra cost of the > pre-flight. > > > > Yeah, at least the Content-Type in the HTTP request gets eliminated. > > > One more question/issue: in some cases it would be good to send some > metadata (headers) along with binary frames. For example in webpush the > content is an encrypted > blob, and needs headers for the key/salt. I would assume a lot of other > 'binary' messages would > benefit if simple metadata could be sent along. Would it be possible to > use one of the reserved > bits for 'has metadata' and add some encoded headers ? I know in websocket > they are intended > for 'extensions', but 'headers' seems a very common use case. > > Q about webpush: is the metadata different for each binary message? > > We discussed about metadata and how to use one of RSV bits etc. For the > current version, let's make sure the WS compatibility is fully addressed > (with minimum wire encoding like WiSH) > > > Agreed. Let's discuss the metadata needs separately. I agree it's > important. > > > > > > Having the binary frame use some MIME encoding to pass both text headers > and the binary blob > is possible - but has complexity and overhead. > > OTOH, if the binary blob relies on text headers (metata) to be useful, > then you probably need define a dedicated MIME encoding. > > > > > Costin > > On Sun, Oct 30, 2016 at 5:27 AM Takeshi Yoshino <tyoshino@google.com> > wrote: > > Thanks, Van, Costin. > > On Sun, Oct 30, 2016 at 2:43 AM, Costin Manolache <costin@gmail.com> > wrote: > > Good point - websocket is widely deployed, including IoT - and the header > is pretty easy to handle anyways. > +1. > > One question: is this intended to be handled by browsers, and exposed > using the W3C websocket API ? > Will a regular app be able to make WiSH requests and parse the stream by > itself, without browser > interference ? And if yes, any advice on how it interact with CORS ? > > > The first step would be using Streams based upload/download via the Fetch > API + protocol processing in JS. > > The next step could be either introduction of an optimized native > implementation of WiSH parser/framer in the form of the TransformStream > which can be used as follows: > > const responsePromise = fetch(url, init); > responsePromise.then(response => { > const wishStream = response.body().pipeThrough(wishTransformStream); > function readAndProcessMessage() { > const readPromise = wishStream.read(); > readPromise.then(result => { > if (result.done) { > // End of stream. > return; > } > > const message = result.value; > // Process the message > // E.g. access message.opcode for opcode, message.body for the body > data > readAndProcessMessage(); > }); > } > readAndProcessMessage(); > }); > > and provide a polyfill that presents this as the WebSocket API, and (or > skip the step and) go further i.e. native implementation for everything if > it turns out optimization is critical. > > We need to discuss this also in W3C/WHATWG. > > Regarding CORS, if the request includes non CORS-safelisted headers, > fetch() based JS polyfills will be basically subject to the CORS preflight > requirement. We could try to exempt some of well defined headers if any for > CORS like WebSocket handshake's headers and server-sent event's > Last-Event-Id are exempted. Regarding the proposed subprotocol negotiation > in the form of combination of the Accept header and the Content-Type > header, the Accept header is one of the CORS-safelisted headers, so it's > not a problem. The Content-Type header is considered to be > non-CORS-safelisted if it's value is none of the CORS-safelisted media > types. So, WiSH media type would trigger the preflight unless we exclude it. > > Origin policy https://wicg.github.io/origin-policy/ might also help. > > > > Costin > > On Fri, Oct 28, 2016 at 12:06 PM Takeshi Yoshino <tyoshino@google.com> > wrote: > > Sorry for being ambivalent. > > We can of course revisit each design decision we made for RFC 6455 framing > and search for the optimal again. But as: > - one of the main philosophies behind WiSH is compatibility with WebSocket > in terms of both spec and implementation > - the WebSocket is widely deployed and therefore we have a lot of > implementations in various languages/platform > - most browsers already have logic for the framing > - the framing is not considered to be so big pain > inheriting the WebSocket framing almost as-is is just good enough. > Basically, I'm leaning toward this plan. > > Takeshi > > On Sat, Oct 29, 2016 at 3:12 AM, Takeshi Yoshino <tyoshino@google.com> > wrote: > > On Sat, Oct 29, 2016 at 2:55 AM, Loïc Hoguin <essen@ninenines.eu> wrote: > > On 10/28/2016 08:41 PM, Costin Manolache wrote: > > Current overhead is 2 bytes if frame is up to 125 bytes long - which I > think it's not very common, > 4 bytes for up to 64k, and 10 bytes for anything larger. > IMHO adding one byte - i.e. making it fixed 5-byte, with first as is, > and next 4 fixed length would > be easiest to parse. > > > Is making it easy (or easier) to parse even a concern anymore? > > Considering the number of agents and servers already supporting Websocket, > the numerous libraries for nearly all languages and the great > autobahntestsuite project validating it all, reusing the existing code is a > very sensible solution. > > > Yeah, I've been having similar feeling regarding cost for parser/encoder > implementation though I might be biased. > > > There are obviously too many options to encode and each has benefits - > my only concern was > that the choice of 1, 2, 8 bytes for length may not match common sizes. > > ( in webpush frames will be <4k ). > > > -- > Loïc Hoguin > https://ninenines.eu > > > > >
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- WiSH: A General Purpose Message Framing over Byte… Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Loïc Hoguin
- Re: WiSH: A General Purpose Message Framing over … Van Catha
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Loïc Hoguin
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Loïc Hoguin
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Wenbo Zhu
- Re: WiSH: A General Purpose Message Framing over … Wenbo Zhu
- Re: WiSH: A General Purpose Message Framing over … Loïc Hoguin
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Costin Manolache
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Wenbo Zhu
- Re: WiSH: A General Purpose Message Framing over … Costin Manolache
- Re: WiSH: A General Purpose Message Framing over … Costin Manolache
- Re: WiSH: A General Purpose Message Framing over … Loïc Hoguin
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Wenbo Zhu
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Van Catha
- Re: WiSH: A General Purpose Message Framing over … Costin Manolache
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Costin Manolache
- Re: WiSH: A General Purpose Message Framing over … Wenbo Zhu
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Costin Manolache
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Van Catha
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Van Catha
- Re: WiSH: A General Purpose Message Framing over … Mark Nottingham
- Re: WiSH: A General Purpose Message Framing over … Andy Green
- Re: WiSH: A General Purpose Message Framing over … Mark Nottingham
- Re: WiSH: A General Purpose Message Framing over … Andy Green
- Re: WiSH: A General Purpose Message Framing over … Mark Nottingham
- Re: WiSH: A General Purpose Message Framing over … Willy Tarreau
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino
- Re: WiSH: A General Purpose Message Framing over … Mark Nottingham
- Re: WiSH: A General Purpose Message Framing over … Van Catha
- Re: WiSH: A General Purpose Message Framing over … Poul-Henning Kamp
- RE: WiSH: A General Purpose Message Framing over … Mike Bishop
- Re: WiSH: A General Purpose Message Framing over … Van Catha
- Re: WiSH: A General Purpose Message Framing over … Andy Green
- Re: WiSH: A General Purpose Message Framing over … Van Catha
- Re: WiSH: A General Purpose Message Framing over … Andy Green
- Re: WiSH: A General Purpose Message Framing over … Martin J. Dürst
- Re: WiSH: A General Purpose Message Framing over … Alexey Melnikov
- Re: WiSH: A General Purpose Message Framing over … Takeshi Yoshino