Re: WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP)

Takeshi Yoshino <> Tue, 25 October 2016 09:00 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 0F2D2129415 for <>; Tue, 25 Oct 2016 02:00:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -6.951
X-Spam-Status: No, score=-6.951 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, RP_MATCHES_RCVD=-0.431, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id Jt3qNbI8FHj6 for <>; Tue, 25 Oct 2016 02:00:27 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 7A3721293DB for <>; Tue, 25 Oct 2016 02:00:27 -0700 (PDT)
Received: from lists by with local (Exim 4.80) (envelope-from <>) id 1byxWb-0001q8-P1 for; Tue, 25 Oct 2016 08:56:01 +0000
Resent-Date: Tue, 25 Oct 2016 08:56:01 +0000
Resent-Message-Id: <>
Received: from ([]) by with esmtps (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <>) id 1byxWU-0001oe-Bo for; Tue, 25 Oct 2016 08:55:54 +0000
Received: from ([]) by with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <>) id 1byxWF-0003fX-Sz for; Tue, 25 Oct 2016 08:55:49 +0000
Received: by with SMTP id h14so8068373ywa.2 for <>; Tue, 25 Oct 2016 01:55:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=GQ2DziZTRqok/mcbD/Qi1yM26PdJiJxVjspOTmNPOsg=; b=KN6ZB+fSr3vY0d6H2lfe9CQjz5c1oWxItOKH9oTwHsGv/DdxC6603h2pdIfgkbvcR3 98z8oSmHXuDwRJwoq4J99nUSJ9/0Nid2nIdvV2uxSRih4nXdZytEFaapu4od1Arj3VfO Jl9ihZ5c5C3rOa+yU7EAt/qH0dJs4sWHNpZ12BN9O7xNUxXQ0mnfFHUK2qYCbsObn7Ev Y2S0AjKlzQpO9n3uU0E4Tb4H3JXaYw5EB5uav2V8VAt0ezob/zF7iOxv8jnjT/ZwxpsV 3OI5eRejSeOacA25p5yQBKAD8RpmNcVXO3k60tI4pNoST6QiOoMA0FZhGnC7jI7/ufog 3NwA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=GQ2DziZTRqok/mcbD/Qi1yM26PdJiJxVjspOTmNPOsg=; b=liiR6O4zUDMPs9Z2BM/CUjsPQc4M5uF5M/p5njpC686IQ0ywfkWpTRFQr5/kUqozDy PUF8HmVmgqSkxU7qk6Rf49B0J7IM0St6y8jym+fSq7nkudA9wy5TYYa1Csu6rDM3S1WT zY7o2i/UFf3SPJEpjBEqOs3oO9U5K58p9zirxuOJzAupu3ifhx6JTWcnVJIMCs5xTOrk bfBHgFvplLZFMLJhz1ki0NItDK/r5MMgwQUagu4lJtlafm+0FKYA7QeKCFbTAQPkhKxL 74q+0oTOPYWzFGYkgaXINnP5+FdshT2uZFxXup1sNa3WBYtYw1BacDvYSNOgs2CIADYa NEoQ==
X-Gm-Message-State: ABUngveOmpHhpcHuAAUkmSBB2vAal4a0smxsFk+c1db/2gLGQILJYpN/QLx4DUd1M2x+EvYKi0mn5LZrkBn53n6Z
X-Received: by with SMTP id j36mr15164888iod.143.1477385713728; Tue, 25 Oct 2016 01:55:13 -0700 (PDT)
MIME-Version: 1.0
Received: by with HTTP; Tue, 25 Oct 2016 01:54:52 -0700 (PDT)
In-Reply-To: <>
References: <> <> <> <>
From: Takeshi Yoshino <>
Date: Tue, 25 Oct 2016 17:54:52 +0900
Message-ID: <>
To: Loïc Hoguin <>
Cc: " Group" <>, Wenbo Zhu <>
Content-Type: multipart/alternative; boundary="001a113eacc6aa1636053facaa2c"
Received-SPF: pass client-ip=;;
X-W3C-Hub-Spam-Status: No, score=-6.4
X-W3C-Hub-Spam-Report: AWL=0.444, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_SORBS_SPAM=0.5, RP_MATCHES_RCVD=-1.339, SPF_PASS=-0.001, W3C_AA=-1, W3C_DB=-1, W3C_IRA=-1, W3C_WL=-1
X-W3C-Scan-Sig: 1byxWF-0003fX-Sz 6921b45869159ed41acff0b5aacbd202
Subject: Re: WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP)
Archived-At: <>
X-Mailing-List: <> archive/latest/32670
Precedence: list
List-Id: <>
List-Help: <>
List-Post: <>
List-Unsubscribe: <>

On Sat, Oct 22, 2016 at 1:34 AM, Loïc Hoguin <> wrote:

> On 10/21/2016 03:04 PM, Takeshi Yoshino wrote:
>> <snip>
>> Good point. I was thinking about using the media type suffix convention
>> like +json, +xml, etc. (RFC 6839, RFC 7303) when we need to represent
>> the type of the messages. As the WebSocket subprotocol is required to
>> conform to the token ABNF, they can be embedded into the subtype part.
> The problem with "+json" and "+xml" is that they do not represent the type
> of the messages, just their encoding.
> This is why we have media types like "application/problem+json" instead of
> using "application/json" for everything, where "application/problem"
> indicates what the representation contains, and "+json" indicates its
> encoding.
> Going back to WiSH, having application/webstream+json would only give
> information about the encoding of the response and frames bodies, not what
> they actually contain or how an endpoint should process the frames. We
> would still need more info to process it.

Oh, sorry for being unclear. I meant that we'll use
"application/subprotocol+webstream". I.e. introducing +webstream as a new
media type suffix.

> Regarding negotiation, my impression to the subprotocol mechanism of RFC
>> 6455 has been that it's not really necessary thing. The server and JS
>> (or native application) may negotiate application level sub-protocol
>> using e.g. URL (some parameters in the query part or even the path part
>> can encode such info) and the initial message sent following the
>> handshake response immediately without any latency loss (this topic was
>> discussed at the HyBi WG sometimes e.g.
>> <>).
> The good thing is that it's standard. There's only one way to select a
> sub-protocol, and you are guaranteed failure (or a normal HTTP response) if
> the endpoint doesn't speak the sub-protocol you want.
> The use of a media type is precious to me as it makes it possible to write
> an autonomous client. Making the sub-protocol as part of the media type
> simplifies things greatly.


> There are potential issues though and a number of things need to be
> defined:
> * The method used to perform the request is important. A client will not
> be able to speak to the server if the method is GET. Similarly, no
> communication can occur if the method is HEAD. I would advise defining
> behavior for both GET and POST methods.

Ah, yeah. It's missing. Thanks for pointing it out.

> * The GET method would be used to enable a server->client stream, similar
> to text/event-stream, except with binary frames supported. The difference
> being the lack of event IDs and Last-Event-ID header. In some cases it
> doesn't matter.

> * The POST method would be used to enable a bidirectional stream. But this
> implies that the client uses Content-Type: application/webstream in the
> request, along with the Accept header. Otherwise the server has no way to
> know what the request body will contain. Let content negotiation deal with
> both headers, it's already well defined.
Yeah. I'll add some text for discussing need for communicating subprotocols
and remark the need for both request and response.

> * Technically this would allow client->server and server->client streams
> to select and use a different sub-protocol. I'm not sure it's worth
> preventing this in the spec; instead let the servers decide if they want to
> allow this. But we probably need to mention it.

Allowing more options (combination of asymmetric subprotocols) wouldn't
block existing WebSocket users from migrating to it. If we want to enable
it for the Web, the WebSocket API needs to be updated to take the
client->server direction subprotocol in addition to the offered subprotocol
(converted into the Accept header and selected by the server using the
Content-Type header in the response to specify the server->client direction

Anyway, let's mention that.

> * By the way, don't know if consistency is desirable, by maybe calling it
> application/web-stream is better. Maybe not.

Could you please elaborate the proposal?

> * The HEAD method behaves as usual. The PUT method is probably not
> compatible with doing this. PATCH and DELETE are not compatible AFAIK.

I'm feeling that we should just limit the scope of the proposal to GET and

> <snip>
>> Oh, interesting. Does this mean that on the browser a JSON codec that
>> takes / produces ArrayBuffers is used?
> I have no idea about browsers, sorry. :-) Not dealing with them often. But
> as far as I understand it involves using a different type to get binaries,
> yes. I have no idea how this is done.

OK. Thanks!

>     Which brings me to my question: do you think it could be worth
>>     adding a note to implementers that perhaps they should consider
>>     optionally disabling the UTF-8 check when JSON is expected for text
>>     frames?
>> We could have removed the valid UTF-8 requirement of RFC 6455. This is
>> something need to be enforced at the binding between the Web API and a
>> WebSocket protocol stack, but not at the protocol level.
>> Until
>> <>,
>> text frames had to be a valid UTF-8 (strictly speaking, data had to be
>> 0xFF-free), but not from the version 01.
>> We can choose to omit the valid UTF-8 requirement from the WiSH spec and
>> instead have it in the spec for gluing WiSH with the WebSocket API in
>> the future. Then, server implementors won't be explicitly encouraged to
>> check validness when implementing WiSH.
> That sounds like a good idea. I think it should still be mentioned in the
> protocol spec (basically explaining what you said and referring to another
> document) so that implementors are aware of the UTF-8 check and decide
> whether to implement it. If it was only written in a browser RFC I would
> most likely never have seen it, personally.

Yeah. I'll add some text for discussing this.