Re: WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP)

[Andy Green <[e-mail]>]

On Fri, 2016-11-25 at 10:35 +1100, Mark Nottingham wrote:
> To clarify, we said that the future of WebSockets really isn't in
> scope for this WG; the proper venue for discussing that is:
>   https://www.ietf.org/mailman/listinfo/hybi

They kind of shut that down, the people managing it have gone away

https://mailarchive.ietf.org/arch/msg/hybi/YVBJTzJcvzytIY46KfIS8bchf5E

the ML is still running but is very quiet.

WS itself seems to be in rude health out there.

> What *is* in-scope here is how (if at all) that protocol interacts
> with HTTP, including HTTP/2; there are several ways you could
> implement WebSockets over HTTP/2, and a few pitfalls in doing so that
> the people on this list will be able to give you feedback on.

It's unfortunate that wasn't considered part of defining HTTP/2, so it
could be baked in.   The subject was certainly raised.  But I can
understand the desire to get the main business out of the door.

> However, it's hard to do that before there's agreement in the WS
> community about what the requirements are. Ideally, that community
> would bring a single proposal that has broad support here for review.

Any formal mechanism to manage that has gone away for hybi AFAICT.  So
this "let the community do it" feels like a bit of a cop-out / bullet
dodging.

At any rate I think the number of people interested in HTTP/2 WS is
still very low compared to the number of people interested in WS client
and server generally in my experience.  But at some point that will
change, possibly suddenly.

-Andy

> Cheers,
> 
> 
> > On 25 Nov. 2016, at 5:39 am, Van Catha <[e-mail]> wrote:
> > 
> > Thanks for clarification. Unfortunate that so little attention was
> > paid to this.  Looks like some of us will be on HTTP1.1 for a long
> > time.
> > 
> > On Mon, Nov 21, 2016 at 11:14 PM, Takeshi Yoshino <tyoshino@google.
> > com> wrote:
> > Ah, no. Martin just warned us that we might face the same issue
> > that SSE faced.
> > 
> > Mark's suggestion is a separate thing. The co-chairs (Mark and
> > Patrick) said that this (WiSH) doesn't seems to be a topic that
> > should be discussed in the HTTP WG given the charter of the WG, I
> > think.
> > 
> > On Sun, Nov 20, 2016 at 12:26 PM, Van Catha <[e-mail]>
> > wrote:
> > I do not understand what this means.  Is the suggestion to ignore
> > WiSH for now in favor of SSE?
> > 
> > On Fri, Nov 18, 2016 at 1:55 AM, Takeshi Yoshino <tyoshino@google.c
> > om> wrote:
> > I'd like to share the feedback on WiSH from IETF 97.
> > 
> > ----
> > 
> > Due to limited time, I got just one on-site comment from Martin
> > about comparison with Server-sent event (EventSource).
> > 
> > As mentioned in the I-D, yes, this is kinda full-duplex SSE with
> > the WS framing, and it might suffer from unexpected buffering by
> > intermediaries if any as Martin said.
> > 
> > WiSH should work well for deployment with TLS only (possibly with
> > some non-TLS part beyond server side front-end but are under
> > control of the service providers). Given the wide trend of
> > encouraging TLS and browser vendors' implementation status of H2, I
> > think we should prioritize layering simplicity than taking care of
> > gain of WiSH/H2/TCP + transparent proxy (with unexpected buffering)
> > case. For H2-less TLS-less environment, we can just use the
> > WebSocket protocol.
> > 
> > There can still be some risk of MITM (trusted) proxy and unexpected
> > buffering with AntiVirus/Firewall for deployment with TLS, but
> > other WebSocket/H2 mapping proposals also have issues of possible
> > blocking, buffering, etc. WebSocket/TCP's handshake success rate
> > for non-TLS port 80 was also not so good when it started getting
> > deployed, and got improved gradually. I think the problems will get
> > resolved once WiSH is accepted widely, and I believe the total pain
> > and cost would be smaller.
> > 
> > ----
> > 
> > Mark suggested that we should find some other right place than HTTP
> > WG. I'll discuss this with Mark. Maybe we'll consult the DISPATCH
> > WG.
> > 
> > ----
> > 
> > Thanks everyone for the feedback.
> > 
> > Takeshi
> > 
> > On Thu, Nov 3, 2016 at 3:20 AM, Costin Manolache <[e-mail]>
> > wrote:
> > Good timing -  http://httpwg.org/http-extensions/encryption-preview
> > .html is addressing my concerns for
> > webpush ( and general 'encrypted content' ), we're still discussing
> > some details, but for this use
> > case metadata won't be needed.
> > 
> > Costin
> > 
> > 
> > On Tue, Nov 1, 2016 at 10:34 PM Takeshi Yoshino <[e-mail]
> > m> wrote:
> > On Mon, Oct 31, 2016 at 5:57 AM, Wenbo Zhu <[e-mail]>
> > wrote:
> > 
> > 
> > On Sun, Oct 30, 2016 at 10:25 AM, Costin Manolache <[e-mail]
> > m> wrote:
> > Thanks for the answer and pointers. From earlier responses, it
> > seems possible to use GET
> > or a non-web-stream request to would avoid the extra cost of the
> > pre-flight.
> > 
> > 
> > Yeah, at least the Content-Type in the HTTP request gets
> > eliminated.
> >  
> > One more question/issue: in some cases it would be good to send
> > some
> > metadata (headers) along with binary frames. For example in webpush
> > the content is an encrypted
> > blob, and needs headers for the key/salt. I would assume a lot of
> > other 'binary' messages would
> > benefit if simple metadata could be sent along. Would it be
> > possible to use one of the reserved
> > bits for 'has metadata' and add some encoded headers ? I know in
> > websocket they are intended 
> > for 'extensions', but 'headers' seems a very common use case.
> > Q about webpush: is the metadata different for each binary
> > message? 
> > 
> > We discussed about metadata and how to use one of RSV bits etc. For
> > the current version, let's make sure the WS compatibility is fully
> > addressed (with minimum wire encoding like WiSH)
> > 
> > Agreed. Let's discuss the metadata needs separately. I agree it's
> > important.
> >  
> >  
> > 
> > Having the binary frame use some MIME encoding to pass both text
> > headers and the binary blob
> > is possible - but has complexity and overhead.
> > OTOH, if the binary blob relies on text headers (metata) to be
> > useful, then you probably need define a dedicated MIME encoding.
> > 
> >   
> > 
> > Costin
> > 
> > On Sun, Oct 30, 2016 at 5:27 AM Takeshi Yoshino <[e-mail]
> > m> wrote:
> > Thanks, Van, Costin.
> > 
> > On Sun, Oct 30, 2016 at 2:43 AM, Costin Manolache <[e-mail]
> > > wrote:
> > Good point - websocket is widely deployed, including IoT - and the
> > header is pretty easy to handle anyways.
> > +1.
> > 
> > One question: is this intended to be handled by browsers, and
> > exposed using the W3C websocket API ?
> > Will a regular app be able to make WiSH requests and parse the
> > stream by itself, without browser
> > interference ? And if yes, any advice on how it interact with CORS
> > ? 
> > 
> > The first step would be using Streams based upload/download via the
> > Fetch API + protocol processing in JS.
> > 
> > The next step could be either introduction of an optimized native
> > implementation of WiSH parser/framer in the form of the
> > TransformStream which can be used as follows:
> > 
> > const responsePromise = fetch(url, init);
> > responsePromise.then(response => {
> >   const wishStream =
> > response.body().pipeThrough(wishTransformStream);
> >   function readAndProcessMessage() {
> >     const readPromise = wishStream.read();
> >     readPromise.then(result => {
> >       if (result.done) {
> >         // End of stream.
> >         return;
> >       }
> > 
> >       const message = result.value;
> >       // Process the message
> >       // E.g. access message.opcode for opcode, message.body for
> > the body data
> >       readAndProcessMessage();
> >     });
> >   }
> >   readAndProcessMessage();
> > });
> > 
> > and provide a polyfill that presents this as the WebSocket API, and
> > (or skip the step and) go further i.e. native implementation for
> > everything if it turns out optimization is critical.
> > 
> > We need to discuss this also in W3C/WHATWG.
> > 
> > Regarding CORS, if the request includes non CORS-safelisted
> > headers, fetch() based JS polyfills will be basically subject to
> > the CORS preflight requirement. We could try to exempt some of well
> > defined headers if any for CORS like WebSocket handshake's headers
> > and server-sent event's Last-Event-Id are exempted. Regarding the
> > proposed subprotocol negotiation in the form of combination of the
> > Accept header and the Content-Type header, the Accept header is one
> > of the CORS-safelisted headers, so it's not a problem. The Content-
> > Type header is considered to be non-CORS-safelisted if it's value
> > is none of the CORS-safelisted media types. So, WiSH media type
> > would trigger the preflight unless we exclude it.
> > 
> > Origin policy https://wicg.github.io/origin-policy/ might also
> > help.
> >  
> > 
> > Costin
> > 
> > On Fri, Oct 28, 2016 at 12:06 PM Takeshi Yoshino <tyoshino@google.c
> > om> wrote:
> > Sorry for being ambivalent.
> > 
> > We can of course revisit each design decision we made for RFC 6455
> > framing and search for the optimal again. But as:
> > - one of the main philosophies behind WiSH is compatibility with
> > WebSocket in terms of both spec and implementation
> > - the WebSocket is widely deployed and therefore we have a lot of
> > implementations in various languages/platform
> > - most browsers already have logic for the framing
> > - the framing is not considered to be so big pain
> > inheriting the WebSocket framing almost as-is is just good enough.
> > Basically, I'm leaning toward this plan.
> > 
> > Takeshi
> > 
> > On Sat, Oct 29, 2016 at 3:12 AM, Takeshi Yoshino <tyoshino@google.c
> > om> wrote:
> > On Sat, Oct 29, 2016 at 2:55 AM, Loïc Hoguin <[e-mail]>
> > wrote:
> > On 10/28/2016 08:41 PM, Costin Manolache wrote:
> > Current overhead is 2 bytes if frame is up to 125 bytes long -
> > which I
> > think it's not very common,
> > 4 bytes for up to 64k, and 10 bytes for anything larger.
> > IMHO adding one byte - i.e. making it fixed 5-byte, with first as
> > is,
> > and next 4 fixed length would
> > be easiest to parse.
> > 
> > Is making it easy (or easier) to parse even a concern anymore?
> > 
> > Considering the number of agents and servers already supporting
> > Websocket, the numerous libraries for nearly all languages and the
> > great autobahntestsuite project validating it all, reusing the
> > existing code is a very sensible solution.
> > 
> > 
> > Yeah, I've been having similar feeling regarding cost for
> > parser/encoder implementation though I might be biased.
> >  
> > There are obviously too many options to encode and each has
> > benefits -
> > my only concern was
> > that the choice of 1, 2, 8 bytes for length may not match common
> > sizes.
> > 
> > ( in webpush frames will be <4k ).
> > 
> > -- 
> > Loïc Hoguin
> > https://ninenines.eu
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> 
> --
> Mark Nottingham   https://www.mnot.net/
> 
> 
> 
> 
>