Re: [hybi] Extensibility mechanisms?

[Jamie Lokier <[e-mail]>]

John Tamplin wrote:
>    I still don't see the argument for servers written by amateurs. Â I
>    have used far more quick-and-dirty web clients (telnet,
>    socket/connect/write in C, wget/curl, etc) than I have quick-and-dirty
>    servers. Â If some amateur needs to write a server anyway, they aren't
>    going to write it from scratch even if it were possible for them to do
>    so -- they will find some library.
>    Contrast this with embedded clients, which might well have tighter
>    constraints than most clients, and could benefit from having optional
>    features they could leave out that would be useful for more powerful
>    clients. Â Meanwhile, the servers are going to almost certainly be on
>    more powerful machines, though granted they have more connections to
>    support.

But I must say I don't see a problem with keeping it in reach to
amateur programers, at the same time as supporting smarter features.
The two requirements are easily reconciled.

I'm a big fan of the "complex protocols" end of the spectrum - because
I consistently advocate multiplexing, feature negotiation, proxy
opt-ins, message splitting and other "complex" things; all of them are
driven by the desire for good performance (over a variety of
measures), and providing a consistently simple "it just works" API to
the API users.

All that's needed, for amateur programmer compatibility, is for the
"complex" performance-enhancing features to be optional negotiated
features on top of an extremely simple base protocol.  And for the
negotiation to be really simple too.

That's a good idea anyway.

We have a great analogue of this in HTTP's optional persistent
connections.  Simple clients and simple servers don't handle
persistent connections; better ones do, and every combination
works just fine.

Here's how I imagine it working:

The big browsers get decent WS implementations, and are good at
performance-related things that are "hidden under the API", like
automatic sharing of connections, merged keepalives, and maybe even
things like network-signalled-callback on phones.  They request the
most useful enhancers in negotiation, but they work if the answer is
"no".  They can continue to be extended if we think of newer techniques.

Well known servers (Apache etc.) get decent implementations, and can
handle the performance-enhancing features.  They negotiate "yes"
to all the most useful enhancers.

Some proxies like those on a mobile network recognise negotiation and
can add enhancements like network-signalled-callback mentioned above,
and even connection fan-in / fan-out if that proves to be technically
workable.  Other proxies, including HTTP ones, don't participate in
negotiation and are simply transparent.

Joe Random wants to write a quick WebSocket server in half a page of
Python.  That's fine: He implements the minimum base protocol, which
is designed to be very simple (providing the minimum needed for the
WebSocket API to work) and it *just works* with all browsers and other
clients.  Negotiation is basically "no" to all features.

Same when Joe Random wants to write a quick client.

The most important part of this model is that negotiation is sane and
extensible right from the start, and that it has room for all the
interested parties: "below the API" client requested features, proxy
requested/recognised features, server requested features.  The
existing subprotocol field provides the fourth: "above the API" client
requested features.

As setup latency is an issue, it is also important that negotiation
does not incur avoidable round trips.

A second important thing is subtler: It is important that features
negotiated transparently "below the API" are carefully designed to be
*invisible* to WebSocket API users.

Connection sharing is the best example (but it also applies to
keepalive merging): Sending an erroneous message on one channel of a
shared connection must not break all the others; if a receiver event
queue fills, thereby stalling that channel, it must not block messages
from flowing to the others; and sending a very large message on one
channel must not block others for a long time.  Otherwise the
behaviour isn't transparent enough as an *automatic* under-the-hood
replacement for separate connections.

-- Jamie