Re: [hybi] Framing take IV

[Jamie Lokier <[e-mail]>]

Greg Wilkins wrote:
> 
>    Jamie,
>    so to make your proposal a little more concrete, I think that you are
>    advocating something like:

Sorry, I'm not able to decode the diagrams you are sending.  They come
out as mangled text in my editor.

But yes, the basic idea: length, opcode, data.

The length must include the opcodes as well as the data, although the
first opcode byte doesn't have to be included if we don't want to.

The order should probably be length, opcode, data, so a "wrapper"
style opcode would end up looking like these made up examples.
First the simple one:

    length, any-opcode, data  (basic simple frames, including text, binary etc.)

Extensions, to be defined by whomever wants to, no standard necessary:

    length, multiplex, flow-id, final-bit, forward-bit, any-opcode, data
    length, crc32-opcode, any-opcode, data, crc32
    length, md5-opcode, any-opcode, data, md5

Putting length after the first opcode suggests length is part of the
opcode's parameters, which it isn't, as you can see in the last three
examples, where any-opcode isn't associated with a length of it's own,
but it's the same set of opcodes.

>           frame-opcode              = %x00 ; text data frame
>                                     | %x01 ; final frame
>                                     | %x02-ff; extension

I'd rename "final frame" as "basic data frame", but it's a nit.

>           frame-length               = %x0000-FFFF

I'm not sure if this limit penalises implementations shifting large
amounts of data over jumbo frames, which are about 64kB per packet, or
over TCP segmentation accelerators, which take large packets and split
them in hardware on the way out.  I.e. a larger limit such as 24 bits
might be better, for implementations which can use "sendfile" or
"splice" style transmission methods to share memory among sockets and
avoid copying the data.  I haven't investigated or tested in
sufficient detail.

However I'm leaning towards the BER-style that Ian Hixie suggests,
with an explicit note in the specification saying what the maximum
permitted BER value is (or equivalently what the maximum number of
bytes encoding the value are), with the implication that all
implementations should count properly up to that (although they may
still reject a message on the grounds of being too large for local
policy of course).

The first byte in BER does not have to use all the bits, so we could
reserve a few bits in the first byte for universal flags _if_ we think
there are any, and the remainder of bits would be sufficient for small
lengths.  I'm not sure if this is needed - do you think there are any
universal bits needed?

>    A client would indicate the extensions that it wants to support with a
>    handshake header like:
>         Websocket-Ext: fragment;mandatory=true, ping;mandatory=true, multiplex, compression

>    The server could then respond with the extensions that it will be
>    used, plus allocations of bits/codes:
>         Wesocket-Ext: fragment;bit=7, ping;code=02-03, multiplex;code=10-1F

>    Is that about right?

Not quite.

The *client* decides opcode values for features it proposes.  The
server must ignore frames with opcodes for features it doesn't
support; this is why the length is universal (and also for
intermediaries).

If we ever use any mechanism to bypass the negotation round-trip (such
as TLS extensions, encoding initial messages in the handshake, etc.),
it is only effective if the client sets the opcodes.

Although we might not use that mechanism with the HTTP handshake, it
is a valuable and simple trap door in case we decided to avoid startup
latency in future, or with a different handshake, and it is also a
good pattern for latency avoidance in general.

-- Jamie