Re: [hybi] method of allocation of reserved bits to extensions

[Brodie Thiesfield <[e-mail]>]

On Fri, Apr 29, 2011 at 1:01 PM, Alan Coopersmith
<[e-mail]> wrote:
> On 04/28/11 07:16 PM, Brodie Thiesfield wrote:
>> Hi Timothy,
>>
>> On Fri, Apr 29, 2011 at 4:31 AM, Timothy Meade <[e-mail]> wrote:
>>> On Apr 27, 2011 11:12 PM, "Greg Wilkins" <[e-mail]> wrote:
>>>>
>>>> this is good. But I think we also need to say the same for op-codes.
>>>>
>>>> I think each extension needs to say how many bits, data op-codes and
>>>> control op-codes it needs and then these should all be allocated in
>>>> order.
>>>>
>>>
>>> It might be instructive to look at X Protocol opcodes for core and
>>> extensions and how that played out over time.
>>
>> I found the following description of the opcodes:
>> http://www.x.org/wiki/Development/Documentation/Protocol/OpCodes
>>
>> From the summary of the scheme above:
>> * each extension is assigned a single major identifying opcode
>> * the extension then defines sub-codes for itself if necessary
>>
>> So this is pretty much the same as we are looking at in websockets but
>> we have a lot less opcode space to allocate.
>>
>> Can you supply a pointer to a summary of the problems that occurred
>> with the X scheme? I'm assuming that it didn't turn out all rosy.
>
> I don't know why you'd assume that, since the X extension mechanism is
> the key that's kept the X protocol alive for 25 years as graphics
> technology changed rapidly.

Just the phrasing that Timothy used sounded to me like there had been
problems that would be relevant to websockets.

The problems that X sees with debugging dynamic opcode space is going
to also be a problem that websockets faces, but again that will need
to be solved via debug information dumped by each end.

Referring to Timothy's own reply though, he was pointing out that the
amount of extension opcode space in websockets is probably
insufficient. Given that we only have 4 bits of opcode space to start
with, and only 10 opcodes left available (%3-7 and %xB-F), it is quite
possible that websockets could run into extension space problems.

Unfortunately, the framing argument was a long and arduous journey, so
attempting to change part of the framing (i.e. to introduce more
extension opcode/reserved bits) is a hard sell. Given also that the
protocol is now officially into last call, it would seem impossible.

Unfortunately, I think that the current framing does need some changes
because of the partitioning of the opcode space.

1) opcode partitioning

Partitioning the opcode space is I believe a good decision, however it
has left us with the equivalent of a reserved bit in the middle of
what was supposed to be the opcode growth space. The frame is now
essentially:

Byte 1:
  FIN
  RSV1
  RSV2
  RSV3
  opcode (4) = CTRL BIT (is this a control frame or not) + opcode(3)

One idea for the RSV3 bit was to allow growth of the opcode space, but
that cannot happen sanely with the equivalent of the CTRL bit in the
middle of it. IF we want to keep that ability, then it would make more
sense to move CTRL next to FIN and say that opcodes are unique only
within the namespace created by the CTRL bit. e.g.

Byte 1:
  FIN
  CTRL (is this a control frame or not)
  RSV1
  RSV2
  RSV3
  opcode (3)

Fitting that into the current spec, it would be:
control frames: FIN(1), CTRL(1), opcodes: 0 = reserved, 1 = close, 2 =
ping, 3 = pong
data frames: FIN(0 or 1), CTRL(0), opcodes: 0 = continuation, 1 =
text, 2 = binary

This sort of rearrangement is only required if we want to keep the
ability to expand to RSV3. If we do the following, it isn't
necessary...

2) extra opcode space

Whether we will use more opcode or not, or even require them is still
debatable, but it is better to design for the future at least. Let's
reserve opcode 7 (or opcodes 7 and F in the current opcode method). If
they are reserved, then in a future websockets protocol version, they
can be defined to mean that the frame has an extended opcode space. If
both opcodes 7 and F are reserved, we maintain the semantics of data
versus control being visible via the opcode top bit, and therefore
backward compatibility with intermediaries using this version of the
spec.

3) extensions

One possible problem with Greg's idea of having both server and client
assuming which bit is in use, is that it breaks when a new version of
the protocol actually uses one of the reserved bits. I assume though
that if that happens (i.e. a new version of the protocol which
requires the use of reserved bits), that somehow the client and server
will agree on the protocol version in the handshake.

Regards,
Brodie

> The primary problem I'm aware of is that while each extension got one of
> the 128 available requests, and bundled all it's actions into subcodes of
> that request, they didn't do similar "namespacing" for event & error codes.
>
> We've not yet had a situation where we needed more than 128 extensions
> simulataneously active, but have hit the limits of 64 extension events
> and errors, as some extensions used quite a few.
>
> It does occasionally make user bug reports challenging as the dynamic
> assignment of extension codes based on which ones are enabled during
> the build and runtime of the X server means that getting an error report
> of a failed request 153 doesn't let you know which extension it was.
> (This is why the standard libX11 error handler always prints the extension
> names, but some applications and toolkits use custom error handlers that
> do not.)
>
> --
>        -Alan Coopersmith-        [e-mail]
>         Oracle Solaris Platform Engineering: X Window System
>
>