Re: [hybi] method of allocation of reserved bits to extensions

[Jim Gettys <[e-mail]>]

On 05/25/2011 01:08 AM, Greg Wilkins wrote:
>
> I don't want them available for extensions if it is just a
> free-for-all, because then non-technical market forces will come into
> play for their allocation and interoperability will be poor unless you
> only use defacto standard extensions.
>
> IANA registry is also a way to avoid the free for all, but it feels a
> bit heavy bureaucracy for just 3 bits, which will soon be consumed and
> then we are back to having no available bits and extensions have to
> use the payload anyway.
>
> But IANA is better than the status quo of no allocation.
>
>
>>>> It sounds to me like you are saying "if I can't have it my way, nobody
>>>> can
>>>> have them".  If they can't ever be used for extensions, it isn't
>>>> entirely
>>>> clear what they could be used for, so we would be better off just
>>>> allocating
>>>> them to the opcode than leaving them unused.
>>> If they are registered it doesn't mean they can't be used. Just that
>>> the extension that uses them must be blessed with the registration.
>>> All other extensions just add a byte to their payload and use it for
>>> their flags.
>> Again, that isn't what Greg was suggesting.
>> Also, if reserved opcodes can't be used (which Greg was also suggesting),
>> then there is no room for any experimentation, which seems a recipe for poor
>> progress getting useful extensions.
> Extensions can always have their own opcodes by putting them as a byte
> in the payload (which is what you have previously advocated for
> extension opcodes anyway).         Sure we can go to IANA and create a
> registry for the 8 or so available opcodes, but I'm sure they will
> soon be allocated to some worthy extensions and then future
> experimentation will have to use payload opcodes anyway.
>
> I just think that we have too few bits and opcodes available to make a
> fixed allocation work (either by defacto standards or by IANA
> registry), as all will soon be consumed and we then have to think of
> other solutions anyway.      I think a simple dynamic allocation
> mechanism would work for a large pool of possible extensions, but we
> would be limited by how many active extensions could be applied on the
> same connection.
>
> So eitherway, I see that eventually we will have to handle extensions
> with flags and codes in the payload, so why not just go there directly
> and not have to deal with either designing a system now or working
> with IANA to allocated very limited resources.
>
There is a technique we used in X11 design we should have pushed on much 
more broadly: enabling the client to do assignment/allocation of 
protocol elements as much as possible: we did this with resource ID's, 
and should have done this elsewhere in the protocol.  Our really big 
mistake was the Atom system, but I'll leave that for a different 
discussion if appropriate.  But the extension system has a similar 
mistake we made.

In the X protocol, extensions are a two phase process.

You perform a "QueryExtension", in which you provide the name of the 
extension (which is a string).  If it exists, the server tells you what 
opcode it will be available at (and a base number in the error numbering 
field for reporting errors, and a base number in the event fields that 
events may return from the extension).

In the X protocol, opcodes are two fields, a major opcode (1 byte) and a 
minor opcode (1 byte).

So long as there is no collision in the names of extensions, there is no 
reason for a registry.

There is also a protocol request to list the extensions available at the 
server.

By convention extensions have version numbering in them.

The downside of what we did (in the X protocol in general), is that 
while maximally extensible, it's more "chatty" than it might be 
requiring round trips to synchronise client and server state, which 
we've had to kludge around by pipelining requests.  We hadn't yet 
internalised that going over the net was so expensive to always optimise 
for that case.

It would have been better to just list the extension names available up 
front at connection open, and (when an extension is first used), have 
the client have to send a request to assign it a free opcode, avoiding 
the RTT delays.  You could also imagine (given your limited opcode 
space) you might be able to support more extensions simultaneously, 
switching assignments dynamically.  You could even have your servers 
list the most frequently used extensions first, and pre-assign them, so 
that the client doesn't have to send this message in many cases.

We haven't needed to in X, because we have 256 available opcodes, and 
haven't exhausted the op-code space.

Dunno if you can go this route: but it's what I'd do if I could and it 
were my headache.
                 - Jim