[vwrap] We need protocol negotiation to be builtin. (was: authentication : remove reference to MD5)

[Carlo Wood <[e-mail]>]

On Tue, Apr 06, 2010 at 11:22:58AM -0700, Meadhbh Hamrick wrote:
> so the question is, to which degree do we add an engineering burden to
> existing implementations that would like to adopt this group's output?
> the question of the two string identifier is a good one. linden could
> probably make systems that adhere to all sorts of different changes.
> but to what degree to we make it easy for existing implementers vs.
> the desires of people who have yet to build and implementation?

I think that the worst we can do is to make a standard that isn't
as good as it could have been.

In other words, we don't want a "legacy" bug in the protocol.

However,
nothing, absolutely nothing, cannot be solved by adding protocol
negotiation to VWRAP.

For example, we define the standard to use SHA256. But Linden Lab
adds an extension, negotiated with the client at connection, to
use an MD5 hash. Surely you're not afraid that viewer developers
will refuse to support that ;).

Lets have a look at the effects of my protocol negotation proposal again:

1) It will allow us to make mistakes now, and fix them entirely in the future.
2) It will allow us to make a start with the protocol, and complete it in the future.
3) It will allow easy transitions of legacy protocols to the standard over an
   arbitrary period of backwards compatibility.
4) It might, hopefully, lead to protocol evolution, where good changes of the protocol
   are copied by everyone; causing the Virtual World protocol to evolve into
   a better and better protocol without limits, without becoming a legacy burden.

Think about it: ALREADY Linden Lab is complaining that VWRAP can't do
this and should do that because it's hard for them to implement it otherwise.
THAT WILL ONLY GET *MUCH* WORSE! Once we have hunderds of grids, thousands
of viewers and ten millions of users, it will be IMPOSSIBLE to change
the protocol. It will be become one big legacy nightmare with no possibility
to fix things.

Don't think for a second that we are capable of designing the perfect
protocol right now, before anything like all this exists (and don't
tell me that Linden Labs current protocol is the perfect protocol
already).

---

Recap of the TECHNICAL DETAILS:

The protocol negotiation system exists of two equally important parts:

1) Negotiation of "Features".
2) Documentation of "Features".

Lets forget the documentation part of a moment, just assume that
it is possible to easily find how to implement any feature.
Then the protocol negotiation boils down to negotiations of booleans:
Yes or No this feature.

There are two types of features: mandatory (M) features and optional (O) features.
For each connection between two hosts, there is a protocol negotiation
and thus there are two sides (A and B) that both can have mandatory or optional
features. The total number of possibly combinations for any given feature
are therefore:
X: unknown or not implemented (non-existant)
O: Optional
M: Mandatory

A B	result

X X	not used
X O	not used
X M	incompatible
O X	not used
O O	used (or not?)
O M	used
M X	incompatible
M O	used
M M	used

There is another level of compatibility however, besides whether a mandatory
feature is available on both sides: compatiblity of the negotiation itself.

Namely, in order to be able to "shift the origin" of the protocol, it must
be possible to make a mandatory option part of the "base" of the protocol
(for which no negotiation is needed).

In otherwords, a feature has four states:

				Protocol version p
X: non-eXistent			     p < V
N: Not implemented		V <= p < W
O: Optional			V <= p < W
M: Mandatory (negotiated)	V <= p < W
B: Base protocol		W < p

Negotiation of such a feature will fail if one side assumes the feature
is in the Base (assumed mandatory), while the other side doesn't even
know of it's existance (X). This is where the (negotiation) compatibility
version comes in: if a new feature is introduced in an implementation
Foo at version V, then implementations made for Foo of an earlier version
does not know about that feature. That is not a problem however, until
that feature is made a part of the Base protocol in version W > V.

Thus, each side tells eachother their "protocol version", and the side
with the largest version decides if they are compatible by comparing
the smaller version with the "compatibility version" V (in the case
above the "compatibility version" of W is V).

How does one determine the "compatibility version"? This is simple:
it is the smallest version where every feature that is part of the base
protocol in the current version still existed (and did exist in all
versions in between).

This simple compare then takes care of (B,X) and (X,B) pairs: they
won't exist if the smaller version is larger or equal than then
the compatiblity version of the larger version.

Each implementation contains two version numbers (only) thus.
After comparing versions, X and N collapse into X as used in the first table.

EXAMPLE

Implementation of Linden Lab server protocol 100, with compatiblity
version 40. Viewer connects (supporting the "Linden Lab" protocol)
with protocol version 50. Negotiation is allowed since 50 is not
less than 40.

The server tells the viewer that feature MD5HASH is mandatory.
The viewer knows about this, because this feature was already
part of the protocol since version 1. It has this feature as
optional, and the negotiation results in it being used.

The factual messages needed upon connect are:

client connects
client --> server: PROT		(this requests protocol negotiation, without it, the legacy protocol is used)
                   (possible other base protocol start up messages)
server --> client: GRID LINDENLAB 100 40		(possible other LABEL CURVERSION COMPATVERSION triplets can be appended)
client --> server: PROT REQ LINDENLAB 50 ENCRYPTION	(ENCRYPTION is known *optional* in version 50)
server --> client: PROT SET LINDENLAB 50 MD5HASH	(server rejects ENCRYPTION because it is now version 100 and demands MD5HASH)
  (server switches to feature set for outgoing messages here)
client --> server: PROT ACK LINDENLAB 50 MD5HASH	(client agrees, and signals it switches protocol here)
  (client switches to feature set for outgoing messages here)

Now suppose that somewhere in the (far) future Linden Lab manages to 
switch to SHA256 passwords (ie, by having everyone register again),
and they add the optional feature SHA256LOGIN in version 1000.
Every client can still connect, because it is only optional.
However, again a few years later when all viewers caught up with
the new feature, it is made mandatory in version 1100 (say).
Then the protocol negotiation could look as follows:

client --> server: PROT		(without this, viewer is rejected as being incompatible)
server --> client: GRID LINDENLAB 1100 1000 OPENSIM 878 512 CABLEBEACH 1250 733
client --> server: PROT REQ LINDENLAB 1050 OGRE LIPSYNC
server --> client: PROT SET LINDENLAB 1040 OGRE
client --> server: PROT ACK LINDENLAB 1040 OGRE

and the MD5HASH legacy feature has completely vanished
from the negotiation: the protocol has now shifted truely
to the standard (or rather, to LINDENLAB version 1000).

-- 
Carlo Wood <[e-mail]>