Re: [mmox] Compact Binary Serialization
Dan Olivares <dcolivares@gmail.com> Wed, 25 February 2009 06:27 UTC
Return-Path: <dcolivares@gmail.com>
X-Original-To: mmox@core3.amsl.com
Delivered-To: mmox@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B9A0C3A6919 for <mmox@core3.amsl.com>; Tue, 24 Feb 2009 22:27:04 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.14
X-Spam-Level:
X-Spam-Status: No, score=-1.14 tagged_above=-999 required=5 tests=[AWL=-1.307, BAYES_00=-2.599, FF_IHOPE_YOU_SINK=2.166, J_CHICKENPOX_61=0.6]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iczaw7M5vjEO for <mmox@core3.amsl.com>; Tue, 24 Feb 2009 22:27:03 -0800 (PST)
Received: from mail-fx0-f167.google.com (mail-fx0-f167.google.com [209.85.220.167]) by core3.amsl.com (Postfix) with ESMTP id A950B3A68A5 for <mmox@ietf.org>; Tue, 24 Feb 2009 22:27:02 -0800 (PST)
Received: by fxm11 with SMTP id 11so3461481fxm.13 for <mmox@ietf.org>; Tue, 24 Feb 2009 22:27:21 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=H9RrZJT/J2qRNbttyx7/sEkAqoLU1dMOEcF1K+verLo=; b=ni4ZxpQYguJXw3KKB9umE53gmAxt2doeXthRyFPz/JVKehWcKhpUBbMZWAkhp8S/DU QfZcUTdTEq1LtKTtjcdaCMGIsR2TMMo94sPLfnKKAGyZl9oq5SlmsIJXwqJQ2LLJZ2jL 5P3dTb56BdCdK/JLRiks+TSVY7DMVmGdMsZVY=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=cOujEUZS6kYRiZ094mPz4AsLU0sANZjkU81T1XIsjKnDkQqi72JNFObWtD1ILsmVoZ U5TY4jPWbKkijCQJ9xp6KL+n/xGwlThSV1VPyumQXnCKGx5YWeMQ3tGXznDGBEDyi4NM aqnaQlQS6OanJbWdSUBkUARKfyrCKsgRfBJ6U=
MIME-Version: 1.0
Received: by 10.103.244.4 with SMTP id w4mr160983mur.90.1235543237965; Tue, 24 Feb 2009 22:27:17 -0800 (PST)
In-Reply-To: <49A4563A.5060008@gmail.com>
References: <ebe4d1860902240901oaff2260o65c4337598a14fc5@mail.gmail.com> <49A4563A.5060008@gmail.com>
Date: Wed, 25 Feb 2009 01:27:17 -0500
Message-ID: <a768bcd90902242227i8edbfe4x26442349c99988e5@mail.gmail.com>
From: Dan Olivares <dcolivares@gmail.com>
To: Jon Watte <jwatte@gmail.com>
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: quoted-printable
Cc: mmox@ietf.org
Subject: Re: [mmox] Compact Binary Serialization
X-BeenThere: mmox@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Massively Multi-participant Online Games and Applications <mmox.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/mmox>, <mailto:mmox-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/mmox>
List-Post: <mailto:mmox@ietf.org>
List-Help: <mailto:mmox-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/mmox>, <mailto:mmox-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 25 Feb 2009 06:27:04 -0000
Hmm, Just one thing to point out here; <colour>127,255,127</colour> <vector>100.0,75.3,28.6</vector> <quat>0.5,-0.5,0.5,0.5</quat> might seem ok, however, consider what happens when someone in a locale with a different way of displaying the number tries to figure out what this means? In the United States, the following number format applies: (-)123,456,789.00 In Germany, the following number format applies: 123.456.789,00 In Russia and Finland, the following number format applies: 123 456 789,00 This, in my opinion, is the case for separating them with explicit separations within the syntax using non-ambiguious terms Is <colour>127,255,127</colour>, one hundred twenty seven million two hundred fifty five thousand one hundred twenty seven? Is <vector>100.0,75.3,28.6</vector> one hundred thousand and seventy five tenths? Best Regards Daniel Olivares On Tue, Feb 24, 2009 at 3:19 PM, Jon Watte <jwatte@gmail.com> wrote: > Catherine Pfeffer wrote: >> >> (in the order x, y, z, s and not math's order s, x, y, z, that's one of >> Second Life's strangenesses) >> > > They are in good hands. Direct3D, OLIVE, and a lot of other 3D systems use > XYZW. It's a bit like the question of row vectors on the left (D3D) or > column vectors on the right (OpenGL). There are two ways to do it, and the > world is split about equally. > > >> LLSD (simplifying): >> LLIDL says: nose orientation is an array of four fp32s >> LLSD data over the wire: <floats>0.5,-0.5,0.5,0.5</floats> >> > > Why do you need to send "floats" at all? > >> >> I see this as another example of "Yeah, that's strange XML, but it maps >> 1:1 to a nice and natural binary serialization" :-(. >> > > I think the binary serialization in LLSD is too verbose. My suggestion was > to allow the SD language to express type information to the point where you > can assume the data types from the context. For example, to update > properties on an object, you would need to send, in binary, something like: > > <object-id> <number-of-updates> > (<property-id> <data-for-property>) * > > "number-of-updates" is a small integer, so will typically fit in a byte. > "property-id" is a small integer in the context of "object-id" and thus will > typically take one byte. "data-for-property" for a quaternion stored as four > fp16 is 8 bytes. Thus, the actual storage for this one property (in the > context of where it's used) is 9 bytes; the total message size (assuming > there is only one property value) is 10 bytes plus the size of an object > identifier. > > Note that "data-for-property" is ONLY the raw data, not with any indication > of what that data is. The schema for the entity already describes what that > data is, so you don't need to send that with every binary packet you send. > Thus, in that form of binary serialization, you optimize for transmission > size, which is important for interactive, real-time transmissions. When you > want a more "resilient" format, and size doesn't matter, you simply use XML > instead. > > To clarify: object of id <object-id> has some type. That type has some > schema, that maps small-integer property-id values to string-name and > data-type. Typically, when you encounter an object, you will understand the > type/structure of that object once, but you will see changes to properties > many times. And, in systems which have fewer types than objects (which is > most systems except for SL, I think), you have additional savings by only > needing to encounter a type once, and then that work is re-used for each > object of the same type. > > In this context, the schema for the object type would be described in some > data description language. That language could perhaps be one area where > something like LLSD would make sense -- but LLSD, as it stands, is not up to > the task IMO. Hence, my suggestions (one of which is to support external > schema for transfer encoding efficiency). > > Btw: if you know that you will have the schema when you > serialize/de-serialize objects, then the "more natural" XML encoding would > also map easily to the internal binary format, because all the data you need > about whether things are arrays or ints or whatnot is available through the > schema, so the more natural "<nose-orientation>1,2,3,4</nose-orientation>" > XML serialization would also be possible (and, IMO, desirable). > >> Now back to your question: "should we move the type information to the >> schema, as all people do in XML?". The answer is probably: "no, because we >> want to keep a simple 1:1 mapping with the binary serialization". >> > > But my point is that the binary serialization does not serve the needs of > what a binary serialization should do, which is save transmission space. It > contains both key and datatype, where the verbose key name is typically > inlined instead of sent by reference-to-external-schema. If you can describe > things with external schema, then you can save a LOT of transmission space > where it matters, which is in the binary serialization. > > Sincerely, > > jw > > _______________________________________________ > mmox mailing list > mmox@ietf.org > https://www.ietf.org/mailman/listinfo/mmox >
- [mmox] Compact Binary Serialization Catherine Pfeffer
- Re: [mmox] Compact Binary Serialization Jon Watte
- Re: [mmox] Compact Binary Serialization Dan Olivares
- Re: [mmox] Compact Binary Serialization Jon Watte