[ogpx] type-system : eliminate closing tags for collections in binary serialization?

[Meadhbh Hamrick <[e-mail]>]

so... just an idea...

the current type system draft, draft-hamrick-vwrap-type-system-00,
requires that maps or arrays that are serialized using the binary
serialization scheme have a closing tag in addition to an opening tag.
for example, to serialize the map:

{
  foo : string,
  bar : integer
}

you would use an opening tag of a left curly brace ('{') followed by a
32 bit count of the number of elements in the map, followed by the two
elements, followed by a closing tag of a right curly brace.

i have a vague concern that the closing tag is unneeded, may confuse
implementers and could lead to security concerns (at worst) or generic
errors (at best.) specifically, what happens when you don't have a
closing tag following the last element in the collection?

i don't think this is a "big deal," and we can certainly provide
guidance for implementers with properly worded statements about how we
think such situations should be handled.

and i think that making this change would require both opensim and LL
to modify their code, so i can totally understand if peeps would
rather keep the closing tag for collections.

but.. i think it would be a good idea to either a) eliminate the
closing tags or b) add some guidance to implementers about handling
encoding errors. i have a separate email coming up for guidance.

so, assuming we wanted to eliminate closing tags, we might want to
change section 4.3 to read thusly.

4.3. Binary Serialization

   The LLSD Binary Serialization is an encoding syntax appropriate for
   situations where high message entropy is required or limiting
   processing power for parsing messages is available.

   Encoding LLSD structured data using the binary serialization scheme
   involves generating tag, (optional) size values, and serialization of
   simple values.  Composite types are serialized by iterating across
   all members of the collection, serializing each simple or composite
   member in turn.  For each element in an
   LLSD structured data object, the following process is used to
   generate a binary output stream of serialized data:

   o  A one octet type tag is emitted to the output stream.  See the
      table below for tag octets.

   o  If the size of the element being serialized is variable (as it
      will be for strings, URIs, arrays and maps), the size or length of
      the element is output to the stream as a network-order 32 bit
      value.  Elements of types with fixed lengths such as undefined
      values, booleans, integers, reals, UUIDs and dates will not
      include size information in the output stream.

   o  Finally, the binary representation of the element is appended to
      the output stream.

   Undefined  Undefined values are serialized with a single exclamation
       point character ('!').  Undefined values append neither size
       information or data to the output stream.

   Boolean  True values are serialized with a single '1' character.
       False values are serialized with a single '0' character.
       Booleans append neither size information or data to the output
       stream.

   Integer  Integer values are serialized by emitting the 'i' character
       to the output stream followed by the four octets representing the
       integer's 32 bits in network order.

   Real  Real values are serialized by emitting the 'r' character to the
       output stream followed by the eight octets representing the real
       value's 64 bits in network order.

   String  String values are serialized by emitting the 's' character to
       the output stream followed by the string's length in octets
       represented as a network-order 32 bit integer, followed by the
       string's UTF-8 encoding.

   UUID  UUID values are serialized by emitting the 'u' character to the
       output stream followed by the sixteen octets representing the
       UUID's 128 bits, with the most significant byte coming first.

   Date  Date values are serialized by emitting the 'd' character to the
       output stream followed by the number of seconds since the start
       of the epoch, represented as a 64-bit real value.

   URI URI values are serialized by emitting the 'l' character to the
       output stream followed by the URI's length in octets represented
       as a network-order 32 bit integer, followed by the binary
       representation of the URI.

   Binary  Binary values are serialized by emitting the 'b' character to
       the output stream followed by the binary array's length in octets
       represented as a network-order 32 bit integer, followed by the
       octets of the binary array.

   Array  Arrays are serialized by emitting the left square bracket
       ('[') character, followed by the count of objects in the array
       represented as a network-order 32 bit integer, followed by each
       array element in order.  Note that compliant implementations MUST
       preserve the order of array elements.

   Map Maps are serialized by emitting the left curly brace ('{')
       character, followed by the count of objects in the map
       represented as a network-order 32 bit integer, followed by each
       key-value element.  Map keys are represented as strings except
       that they use the character 'k' instead of the character 's' as a
       tag.  Note that preserving the order of maps is not REQUIRED.

-cheers
-meadhbh

--
meadhbh hamrick * it's pronounced "maeve"
@OhMeadhbh * http://meadhbh.org/ * [e-mail]