Re: [vwrap] type-system : binary elements in JSON serializations

[Mark Lentczner <[e-mail]>]

There are several interrelated issues here:

1) There appears to be consensus that the JSON serialization of binary values should be as a JSON string value containing the Base64 encoded version of the binary data. (Rather than the current array of small integers.)

However, at a receiving end, JSON would provide no way to distinguish a LLSD binary value and an LLSD string value that happened to be decodable as Base64.  For example, the JSON string "NoteWell" could be the Base64 encoding of the octets 0x36.8b.5e.59.e9.65

In the usage model of LLSD, the receiving end knows what it wants and reads a value as the type it needs. (This is why all the conversion rules are defined as how to read an X when given a Y in the data.) In order to allow a program wanting binary to read it when transmitted via JSON, we'd need to define a default conversion to Binary from String. 

2) Should we add a conversion to Binary from String? It would be defined as Base64 decode the string if valid, otherwise result in the default Binary value (zero octets).

All the other conversions are symmetric. Adding a conversion to String from Binary implies that there are cases where the receiving end expects a string, but there was a Binary value on the wire (this cannot happen with JSON encoding). I cannot come up with a use case for this conversion. Perhaps if there were a programming environment where binary values are natively handled as Base64 encoded strings? (This is the rationale for the other conversions: Such as environments that don't distinguish between numbers and strings...)

There is also a possible temptation to keep people from: The intention of such a string from binary conversion is NOT to circumvent LLSD's commitment to all text being terms of Unicode strings. Text encoded in some encoding should NOT be sent as Binary, relying on automatic Base64 encoding to get it into some environment via Strings if needed, only to be decoded on the other side. The right thing to do is to send the Unicode string as an LLSD String, transcoding as needed by the chosen serialization method. Admittedly this worry might over blown, but I have seen programmers "just try anything until it works for some broken case" when it comes to character encoding errors. I originally left out the Binary<->String conversions to ensure no one could ever go there!

3) Should we add the reverse conversion: to String from Binary?. It would be simply Base64 encoding the binary values.

	- Mark

Mark Lentczner
Sr. Systems Architect
Technology Integration
Linden Lab

[e-mail]

Zero Linden
[e-mail]