Re: [Json] JSON by example

[Phillip Hallam-Baker <[e-mail]>]

On Thu, May 26, 2016 at 4:50 PM, John Cowan <[e-mail]> wrote:

> Phillip Hallam-Baker scripsit:
>
> > 'Wirthless'
>
> "You may call me by value, and call me 'Worth', or you may call me by
> name, and call me 'Veert'."  --Wirth on how to say his name.
>

The nickname I used was given him by someone just as well known in that
crowd :)



> > Yes, real numbers and integers are probably things you want to be able
> > to treat differently in code because it is very very rare to need or
> > want a real64 number in a protocol.
>
> Okay, an integer encodes the type "JSON number which is an integer",
> and a non-integer encodes the type "JSON number".  Note that this might
> be represented as a Java BigDecimal or equivalent.
>

Yes, there are cases where distinguishing representations is useful. This
does not include big nums for crypto in my view, I would prefer those to be
in hex or base64.

I do think it useful to distinguish integers and reals at the schema level
as the treatment is very different. In fact I do that but I have never ever
used that code. If you were doing scientific stuff, being able to represent
numbers as fractions and such like Wolfram does would be very desirable.



> > But I think that you absolutely want to be able to put binary blobs of
> > data in a stream and distinguish them from strings.
>
> Good idea.  If the string is "deadbeef", it encodes the type of binary
> blobs represented in JSON as a string. :-)
>

Yes, there are a small number of intrinsic types that we use in protocols.
It is slightly larger than JSON gives us but not very much larger.
Specifically:

* Boolean, Integer, String
* Binary  (Base64 (data) in string)
* DateTime (IETF time format in string)
* URI (in string)
* Label (String)
* Reference (string)
* Token (basically a reference but does not require the label to be defined)

I have parser generator tools for pretty much every IETF format (ASN.1, TLS
Schema, XML, DNS, JSON, RFC822). Those are the only ones I have ever found
a need for. (counting OIDs as a subset of URI).



> > That is an interesting one because it comes down to how do you want
> > extensibility to work. In particular, when do you want adding things
> > to be backwards compatible and when do you want to cause things to
> > halt rather than have an application try to act on data it did not
> > fully understand?
>
> I think you want to be able to choose, but I'm not sure how to represent
> it within the JSON-by-example format.  The obvious way is to reserve a
> key like "...", but I haven't wanted to do that.
>

The way that I am doing it now is that I support both as follows:

* Fields can be added to any data structure. These are simply ignored if
the parser does not understand them.

* If the field label changes, the semantics are different and so the
decoder should abort.


My encodings always give the type of an object as a field name. So if I
have a message Hello, the C#, JSON encoding are:

class Hello {
    string Param1, Param2; }

{ "Hello" : { "Param1" : "Value1", "Param2" : "Value2" } }

If you change the protocol so that you can say who you are saying hello to,
this is backwards compatible :

{ "Hello" : { "Param1" : "Value1", "Param2" : "Value2", "To" : "Bob" } }

But if we want to add in a completely different message that replaces
Hello, we use a different tag:

{ "Hello2" : { "Param1" : "Value1", "Param2" : "Value2" } }

This approach makes it very easy to ensure that an interpreter that does
not understand Hello2 does not attempt to do the right thing and get it
wrong.


This works very well and doesn't give the problems that people are getting
wrapped around their axle when they try to use type fields:

{ "Param1" : "Value1", "Param2" : "Value2", "Type" : "Hello" } }

Why should anyone expect "Type" to be more important than the other fields?
It really doesn't make any sense. It also comes in last which means that if
we have a long message and the type field comes last we have to read the
whole thing before we try to make any sense of it.

*Object type should always be specified in a key, never a value.*



> This is actually a little problematic as in JSON any slot can hold
> > null while many languages don't allow integers or booleans to be null.
>
> True.  We can say then that null is always permitted wherever the schema
> allows an array or object.
>

That isn't necessarily what you want though. If you are using a schema,
there are actually four possibilities for a boolean slot 'fred':

true  { "fred" : true }
false  { "fred" : false }
null  { "fred" : null }
empty  {  }

Now you could in theory write a protocol which distinguished between the
four values but I don't think that is very useful at all. I don't think it
is practical to rely on a difference between empty and null.

You could also argue that omitting a value is equivalent to false. But that
is a case where I think you actually should be required to declare it.