Re: [Json] JSON Schema Language

[Phillip Hallam-Baker <[e-mail]>]

On Mon, May 6, 2019 at 6:38 PM Tim Bray <[e-mail]> wrote:

> 1. I'm pretty sure that we need something better than what we have in the
> area of JSON schemas.  At least, I'm 100% sure that my job at Amazon Web
> Services would be easier, and our customer experiences would be more
> pleasant, if we had something.
>

I agree. And I think most others here agree. The problem is that when we go
down the 'schema' road, we tend to end up with schema languages that become
baroque and more hassle than they are worth.



> 2. One thing schemas are useful for is to syntax-check JSON texts that
> claim to conform to some language specification or another. Obviously no
> schema can ever completely satisfy this requirement - there are always
> things in specifications which are semantic and not addressable by schemas
> - but they can still be super useful.
>
> 3. Another thing they are useful for is for providing help to developers
> working in strongly typed programming languages. With a well-built schema
> it is reasonably straightforward to auto-generate nice idiomatic class
> declarations in modern programming languages, and also to build
> serializers/deserializers that will move data back and forth between JSON
> blobs and programming-language constructs, or fail in a clean deterministic
> way if the JSON fails to match the schema.
>

That is one reason I need a schema language. The other is that I want to
document the protocol design so that I can generate the reference manual
and the reference code from the same source.

This is NOT something I have seen in any proposal other than mine to date.
But it is the one I think most relevant to IETF purposes. For example, this
is a fragment of a schema I am working with right now:

Section 1 "Shared Classes"
Description
|The following classes are used as common elements in
|Mesh profile specifications.a


Structure HostEntry
Description
|Describes a current or pending connection to a Mesh account
String ID
Description
|Unique object instance identifier.

The description sections flow straight into my Internet Drafts.

This is not how I would do the same thing now. That would be more like:

Shared: Section 1 "Shared Classes"
// The following classes are used as common elements in
// Mesh profile specifications.

HostEntry: Class
// Describes a current or pending connection to a Mesh account
ID: String
//Unique object instance identifier.

In fact, I would probably get rid of the colons as they aren't needed by
the parser and are therefore clutter.

The point is that documentation and code should be integrated.


> I mostly fail to understand the debate about jq and integers and so on.
> Clearly, the following is a valid JSON text and will be parsed successfully
> by any JSON parser.
>
> {
>   "foo": 3.0
> }
>

It will be parsed successfully but the problem that comes up are that a
lexical analyzer may legitimately interpret 1.0 as a float and 1 as an
integer and so when the parse tree is traversed end up rejecting the data
as invalid. But that is only an issue if your schema validator doesn't know
how the parse tree handles numbers.


> I imagine that most schema-driven software would first deserialize it into
> a tree,
>

That isn't what I do. I parse directly to the memory data structures. I
don't need the tree structure.

That said, I am thinking of rewriting the code so that it does both at the
same time


probably something like Jackson ObjectMapper's JsonNode, and then apply
> schema constructs to the tree.   I would hope that a sane schema would
> accept this whether a top-level "foo" was required to be an integer or
> double or most other flavors of number, and reject it if "foo" was required
> to be a string or boolean.
>
> Put another way, no JSON schema spec can change the definition of what
> JSON is, or make the built-in type system anything but what it is.
>

But do we actually want a JSON schema spec or a general data schema spec
that supports JSON?

JSON meets pretty much all the requirements I have for writing protocols
except for representing binary data. The application that JSON does not
currently support is data representation because as things stand, floats do
not round trip.

One option would be to write a profile of JSON which ensures floats round
trip. But I can't see that being adopted consistently enough to get
traction. A better solution would be to introduce new float encodings which
do round trip. It wouldn't be JSON but is could use 95% of JSON, be 100%
backwards compatible reading old data and not corrupt data when writing the
new.

Point is that any spec that attempts to solve interop issues by declaring
it can never ever change will fail. Instead of there being one extension,
there will be many and we will end up in the Markdown situation where it
has taken ten years for the market to converge on a common set of tags. And
that mainly because GitHub chose one particular flavor and so
iPython/Jypityr did, etc.