Re: [Json] JSON Schema Language

[Austin Wright <[e-mail]>]

> On May 6, 2019, at 12:24, Nico Williams <[e-mail]> wrote:
> 
> On Mon, May 06, 2019 at 09:32:13AM +0200, Carsten Bormann wrote:
>> On May 6, 2019, at 09:22, Austin Wright <[e-mail]> wrote:
>>> But also, why not build a parser that uses a schema to influence how
>>> values are parsed?
> 
> Because some of us have generic parsers with associated DSLs and we
> don't want to be left out.
> 
> Specifically, in my case, jq https://stedolan.github.io/jq -- something
> of an XSLT/XPath for JSON.
> 
> It's perfectly reasonable to build a jq application that adheres to a
> JSON schema, but it's not reasonable to expect the jq encoder to know to
> do encoding in context-specific ways -- the jq encoder is too generic
> and divorced from the rest of the jq machinery to possibly be able to do
> that.

Left out of what, exactly? See below.

> 
> By pursuing a schema design that requires context-specific parsing or
> encoding behaviors, though you could say that the result looks like
> JSON, it precludes use of existing JSON tooling such as jq.
> 
> Substitute ECMAScript for jq if you like -- the result is the same.
> 
> I strenuously object to this sort of schema.
> 
> As a maintainer of jq, I wouldn't know what to say to users who then
> complain about non-interop with such a schema other than to point to
> this thread and blame the people who allowed it to happen.  But the
> damage to jq would be real and unavoidable.  To avoid that damage, if I
> cannot convince you, then I assure you that I would appeal any finding
> of consensus for such a schema design.
> 
> I'm sorry, but 10.0 is perfectly acceptable as an integer, and MUST be
> accepted as an integer when you expect an integer even if the JSON RFC
> says encoders SHOULD NOT include zero fractional parts.
> 
> I would suggest that if you have a schema that says "and this is an
> integer", the schema MUST accept zero fractional parts, and SHOULD let
> you specify what to do if a non-integer real number is parsed for that
> field (e.g., truncate the fractional part, floor, round).

I’m not suggesting a variance from the JSON semantics.

My suggestion is an alternative to parsers that lose data when they parse JSON documents, for example, ECMAScript's JSON.parse number parsing. You can, of course, always parse a JSON document according to the generic semantics.

The issue here is how does a program parse a JSON document that with a wider range of values than what the program has room for? Either a string that’s too long, an array with too many items, or a number with too many significant figures?

Normally you would have to write your own parser, or use a tokenizer that preserves the lexical values without casting them to native types. Then you perform your own validation in code, and decide how to convert a lexical JSON number into a native number, depending on context.

Right now, programs usually do this in code. My suggestion is to write these rules as a document. The idea of JSON Schema is you make assertions about the document—“this field must be an RFC3339 date string”—and let the parser or application make additional optimizations (in this case, an RFC3339 string will never be more than 25 characters).

> 
>> That is exactly what most of the members of this WG are afraid of when
>> the talk comes to “schemas”.
> 
> +1
> 
>> XML (and later JSON) learned one thing from the problems of SGML:
>> schema-oblivious decoders are more robust than schema-depending ones.
>> (They are, in the first place, and protocol evolution only amplifies
>> this.)
> 
> Not only that!
> 
> Can you imagine using XSLT/XPath if some XML schema requires different
> parsing behavior for different elements?!  It wouldn't be possible to do
> that *and* still be able to use XSLT/XPath because XSLT/XPath processors
> generally have a built-in generic XML parser.  It would destroy the
> utility of XSLT/XPath!
> 
> And that proves that an XML schema of that sort... would yield something
> that is NOT XML, but merely looks like XML.
> 
> The same exact reasoning applies in the case of JSON.  The only
> difference being that JSON doesn't [yet] have a standard equivalent to
> XSLT/XPath, but since generic parsers and encoders exist, and since
> node.jq, JavaScript, ECMAScript, jq, and such exist, the point stands.
> 
>>> Suppose you have a dynamically typed language, and want to parse
>>> some value, e.g. sometimes as an int, sometimes as a bignum,
>>> depending on the property. You might provide a JSON Schema to
>>> specify this behavior.
>> 
>> There goes the generic decoder.
>> 
>> (There is nothing wrong with using a “schema” to specify upconversion
>> from the JSON data model to your application data model.  What I’m
>> objecting to is getting rid of JSON and replacing it with a gazillion
>> dialects that happen to be syntactically compatible with JSON but the
>> semantics of which are controlled by a “schema” mechanism.)

There’s scarcely a completely generic decoder in existence. They should be, but they’ve typically decided it’s easier to make assumptions of one sort or another, especially around numbers.

> 
> +1.
> 
> Nico
> --