Re: [Json] JSON Schema Language

[Austin Wright <[e-mail]>]

> On May 4, 2019, at 23:49, Anders Rundgren <[e-mail]> wrote:
> 
> On 2019-05-05 08:16, Austin Wright wrote:
>>> On May 4, 2019, at 22:07, Anders Rundgren <[e-mail]> wrote:
>>> 
>>> On 2019-05-05 04:51, Austin Wright wrote:
>>>>> On May 4, 2019, at 02:42, Carsten Bormann <[e-mail]> wrote:
>>>>> 
>>>>> Curious:
>>>>> 
>>>>> On that web page, it also says “For consistency, integer JSON numbers SHOULD NOT be encoded with a fractional part.”
>>>>> 
>>>>> What does that mean?
>>>> It’s a non-normative suggestion with the aim of enhancing performance: Many programming languages distinguish between integers and IEEE floats by the presence of a decimal point. While JSON makes no such distinction (all numbers are arbitrary precision decimal), some parsers do make that distinction, and it’s slightly easier to determine if an int32_t is an integer than if a double is an integer.
>>> 
>>> in the C# example I provided before this is (by default) not non-normative, since it threw an exception.   Here is a little bit more detail:
>>> 
>>> class MyObject {
>>>  int Counter;
>>>   .
>>>   .
>>> }
>>> 
>>> Deserializing JSON into this type (which BTW works as as "schema"), REQUIRES "Counter" data to adhere to normal integer notation, including value span.
>>> 
>>> A scheme language should align with the actual use and interpretation of JSON.  This involves alternative serialization formats as well.  As an example monetary values are (probably without exceptions) expressed as JSON Strings since floating point is unsuited for decimal arithmetic.
>>> 
>>> Based on the RFC, one might come to the conclusion that JSON is an inferior information transfer format, but aided by external mapping it actually works extremely well, albeit being slightly verbose :)
>> The Newtonsoft.Json parser behavior would be incompatible with JSON Schema, according to what you provided. But I can’t imagine it’s much of an issue: If the encoder knows the value will always be an integer, why add a fractional part?
> 
> Right, my guess is that not a single mainstream serializer adds a fractional part to a number that can be exactly represented as an integer regardless if they underlying number type is an integer or not.
> 
> 
>> That behavior still seems arbitrary to me, though. Instead of erroring on the first period, all you have to do is error on the first nonzero after the period. Scientific notation is another matter, but presumably it allows 1.9e4 as a valid integer? (I’m not sure, off-hand.)
> 
> To me the whole idea of having a scheme is to create a *clean* description of an object.  Predecessors like XML Schema supported this in (IMO) a pretty good way.  However, this was straightforward since XML doesn't come with a built-in data model.
> 
> JSON was derived from JavaScript (anno 2003 or so) which didn't have an explicit integer type, only an unconstrained Number type.  This obviously creates certain issues when communicating with platforms having a completely different handling of numeric data.  The new proposal doesn't take this is consideration.
> 
> 
>> Probably a better way to approach JSON parsers is to raise an error if it can’t preserve all the information in the source. For example, you can preserve 1e10, 0.5, and even 0.1 (if you permit a small error, such that if you converted the float back to decimal, 0.1 would still be the closest decimal representation). But you can’t preserve 9007199254740993.5 as a double: it’s too many significant figures, and above that value, the precision is below 1. In this case, you would throw.
> 
> A schema should not try to change how parsers work.
> 
> 
>> Parsers that ensure the preservation of the encoded data like this might encourage better use of the JSON types, like for monetary values, even if you are reading into an IEEE floating point.
> 
> See previous point.

Can you elaborate on this? My point is primarily concerned with JSON parsers, not JSON the media type.

Suppose a string were too large to store in memory, most ECMAScript engines would bail with an uncatchable error. The suggestion is to implement this more robustly, like throw an error if a string would be too long, or if a number has too many significant figures; instead of either crashing or silently approximating the number.

But also, why not build a parser that uses a schema to influence how values are parsed? Suppose you have a dynamically typed language, and want to parse some value, e.g. sometimes as an int, sometimes as a bignum, depending on the property. You might provide a JSON Schema to specify this behavior.

>> Still, now that we bring it up, that SHOULD NOT seems suspicious. It’s stating the obvious.
>> Austin.
>>> 
>>> Cheers,
>>> Anders
>>> 
>>>> Cheers,
>>>> Austin.
>>>>> 
>>>>> Grüße, Carsten
>>>>> 
>>>>> 
>>>>>> On May 4, 2019, at 11:36, Austin Wright <[e-mail]> wrote:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> On May 4, 2019, at 00:58, Carsten Bormann <[e-mail]> wrote:
>>>>>>> 
>>>>>>> On May 4, 2019, at 06:47, Anders Rundgren <[e-mail]> wrote:
>>>>>>>> 
>>>>>>>> Example: although 10.0 is a valid JSON Number, in system where you expect
>>>>>>>> an integer, this should be flagged as a syntax error.
>>>>>>> 
>>>>>>> 10.0 is an integer number.
>>>>>>> 
>>>>>>> “Schema Languages” operate at the data model level.  In the JSON data model, there is only one kind of number.
>>>>>>> Of course, the JSON data model is not actually defined in a standard, which is one of the major shortcomings of JSON.
>>>>>>> 
>>>>>> 
>>>>>> JSON Schema handles this exactly the same way, defining a data model [1]. The lexical representation is surjective onto the data model: as you point out, 10.0 is the same as 10, which is an integer.
>>>>>> 
>>>>>> The one case where this might fall apart is if significant digits are important, such that 10.0 is different than 10.00 (i.e., 10.00 is more precise by an order of magnitude). However, I’m not aware of any JSON parsers that keep track of the precision of numbers, even ones that support arbitrary precision. I imagine scientific applications would want to store an explicit precision, since they’re not always powers-of-10 (e.g. {value: 32.0, precision: 0.5}).
>>>>>> 
>>>>>> [1] http://json-schema.org/latest/json-schema-core.html#rfc.section.4.2.1 "4.2.1. Instance Data Model"
>>>>>> 
>>>>>> Austin.