Re: [Json] [jose] EcmaScript V6 - numbers

[Anders Rundgren <[e-mail]>]

On 2015-11-02 03:32, Manger, James wrote:
> Hi Anders,

Hi James,

Disclaimer: I'm not a mathematician, I design and build systems.

Comments in line:


>
> For floating point numbers in JSON, I am not certain that removing digits beyond 15 guarantees a canonical form. Most (64-bit) doubles might have 15.95 digits of precision, but what about the range of "denormal" doubles that have less precision?

The original idea was defining a robust number canonicalization scheme that would be a true subset of ES6 at some expense of precision.
My (maybe somewhat naive) belief is that 15 digits of precision should suffice for some 99.9999999999999% of all use-cases :-)
This is addressed by the parseFloat(value.toPrecision(15)) "workaround".

Anyway, whatever solution we come up with, it must IMO be possible to use today with ES6 as implemented in browsers (modulo apparent bugs...) otherwise absolutely nothing will happen.

I don't see that a few restrictions or constraints would be show-stoppers.


>
> My concern wasn't so much that the last digit might vary, but whether only some implementations would round to shorter forms.

Of course, what we need/want/require is a deterministic scheme.


>   For example, consider three successive 64-bit doubles near 0.3. In hex with a base 2 exponent the values are (java.lang.Double#toHexString(double)):
> 0x1.3333333333332p-2
> 0x1.3333333333333p-2
> 0x1.3333333333334p-2
> The exact decimal values for these are:
> 0.29999999999999993338661852249060757458209991455078125
> 0.299999999999999988897769753748434595763683319091796875
> 0.3000000000000000444089209850062616169452667236328125
> Canonical JSON forms need to be:
> 0.29999999999999993
> 0.3
> 0.30000000000000004
> which have 17, 1, and 17 significant digits; but might some implementations use 17 digits for the middle one as well?
> 0.29999999999999999

This is the same using Java BigDecimal:
  original: 0.29999999999999993338661852249060757458209991455078125
  rounded: 0.300000000000000 to 15 digits of precision
  rounded: 0.2999999999999999 to 16 digits of precision
  rounded: 0.29999999999999993 to 17 digits of precision
  rounded: 0.299999999999999933 to 18 digits of precision

  original: 0.299999999999999988897769753748434595763683319091796875
  rounded: 0.300000000000000 to 15 digits of precision
  rounded: 0.3000000000000000 to 16 digits of precision
  rounded: 0.29999999999999999 to 17 digits of precision
  rounded: 0.299999999999999989 to 18 digits of precision

  original: 0.3000000000000000444089209850062616169452667236328125
  rounded: 0.300000000000000 to 15 digits of precision
  rounded: 0.3000000000000000 to 16 digits of precision
  rounded: 0.30000000000000004 to 17 digits of precision
  rounded: 0.300000000000000044 to 18 digits of precision

So what you are saying is that since some IEEE numbers can be expressed with higher precision than 15.95 digits. this is what should also be done in signed data? I feel a bit leery about that since I don't have full insight in this topic and status of implementations.


>
> I now think that specifying a canonical form as the "shortest correct representation" can work; it does give a unique string for each 64-bit double (it gives 0.3 above). ECMAScript "ToString Applied to the Number Type" is not quite phrased in this way, but it might be equivalent (with the recommended "accurate conversion" version).
>
> I am confident that V8 (ECMAScript in Chrome) produces this "shortest correct representation". See DTOA_SHORTEST in https://chromium.googlesource.com/v8/v8.git/+/master/src/dtoa.h. If a few other implement this form as well, then it looks like the best way to define a canonical form for 64-bit doubles.

I'm sure about that but this appears to be a task for TC 39 rather than the IETF.
This won't happen though unless there is genuine support for a use-case that motivates such a work-item.  Is there?


>
> Hopefully, interoperability with non-ECMAScript languages can be simpler than your es6JsonNumberSerialization(double) function. Ideally the following should work: use ECMAScript's choice of when to include or omit an exponent, and omit trailing zeros (and use lower-case). [may need to add a precision and locale]
>
> 	static String toJsonString(double d)
> 	{
> 		double ad = Math.abs(d);
> 		if (1e-6d <= ad && ad < 1e21d) return String.format("%f", d).replaceFirst("\\.?0++$", "");
> 		else if (ad == 0) return "0";
> 		else return String.format("%g", d).replaceFirst("\\.?0++e", "e");
> 	}

Thanx! Will test ASAP.


>
> Anders,
> Do you still think "the textual representation of numbers MUST be preserved during parsing"?
> I would prefer to drop that and require serialization to use the "shortest correct representation" + specify when to omit an exponent.

Well, this is my "pre-ES6" scheme which indeed isn't strictly be necessary for what we are discussing now.
I guess it at least nicely fits the IETF interoperability mantra "be conservative in what you do, be liberal in what you accept from others", right?

Cheers,
Anders R


>
> --
> James Manger
>
>
>
> -----Original Message-----
> From: jose [mailto:[e-mail]] On Behalf Of Anders Rundgren
> Sent: Monday, 26 October 2015 4:15 PM
> To: Manger, James <[e-mail]>; [e-mail]; [e-mail]
> Subject: Re: [jose] EcmaScript V6 - Defined Property Order
>
> On 2015-10-26 00:10, Manger, James wrote:
>> Hi Anders,
>>
>> I agree that the EcmaScript string format for numbers is a better basis for a canonical JSON format than, say, normalized scientific notation - particularly for the dominant case of integers less than 2^64. However, EcmaScript's ToString(number) doesn't quite give a canonical form. 7.1.12.1 step 5 says "the least significant digit of s is not necessarily uniquely determined by these criteria". EcmaScript guarantees that ToNumber(ToString(x)) gives the same number x, but that is not quite what we need for signing. We need ToString(ToNumber(s)) to give the same string. I guess you could sign the 8 bytes of a 64-bit float, instead of the JSON decimal digits.
> Hi James,
> Thanx for pointing out this, it is apparently always a very good idea testing concepts with other knowledgeable people before you actually start building something :-)
>
> I guess the ES committee wasn't entirely happy about having to adjust their spec. due to improper reliance on JavaScript property order by parts of the development community.  But they probably did the right thing.
>
> I'm thinking in a similar way.  Why let an edge-case spoil all the fun?  Maybe the ES6 vendors implement the same broken ToString algorithm or the improved version mentioned as a note after the section you referred to?  I won't research this issue now because I consider Ecma the sole "owner" of this problem :-)
>
> So this is my (latest) suggestion for an upgraded in-object JSON clear-text signature specification:
>
>       "Due to limitations in the EcmaScript V6 [ECMA-262] specification regarding
>        the ToString(number) method, it is for interoperability reasons RECOMMENDED
>        to utilize a maximum of 18 digits of precision for non-integer Numbers."
>
> It sure isn't pretty but since "business messaging" can't even use JSON/ES numbers for expressing monetary amounts, it is hardly a show-stopper.
>
> Anders Rundgren
>
>
>> James Manger
>>
>> -----Original Message-----
>> From: jose [mailto:[e-mail]] On Behalf Of Anders Rundgren
>> Sent: Monday, 26 October 2015 2:33 AM
>> To: [e-mail]; [e-mail]
>> Subject: Re: [jose] EcmaScript V6 - Defined Property Order
>>
>> Since the ES6 Number type is 64-bit IEEE, there's no need to worry about number canonicalization either if you base the signature system on ES6 which seems like a pretty safe bet.
>>
>> http://www.ecma-international.org/ecma-262/6.0/index.html#sec-tostring-applied-to-the-number-type
>>
>> That is, AFAICT, clear-text in-object JSON signatures are already compatible with ES6 (and I must drop my "number preservation" stuff...).
>>
>> Folks working with constrained devices will probably settle for CBOR.
>>
>> On 2015-10-25 10:08, Anders Rundgren wrote:
>>> http://www.ecma-international.org/ecma-262/6.0/index.html#sec-ordinary-object-internal-methods-and-internal-slots-ownpropertykeys
>>>
>>> I can't say I'm able "deciphering" the ES6 specification but it seems that the largest base of JSON parsers (the browsers), now are compliant with in-object JSON clear-text signature schemes of the kind I have proposed (pushing maybe...), albeit with some (IMO for practical purposes insignificant) limitations:
>>>
>>> - Integer property names doesn't work.
>>> - Numeric values would have to be normalized.
>>>
>>> Java, Python, and C# already manages this as well.
>>>
>>> Yay!
>>>
>>> Anders
>>>
>> _______________________________________________
>> jose mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/jose
> _______________________________________________
> jose mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/jose
>
> _______________________________________________
> jose mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/jose