Re: [Json] JSON and int64s - any change in current best practice since I-JSON

Joe Hildebrand <hildjj@cursive.net> Wed, 17 January 2024 04:49 UTC

Return-Path: <hildjj@cursive.net>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6AEE1C15106F for <json@ietfa.amsl.com>; Tue, 16 Jan 2024 20:49:56 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.107
X-Spam-Level:
X-Spam-Status: No, score=-2.107 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cursive.net
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E5rXyn_7iu5O for <json@ietfa.amsl.com>; Tue, 16 Jan 2024 20:49:52 -0800 (PST)
Received: from mail-qt1-x82a.google.com (mail-qt1-x82a.google.com [IPv6:2607:f8b0:4864:20::82a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F2661C15106C for <json@ietf.org>; Tue, 16 Jan 2024 20:49:51 -0800 (PST)
Received: by mail-qt1-x82a.google.com with SMTP id d75a77b69052e-42a0b13542aso6303291cf.3 for <json@ietf.org>; Tue, 16 Jan 2024 20:49:51 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cursive.net; s=google; t=1705466990; x=1706071790; darn=ietf.org; h=message-id:in-reply-to:to:references:date:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date:message-id :reply-to; bh=otF4S7caN1BKuOpb30Tfvlqbzfl+VfTOHH3eG35M8GU=; b=cqhMiJ2d7JDVYZyRT2WNRWe+obvQGSLEp2T7ebkx0Fypb7d3Yq/EgFafN1P5nGP7lU Ax/geHBHgiYdKI1rWWVAmeDG4gK5HUbp6DLzDAVl7G62soyH1ELDzsEJVqKB9ttLAgg3 weUYdqpl76RIkTEqW22LRlHDrvDLf5HytL6gA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705466990; x=1706071790; h=message-id:in-reply-to:to:references:date:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=otF4S7caN1BKuOpb30Tfvlqbzfl+VfTOHH3eG35M8GU=; b=sh+bFjnfzOxCLUOFM4NPGcOc2gDgVqmKp97atJhbS43HSKSWiqjnipN3EAURbieasl zXIcSMwOW9b7NbHod0NGnM3nbgyNyvQLOG+pcYm/rDfZsSKPyI+9VkqTTq1BXW4Fa34K 8LTqLVHt/pds+Qy0/NahJ5Qb7LiWHVjupktPl8GZq0GYSsyS4/efu5AO27tmONQ8TdMY DouEKPJlP/r00uCZr7tufKAE6PxjAt9rNg2ADt0A7QUcsI5E+JDeGXnlAKAjlQiyzv7P DgKEX0RvHpmBCsyb8bzAface5wG0eKw2gSaUP5greoxO8GoRfIgebnBS2W+a0i8hjgXE odlA==
X-Gm-Message-State: AOJu0YyNNQiGy7DF5y/9JCZMCwkHv8yjQKKercYkgcDJp5Wzm2KTANYv iV94NaSRi+0FpO3o17ry6TLd2NlaEBvr3DEO3brgBn9oPw==
X-Google-Smtp-Source: AGHT+IG4OuzUA2zO9Rag3PUaxZkW6F76hwxST8K9FnJfi5ODh4JXc6fbEhElA3kSGgingVAKNp60WA==
X-Received: by 2002:a05:622a:1015:b0:429:e97d:6aac with SMTP id d21-20020a05622a101500b00429e97d6aacmr5513611qte.32.1705466990003; Tue, 16 Jan 2024 20:49:50 -0800 (PST)
Received: from smtpclient.apple (pool-71-163-33-223.washdc.fios.verizon.net. [71.163.33.223]) by smtp.gmail.com with ESMTPSA id bv11-20020a05622a0a0b00b0042a0952805csm879941qtb.3.2024.01.16.20.49.49 for <json@ietf.org> (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2024 20:49:49 -0800 (PST)
From: Joe Hildebrand <hildjj@cursive.net>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\))
Date: Tue, 16 Jan 2024 23:49:38 -0500
References: <87527a42-aaac-4f39-b320-05f18a2808c1@codalogic.com> <C31BF4C8-9E6C-48F8-BF7B-D2C379273B3F@tzi.org> <CAHBU6it4SaLawSiBgK9ySkbxjtHE6CX-P3r=hzcVy4ksoQo-Cg@mail.gmail.com> <CAChr6SxHfLW-A1asAndKJz-AiyJv5QP18bi=_bNdKXw7zYHThw@mail.gmail.com> <CAChr6SweYdCWxSABZ7g20Zd-xBFzcK0Ritq53S7WtjSwc-vLmw@mail.gmail.com>
To: "json@ietf.org" <json@ietf.org>
In-Reply-To: <CAChr6SweYdCWxSABZ7g20Zd-xBFzcK0Ritq53S7WtjSwc-vLmw@mail.gmail.com>
Message-Id: <E5A68370-CC2F-4618-AB39-39A382656616@cursive.net>
X-Mailer: Apple Mail (2.3774.300.61.1.2)
Archived-At: <https://mailarchive.ietf.org/arch/msg/json/8z75o0kY5g8qSclu6reVh6C-jNM>
Subject: Re: [Json] JSON and int64s - any change in current best practice since I-JSON
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/json/>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 17 Jan 2024 04:49:56 -0000

Another approach on the horizon is:

https://github.com/tc39/proposal-json-parse-with-source

Still, moving to CBOR is likely to get you much better interoperability, as mentioned in this thread.

— 
Joe Hildebrand

> On Jan 16, 2024, at 3:04 PM, Rob Sayre <sayrer@gmail.com> wrote:
> 
> To suggest a positive path forward:
> 
> https://es.discourse.group/t/cbor-js-and-the-cbor-object/1759
> 
> That is the better fix in my opinion, because BigInts are not the only problem.
> 
> JavaScript has precise number types now (Uint8Array etc etc), and CBOR can represent these well.
> 
> Browsers also have another serialization method called "Structured Clone", which does a lot more than JSON:
> https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm
> 
> I mention this one to show that CBOR is totally tractable here.
> 
> People often try to adjust JSON or write schema systems to deal with these issues, but CBOR is a better way forward (imo) because it fixes many issues all at once.
> 
> thanks,
> Rob
> 
> 
> On Tue, Jan 16, 2024 at 9:09 AM Rob Sayre <sayrer@gmail.com> wrote:
> Hi,
> 
> I had the same thought as Pete a while back, but found that JS engines actually bar you from encoding these as JSON:
> 
> > let two_55th = BigInt(2) ** BigInt(55)
> > two_55th
> 36028797018963968n
> > JSON.stringify(two_55th)
> VM189:1 Uncaught TypeError: Do not know how to serialize a BigInt
>     at JSON.stringify (<anonymous>)
>     at <anonymous>:1:6
> 
> I can't find the reference for this one, but the reason they gave was that /adding/ support broke too much running code, even though it would be easy to add, since the second command there sure does stringify that number.
> 
> I think the issue was that too many programs rely on JSON.parse returning a JavaScript Number type (distinct from BigInt), so you'll run into problems like this TypeError:
> 
> > let my_plain_number = JSON.parse("42")
> > my_plain_number
> 42
> > my_plain_number + two_55th
> VM429:1 Uncaught TypeError: Cannot mix BigInt and other types, use explicit conversions
>     at <anonymous>:1:17
> 
> So, I-JSON remains correct.
> 
> thanks,
> Rob
> 
> 
> On Tue, Jan 16, 2024 at 7:11 AM Tim Bray <tbray@textuality.com> wrote:
> Carsten is right. You probably wouldn’t use JSON if you weren’t likely to be interchanging it with other parties over the network, other parties you don’t control.  For quite a while, it has been quite likely that the other parties use JavaScript, and the limits I-JSON places on numbers are a result of this fact. This fact isn’t going away AFAIK.
> 
> On Jan 16, 2024 at 6:22:31 AM, Carsten Bormann <cabo@tzi.org> wrote:
>> Hi Pete,
>> 
>>> On 2024-01-16, at 14:46, Pete Cordell <petejson@codalogic.com> wrote:
>>> 
>>> Hi All,
>>> 
>>> In I-JSON it recommends encoding 64-bit values using a string because many parsers (in particular browsers) always represent numbers using floating point doubles.
>> 
>> (many parsers == JavaScript’s JSON.parse)
>> 
>>> We're over 8 years on from I-JSON and browsers now support things like BigInt.
>> 
>> I’d say the problem wasn’t so much the absence of BigInt, but the fact that JavaScript’s JSON.parse always parses JSON numbers as a JavaScript Number, which is exact only inside (-2**53, 2**53).
>> 
>>> Therefore, I'd be interested to hear if there is any community change of opinion on how 64-bit ints should be handled in JSON.
>> 
>> I think we’d first need wide support for a new equivalent to JSON.parse in JavaScript.
>> 
>> Grüße, Carsten
>> 
>> _______________________________________________
>> json mailing list
>> json@ietf.org
>> https://www.ietf.org/mailman/listinfo/json
> _______________________________________________
> json mailing list
> json@ietf.org
> https://www.ietf.org/mailman/listinfo/json
> _______________________________________________
> json mailing list
> json@ietf.org
> https://www.ietf.org/mailman/listinfo/json