Re: [Json] JSON and int64s - any change in current best practice since I-JSON

Tim Bray <tbray@textuality.com> Wed, 17 January 2024 05:01 UTC

Return-Path: <tbray@textuality.com>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E2CFC15106C for <json@ietfa.amsl.com>; Tue, 16 Jan 2024 21:01:49 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.106
X-Spam-Level:
X-Spam-Status: No, score=-2.106 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=textuality.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y_2F6ZDYpeWh for <json@ietfa.amsl.com>; Tue, 16 Jan 2024 21:01:45 -0800 (PST)
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 152C1C151078 for <json@ietf.org>; Tue, 16 Jan 2024 21:01:44 -0800 (PST)
Received: by mail-ed1-x535.google.com with SMTP id 4fb4d7f45d1cf-559dbeba085so147655a12.0 for <json@ietf.org>; Tue, 16 Jan 2024 21:01:44 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=textuality.com; s=google; t=1705467703; x=1706072503; darn=ietf.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=2YK+hKLuEIJat+rQ0DNPLXFT+hky6Qtnqg3F2lKu9do=; b=PO7sc+XDzAW8WFQmgWD7cIfh0eCNhelBQmrzB57sMkM86bz0QAQnzGqoA/8vJw2Otx vbNIi1Nrs3Y0Q9DPLdDj8YfAzU6iBUbrWi+6hhtMGE+imQG2gIcN9ZjVaeFyQ5V1CFfW 5o9CU0jk1Y2cnzb9fBkX1JOCjXEB5FVOfNyTU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705467703; x=1706072503; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2YK+hKLuEIJat+rQ0DNPLXFT+hky6Qtnqg3F2lKu9do=; b=ZNeSvJ37U9sj9KApEFXaZyrqcrTevwwnzX0vISNi2IF1YAfs9fYWtHMENvrJEEt0YY vxU7Yw2VuZ0bEuoFcYnjAQSDq4FWfw2OXLUu8+dMRe/iRgkNsAET6NMQiIzZeoB0zDdK WZBozHjU0GiDgk2Bu0k8o8K3zCjBPijws+L5N6lh7dD86ehLIKxRERmyObdTCxXf5y/v LGkddjmMo+MKquDbO5PO6ftxtfa4ZdvmUy3xefOMSKqVUSkQt2xvDWx5LKQVYZOb/Ln1 aESgdi0WNWdXKFF9oRplbE3eHjt6IeRIBVgZfqBJntqf1DwjvQ5WnZV8WPqDr68dugGA ZsgA==
X-Gm-Message-State: AOJu0YyqSB6zBL6ypYcAjQUqzgG66l5esbI6qNazttIn8KgHPmZrboi3 I6ranHtH/gcW3O+rx9P7xm2S/9Q16Gu/nyaUx/FrEl830T+Cog==
X-Google-Smtp-Source: AGHT+IGFVAzk8hIzBjAMdDZSbqWMWi64njbISHkYatYHtO9V6fLG8kW/ZOUNnKUuCovyCfU6GliAbhueeHfFYFaazcQ=
X-Received: by 2002:a05:6402:741:b0:559:b870:e868 with SMTP id p1-20020a056402074100b00559b870e868mr943137edy.12.1705467702494; Tue, 16 Jan 2024 21:01:42 -0800 (PST)
Received: from 1064022179695 named unknown by gmailapi.google.com with HTTPREST; Wed, 17 Jan 2024 00:01:41 -0500
Received: from 1064022179695 named unknown by gmailapi.google.com with HTTPREST; Wed, 17 Jan 2024 00:01:38 -0500
MIME-Version: 1.0 (Mimestream 1.2.4)
References: <87527a42-aaac-4f39-b320-05f18a2808c1@codalogic.com> <C31BF4C8-9E6C-48F8-BF7B-D2C379273B3F@tzi.org> <CAHBU6it4SaLawSiBgK9ySkbxjtHE6CX-P3r=hzcVy4ksoQo-Cg@mail.gmail.com> <CAChr6SxHfLW-A1asAndKJz-AiyJv5QP18bi=_bNdKXw7zYHThw@mail.gmail.com> <CAChr6SweYdCWxSABZ7g20Zd-xBFzcK0Ritq53S7WtjSwc-vLmw@mail.gmail.com> <E5A68370-CC2F-4618-AB39-39A382656616@cursive.net>
In-Reply-To: <E5A68370-CC2F-4618-AB39-39A382656616@cursive.net>
From: Tim Bray <tbray@textuality.com>
Date: Wed, 17 Jan 2024 00:01:41 -0500
Message-ID: <CAHBU6itRMhFk-38LPnB=c4vjZuDEp+UVXiQPzKrCwVmugUizag@mail.gmail.com>
To: Joe Hildebrand <hildjj@cursive.net>
Cc: "json@ietf.org" <json@ietf.org>
Content-Type: multipart/alternative; boundary="0000000000009557de060f1d2458"
Archived-At: <https://mailarchive.ietf.org/arch/msg/json/mMiUlnWA8zKobnX_HC8kSrDLd4U>
Subject: Re: [Json] JSON and int64s - any change in current best practice since I-JSON
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/json/>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 17 Jan 2024 05:01:49 -0000

 I would so so love to fix up JSON just a little bit. Lose the
(syntactically superfluous) commas, RFC3733 date literals, add a couple of
numeric types. I can dream the impossible dream.

CBOR’s fine but I’ve never managed to convince myself that the advantages
of binary over text are worth the aggravation.

On Jan 16, 2024 at 8:49:38 PM, Joe Hildebrand <hildjj@cursive.net> wrote:

> Another approach on the horizon is:
>
> https://github.com/tc39/proposal-json-parse-with-source
>
> Still, moving to CBOR is likely to get you much better interoperability,
> as mentioned in this thread.
>
> —
> Joe Hildebrand
>
> On Jan 16, 2024, at 3:04 PM, Rob Sayre <sayrer@gmail.com> wrote:
>
>
> To suggest a positive path forward:
>
>
> https://es.discourse.group/t/cbor-js-and-the-cbor-object/1759
>
>
> That is the better fix in my opinion, because BigInts are not the only
> problem.
>
>
> JavaScript has precise number types now (Uint8Array etc etc), and CBOR can
> represent these well.
>
>
> Browsers also have another serialization method called "Structured Clone",
> which does a lot more than JSON:
>
>
> https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm
>
>
> I mention this one to show that CBOR is totally tractable here.
>
>
> People often try to adjust JSON or write schema systems to deal with these
> issues, but CBOR is a better way forward (imo) because it fixes many issues
> all at once.
>
>
> thanks,
>
> Rob
>
>
>
> On Tue, Jan 16, 2024 at 9:09 AM Rob Sayre <sayrer@gmail.com> wrote:
>
> Hi,
>
>
> I had the same thought as Pete a while back, but found that JS engines
> actually bar you from encoding these as JSON:
>
>
> > let two_55th = BigInt(2) ** BigInt(55)
>
> > two_55th
>
> 36028797018963968n
>
> > JSON.stringify(two_55th)
>
> VM189:1 Uncaught TypeError: Do not know how to serialize a BigInt
>
>     at JSON.stringify (<anonymous>)
>
>     at <anonymous>:1:6
>
>
> I can't find the reference for this one, but the reason they gave was that
> /adding/ support broke too much running code, even though it would be easy
> to add, since the second command there sure does stringify that number.
>
>
> I think the issue was that too many programs rely on JSON.parse returning
> a JavaScript Number type (distinct from BigInt), so you'll run into
> problems like this TypeError:
>
>
> > let my_plain_number = JSON.parse("42")
>
> > my_plain_number
>
> 42
>
> > my_plain_number + two_55th
>
> VM429:1 Uncaught TypeError: Cannot mix BigInt and other types, use
> explicit conversions
>
>     at <anonymous>:1:17
>
>
> So, I-JSON remains correct.
>
>
> thanks,
>
> Rob
>
>
>
> On Tue, Jan 16, 2024 at 7:11 AM Tim Bray <tbray@textuality.com> wrote:
>
> Carsten is right. You probably wouldn’t use JSON if you weren’t likely to
> be interchanging it with other parties over the network, other parties you
> don’t control.  For quite a while, it has been quite likely that the other
> parties use JavaScript, and the limits I-JSON places on numbers are a
> result of this fact. This fact isn’t going away AFAIK.
>
>
> On Jan 16, 2024 at 6:22:31 AM, Carsten Bormann <cabo@tzi.org> wrote:
>
> > Hi Pete,
>
> >
>
> >> On 2024-01-16, at 14:46, Pete Cordell <petejson@codalogic.com> wrote:
>
> >>
>
> >> Hi All,
>
> >>
>
> >> In I-JSON it recommends encoding 64-bit values using a string because
> many parsers (in particular browsers) always represent numbers using
> floating point doubles.
>
> >
>
> > (many parsers == JavaScript’s JSON.parse)
>
> >
>
> >> We're over 8 years on from I-JSON and browsers now support things like
> BigInt.
>
> >
>
> > I’d say the problem wasn’t so much the absence of BigInt, but the fact
> that JavaScript’s JSON.parse always parses JSON numbers as a JavaScript
> Number, which is exact only inside (-2**53, 2**53).
>
> >
>
> >> Therefore, I'd be interested to hear if there is any community change
> of opinion on how 64-bit ints should be handled in JSON.
>
> >
>
> > I think we’d first need wide support for a new equivalent to JSON.parse
> in JavaScript.
>
> >
>
> > Grüße, Carsten
>
> >
>
> > _______________________________________________
>
> > json mailing list
>
> > json@ietf.org
>
> > https://www.ietf.org/mailman/listinfo/json
>
> _______________________________________________
>
> json mailing list
>
> json@ietf.org
>
> https://www.ietf.org/mailman/listinfo/json
>
> _______________________________________________
>
> json mailing list
>
> json@ietf.org
>
> https://www.ietf.org/mailman/listinfo/json
>
>
> _______________________________________________
> json mailing list
> json@ietf.org
> https://www.ietf.org/mailman/listinfo/json
>