[Cbor] Re: I-D Action: draft-mcnally-deterministic-cbor-10.html

[Anders Rundgren <[e-mail]>]

On 2024-07-31 10:17, Wolf McNally wrote:
> Anders,
> 
>> On Jul 30, 2024, at 11:52 PM, Anders Rundgren <[e-mail]> wrote:
>>
>> The only thing that remains clear, is that anybody wanting to understand the rationale for numeric reduction would have to wade through literally hundreds of pretty voluminous messages, typically mixing things actually are unrelated, like "you can build dBCOR on top of CDE”.
> 
> It’s really not that hard. Today I had GPT-4o explain it to me from first principles. If it can make sense of this, I think you can.

Hi Wolf,

What can I say? A major reason for my strong aversion against numerical reduction (canonicalization), are my experiences with JSON.

Most RFCs claiming to be based on JSON, are in practice rather based on the crippled version known as I-JSON (RFC 7493).  This looks like a specification mismatch.

Officially, many IETFers consider JSON canonicalization an abomination. In practice the very same IETFers use it anyway, but without mentioning the offending C-word:
https://www.rfc-editor.org/rfc/rfc7638.html#section-3.3

Been there, done that, never going back: https://github.com/cyberphone/cbor-everywhere

CONSISTENCY

It is true that treating 2 and 2.0 as equivalent is an example of consistent handling of numbers.

However, always treating integers as integers and floating point values as floating point values, is also an example of consistency.


Anyway, it doesn't really matter what I think since you have decided to go with numeric reduction no matter what.


SCOPE CONSIDERATIONS

Finally, the scope of a solution may also impact choices.  My s.c. "Enterprise Application" profile is designed for the market constituting of 90%+ all software developers, while dCBOR (according to Laurence), targets a small niche.

Regards,
Anders


> 
> 
> Me: According to proffered schemes for JSON canonicalization, what is the canonical form of the numeric value `2.0`?
> 
> GPT: In JSON canonicalization schemes, the canonical form of the numeric value `2.0` is typically represented as `2`. This representation removes any unnecessary decimal points or trailing zeros, adhering to the principle of representing numbers in their simplest form. Thus, `2.0` becomes `2` in the canonicalized JSON.
> 
> 
> Me: Why does JSON canonicalization uphold the idea of `2.0` being the same as `2`?
> 
> GPT: JSON canonicalization treats `2.0` as equivalent to `2` because it aims to standardize data representation in a way that eliminates ambiguity and redundancy. The key reasons include:
> 
> 1. **Data Consistency**: By removing unnecessary decimal points and zeros, canonicalization ensures that equivalent values are represented consistently, reducing the risk of mismatches or errors when comparing data.
> 
> 2. **Simplicity**: The principle of representing numbers in their simplest form means that there is no distinction between `2` and `2.0` in the JSON format. This simplifies processing and parsing, as the same value is always represented the same way.
> 
> 3. **Interoperability**: JSON is widely used for data interchange, and canonicalization helps ensure that different systems interpret and process the data consistently. By defining a single representation for numbers, it facilitates interoperability across various platforms and programming languages.
> 
> 4. **Avoiding Semantic Ambiguity**: In many programming languages and systems, there is no meaningful difference between `2` and `2.0`—both represent the same numeric value. Canonicalization reflects this by treating them as identical, avoiding any potential confusion over their meaning.
> 
> These reasons contribute to the decision to represent `2.0` as `2` in JSON canonicalization, promoting clarity and uniformity in data representation.
> 
> 
> Me: Setting aside the `number` type in JavaScript, and just considering JSON as a method for data interchange among heterogenous systems, how would you respond to the argument that `2.0` and `2` should be kept distinct because one represents a floating point value and the other represents an integer value?
> 
> GPT: While the argument that `2.0` (floating point) and `2` (integer) should be distinct has technical merit in some contexts, it overlooks the primary purpose of JSON as a data interchange format designed for simplicity and broad compatibility among heterogeneous systems. Here's why maintaining the distinction may not be necessary or beneficial in the context of JSON:
> 
> 1. **Language and System Agnosticism**: JSON is designed to be a lightweight, text-based format that can be easily parsed and generated by a wide range of programming languages and systems. Many of these languages do not differentiate between `2` and `2.0` when parsing JSON, treating both as the same numeric value. This standardization minimizes compatibility issues and simplifies data interchange.
> 
> 2. **Data Normalization**: By normalizing numbers to their simplest form (e.g., converting `2.0` to `2`), JSON canonicalization helps ensure consistent data representation. This is crucial for tasks like digital signatures and data comparison, where any difference in representation, even if semantically insignificant, could lead to false mismatches or verification failures.
> 
> 3. **Pragmatic Considerations**: In practical terms, the distinction between integers and floating-point numbers is often more relevant within specific computational contexts rather than at the level of data interchange. JSON’s purpose is to facilitate the exchange of data, not to enforce specific numerical types. Systems receiving the data can interpret it according to their own needs and constraints.
> 
> 4. **Precision and Accuracy**: For most use cases in JSON, the precision and accuracy differences between integers and floating-point representations are not significant enough to warrant maintaining a distinction.
> 
> In summary, while there is a theoretical basis for distinguishing `2.0` from `2`, JSON prioritizes simplicity and universal accessibility. The distinction between integer and floating-point numbers is not preserved in JSON canonicalization to maintain broad compatibility and avoid unnecessary complexity in data interchange.
> 
> 
> Me: Are you familiar with CBOR/CDE and dCBOR in particular? Search if you’re not sure.
> 
> GPT: *Searched 6 sites* CBOR/CDE (Concise Binary Object Representation/Common Deterministic Encoding) and dCBOR (Deterministic CBOR) are formats designed for efficient and deterministic data serialization. The primary goal of dCBOR is to ensure that semantically equivalent data items produce identical byte streams, which is crucial for applications that rely on data integrity verification, such as cryptographic systems.
> 
> Specifically, dCBOR requires that floating point values without a fractional component (like `2.0`) must be encoded as integers (`2`) to reduce redundancy and ensure uniformity across different systems.
> 
> The distinction between integers and floating point numbers is thus collapsed in dCBOR when a floating point value can be exactly represented as an integer. This decision is based on several considerations:
> 
> 1. **Deterministic Encoding**: Ensuring consistent representation simplifies the verification process, as identical data must result in identical encoding.
> 
> 2. **Efficiency**: Using the reduced form generally reduces data size and enhances transmission efficiency.
> 
> 3. **Practicality**: Most applications do not distinguish between `2.0` and `2` in terms of data processing. This approach simplifies implementations across platforms and ensures compatibility.
> 
> By adhering to these principles, dCBOR ensures a robust and consistent encoding mechanism suitable for environments requiring precise data integrity, such as digital signatures and secure communications.
> 
> 
> Me: Would you mount a defense of dCBOR’s numeric reduction similar to what you used for JSON canonicalization of numeric values?
> 
> GPT: Yes, a defense of dCBOR's numeric reduction can be mounted on similar grounds as the justification for JSON canonicalization of numeric values. Here are the key points:
> 
> 1. **Deterministic Encoding**: The primary goal of dCBOR's numeric reduction is to ensure that data with the same semantic value is represented identically in binary form. This deterministic encoding is critical for systems that require data integrity checks, such as digital signatures or cryptographic operations, where even a slight difference in encoding would result in different hash values, breaking verification processes.
> 
> 2. **Simplicity and Efficiency**: Numeric reduction can reduce data size in many cases, making data transmission and storage more efficient. By encoding values like `2.0` as `2`, dCBOR eliminates unnecessary data, which simplifies the decoding process and reduces the possibility of errors due to inconsistent representations. This also aligns with the practical consideration that, in most applications, the distinction between `2` and `2.0` is irrelevant.
> 
> 3. **Consistency Across Platforms**: Different systems and languages may handle floating-point and integer types differently. By standardizing on a single representation for values that can be expressed both as integers and floating-point numbers, dCBOR ensures consistency across heterogeneous systems. This uniformity is particularly important in distributed systems and environments where data needs to be reliably compared or combined from multiple sources.
> 
> 4. **Avoiding Ambiguity**: The distinction between `2` and `2.0` might suggest a difference in data type (integer vs. floating-point), but in the context of serialization and data interchange, this distinction often lacks practical significance. By reducing numbers to their simplest form, dCBOR avoids ambiguity and ensures that all parties interpret the data the same way.
> 
> Overall, dCBOR's approach to numeric reduction enhances interoperability, data integrity, and efficiency, making it a robust choice for applications where determinism and consistency are paramount. This rationale closely mirrors the reasoning behind JSON canonicalization, where simplicity and universal understanding are prioritized over maintaining distinctions that may not be meaningful in the context of data interchange.
> 
> 
> Anders, if you have any followup questions for GPT I’d be happy to forward them for you.
> 
> ~ Wolf
>