Re: [Json] Proposed minimal change for strings

[Bjoern Hoehrmann <[e-mail]>]

* Paul Hoffman wrote:
>Proposal 2 (prohibit unescaped surrogates):
>
>In section 1 (Introduction):
>  A string is a sequence of zero or more Unicode scalar values [UNICODE].

Section 1 describes the JSON data model and the proposed change would
prohibit escaped unpaired surrogates, implying that something like

  JSON.stringify([ JSON.parse('["..."]')[0].substring(0, 140) ]);

occasionally returns a string that is not JSON text, even if the input
to `JSON.parse` is proper JSON text, when there is a non-BMP character
in just the wrong place (`substring` counts UCS-2 code units).

With JSON.stringify this can be avoided using a replace function like

  JSON.stringify(obj, function(key, value) {
    if (typeof value == 'string') {
      return value.replace(/.../g, "\uFFFD");
    }
    return value;
  })

but then you have dataloss, unnecessarily so because you might just be
putting an object into a datastore for later use and ecmascript doesn't
mind unpaired surrogates, and the code is slower and harder to read.

It seems pretty clear to me that decoders are a better place to deal
with unpaired surrogate code points. In environments where they pose no
problem you probably want them to roundtrip, in others substituting is
a better option, and in yet others you might want to fail hard on them.

Prohibiting unpaired surrogate escapes would place the burden to avoid
them on senders, and I have not seen in argument why that is a better
option, all things considered.
-- 
Björn Höhrmann · mailto:[e-mail] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: [phone] · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/