[Json] Call for Consensus: Proposed Text for "8.1 Character Encoding"

[Matthew Miller <[e-mail]>]

Hello JSONbis,

The security directorate review discussion has raised the issue of
encoding detection.  The original table from RFC 4627 was removed from
RFC 7159 due to a lack of consensus.  In this latest round, there have
been a number of comments have been made supporting (and against) adding
more guidance than is currently present.

The chair asks for a call on the following from the working group:

1) Does the working group think adding any text on how to detect the
encoding worthwhile?

2a) If such text is worthwhile, is the following proposed text from Nico
Williams acceptable (to be appended to Section 8.1)?

"""
   Implementors MAY count the number of ASCII NULs in the first four
   bytes of any JSON text to detect which of UTF-8, UTF-16, or UTF-32
   the text is encoded in:

    - if the count is zero, then the text is encoded in UTF-8
    - if the count is one or two, then the text is encoded in UTF-16
    - if the count is three, then the text is encoded in UTF-32

   This results from a) JSON texts having to start with an ASCII
   character, b) no unescaped NULs being allowed in JSON strings, and c)
   any type being allowed at the top-level, thus the first character may
   be a double-quote and the second may be any permissible, unescaped
   Unicode codepoint.  An ASCII character requires a NUL-valued byte in
   UTF-16 encoding, three in UTF-32, and none in UTF-8.

"""
2b) If such text is worthwhile but Nico's proposal is not worthwhile,
what would be acceptable?

Please respond by March 16.


Thank you in advance,

-- 
Matthew A. Miller
JSONbis chair