Re: [Cbor] [Technical Errata Reported] RFC8610 (6527)

[Carsten Bormann <[e-mail]>]

> This ABNF looks correct to me, but it obfuscates more than it clarifies.

I don’t agree — it is a very clear statement of what is allowed.

> I would suggest following the example of the JSON RFC (https://www.rfc-editor.org/rfc/rfc8259.html#section-7) The ABNF there allows arbitrary hex digits (%x75 4HEXDIG), but the text clarifies how non-BMP characters are handled.

RFC 8259 is a product of a process that was more political than technical.  There was a lot of discussion of unpaired surrogates etc., spilling over from UTF-16 land.
There was always a danger of ECMA bursting out of the fragile agreement to align ECMA 404 and RFC 8259, and that precarious situation limited the amount of fixing that could be done between 7159 and 8259.
While RFC 8259 makes it clear that the JSON text itself needs to be UTF-8, it wasn't really possible to summon the courage to write text that explicitly disallows constructing invalid text from escapes in text string data items.

To summarize, RFC 8259 is in many ways not a good model for how to specify things.

As my ABNF proposal shows, it is easy to specify how surrogates can be used in the ABNF, and doing that saves a validation process step in the implementation.

> A similar situation already exists for base64 bytestrings, for which CDDL's ABNF allows invalid base64.

Yes, but it is much harder to do an ABNF that combines all the syntax for whitespace, comments, *and* that for base64.

> It's also not clear to me whether \' should be allowed in text strings,

The text in section 3.1 doesn’t support allowing that:

   o  Text strings are enclosed by double quotation '"' characters.
      They follow the conventions for strings as defined in Section 7 of
      [RFC8259].  […]

> or whether \" should be allowed in byte strings.

It is:

      If unprefixed, the string is
      interpreted as with a text string, except that single quotes must
      be escaped and that the resulting UTF-8 bytes are marked as a byte
      string (major type 2).

Note the absence of a statement like “double quotes must not be escaped”.

> Allowing them would make both types of strings consistent with each other, but prohibiting them would let text strings use the exact same syntax as JSON.

JSON doesn’t have byte strings, but for text strings we indeed should have full alignment.

> And one more issue: the ABNF currently allows CRLF in unprefixed byte strings, and I'm not sure whether that's intentional.

It is.
(And that is important for draft-ietf-cbor-cddl-control.)

BTW:
> FWIW, I'm not actually using CDDL at the moment. I'm using the extended diagnostic notation, and I was using the CDDL ABNF to narrow down my choice of syntax.

Maybe providing ABNF for EDN would be a good idea, after all.
It *is* subtly different in a number of details from CDDL syntax.
One formal interpretation of EDN syntax (albeit not in ABNF form, as it predates the abnftt tool) can be found in https://github.com/cabo/cbor-diag/blob/master/lib/cbor-diag-parser.treetop (and, yes, this contains the equivalent of the surrogate pair handling that is in the ABNF I proposed).

Grüße, Carsten