[Cbor] EDN redux

[Carsten Bormann <[e-mail]>]

We didn't have enough time in Vancouver to complete the discussion
about the EDN ABNF in draft-ietf-cbor-edn-literals.

## Explanation: Analog

In Vancouver, Christian Amsüss had a good explanation in the form of
an analog that I would like to try repeating here.

For JSON, RFC 8259 provides an ABNF.
This ABNF describes the grammar for valid JSON texts.
These JSON texts are typically parsed by a JSON parser to yield a
(parsed) JSON value.
The JSON value is not described by RFC 8259 in any formal way, but
intuition (= previous knowledge) about programming languages with
built-in data representation formats (and specifically JavaScript)
makes it relatively clear how the JSON value looks like and is
obtained from the parsed JSON text.

Some application may want to transport a URI in a JSON string.
RFC 3986 provides an ABNF grammar for URIs.
(This was done before JSON was written up, but the sequence here isn't
relevant; it just shows the independence/loose coupling achieved.)

To use URIs in a JSON text, the application parses/validates the JSON
text (e.g., using the ABNF in RFC 8259), constructs the JSON value
(which embeds a JSON string) based on the above intuition (ABNF
doesn't help with that), and then parses/validates the JSON string
(e.g., using the ABNF in RFC 3986).

This procedure is so obvious I'm not aware of any document than even
describes how the two ABNF grammars interact (they don't, directly,
only via the intuition-based transformation of the parsed JSON text
into a JSON value).

## Applying the analog to EDN

We would like to do exactly the same with application-oriented
literals in EDN: represented in sqstr (~ JSON text), parsed/validated
(e.g., using the ABNF in Section 4.1 of EDN), transformed to the
content of the sqstr (same intuition works here right away, as sqstr
is mostly adapted from JSON strings), then parse/validate (e.g. using
the ABNF in 4.2.x or a separate document) for the specific application
oriented literal.

What PR #49 proposes would be as follows in the JSON/URI example:

* Define ABNF that merges the ABNF grammar for strings in JSON texts
  (RFC 8259), the transformation of parsed input into a JSON value,
  and the ABNF grammar for URIs (RFC 3986), into a single piece of
  "single-pass" ABNF.

* Modify the ABNF grammar for JSON to include this new ABNF.

* Build a new JSON parser that includes this modified grammar; make
  sure that such a modified JSON parser is available everywhere where
  JSON is ingested with URIs to be included in JSON strings.

This clearly *can* be done.

It is not what we want to do; we are quite happy with the way JSON is
processed today.
(EDN is an extended form of JSON, by the way, so the analog is
actually rather close here.)

## Side discussions

The side discussion on whether ABNF is used correctly in the WG
specification is somewhat sophistic.
I can't follow this discussion, as it is not about anything that is
material for the use of ABNF and its ecosystem for EDN.

We cited RFC 8288 as an example that corrected the earlier approach of
the spec it obsoleted (RFC 5988), which was conflating the base
grammar of Web links with the grammars for each specific parameter in
a similar way that PR #49 does.

That approach created interoperability problems (as many of us noticed
when implementing RFC 6690, which is based on the single-pass grammar
attempt in RFC 5988).

### Side-side discussion

RFC 8288 describes individual parameter grammars without typing in a
name and an equals sign as in the ABNF production in ABNF's grammar:

   rule           =  rulename defined-as elements c-nl

...essentially only keeping elements and c-nl; one can discuss whether
that description technique is strictly following STD68.
We are not doing this (all ABNF grammars in the WG spec parse
correctly using the grammar for ABNF in RFC 5234), so this point is
not relevant for this discussion.

## Conclusion

I'm proposing that we now decide to keep the approach for describing
EDN that maintains separation between the parsing and transformation
of sqstr on one hand and the grammars for individual
application-oriented literals on the other hand.
This will allow us to use ABNF grammars such as those in RFC 3986 and
RFC 3339 as well as those of future application-oriented literals
already described in RFCs, unchanged.

With that, I'm asking to stop blocking on the proposal made in PR #49
and continue processing the document.

Grüße, Carsten