[Cbor] Re: Consensus call on EDN literals single ABNF

[Ira McDonald <[e-mail]>]

Rohan,

I can't write a substantive comment. I had major shoulder surgery two weeks
ago and have my right arm in a rigid brace 24x7 for eight weeks. I endorse
the WG consensus and Christian and Carsten's replies.  This note took me
ten minutes to write with my left hand on a touchpad on my phone.🙂

On Thu, Jul 25, 2024, 3:05 PM Carsten Bormann <[e-mail]> wrote:

> On 21. Jul 2024, at 18:25, Rohan Mahy <[e-mail]> wrote:
> >
> > Hi Ira,
> > Could you please give your reasoning in a few words?
>
> I’m also not Ira.
>
> What Christian said (606 words).
>
> I have written up my own view below (716 words), if you care.
>
> Grüße, Carsten
>
>
> There are strong technical reasons for the two-layer model.
>
> The original focus of the original edn-literal proposal was to have a
> pluggable syntax for application-specific literals.
> This idea is quite successful and has been embraced already to “plug” a
> number of holes in EDN.
> From an architectural point of view, a pluggable literal syntax fits
> exceedingly well with the whole idea of CBOR tags, which are a pluggable
> extension mechanism for the data model behind the interchange format.
> (Side observation: It took us a lot of energy in 2013 to get the concept
> of tags accepted, because the concept was rather innovative at the time.)
>
> A pluggable literal syntax can only be done cleanly when it is based on a
> common base syntax for all those pluggables.
> This becomes clear when you look at section 3.1.
> The functionality to handle unknown application-extension identifiers is
> rather important for deployability, to minimize friction when introducing
> new identifiers.
> A lot of experience went into getting this right.
>
> Increasing the coupling between the base ABNF and each
> application-extension by moving to a one-layer model would damage the
> proposal by making the base ABNF unstable and by requiring it to change it
> for each new application-extension.
>
> I don't see a point in mashing up the ABNF grammar in the specification to
> make it single-level.
> The argument that this has always been done this way in production parser
> implementation is not at all compelling for our case (the intuition here
> appears to come from SIP, which does require production-quality text-based
> parsing — exactly what we are trying to get rid of with CBOR.  OBTW,
> SIP/SDP is two-layer; you wouldn't munch the SDP syntax into the SIP
> syntax.)
> EDN is not about production parsing, it is a tools and documents syntax.
>
> More importantly, an EDN implementation does not have to be built directly
> from the ABNF the document uses for defining the syntax.
>
> The next little innovation was recognizing that we could treat hex/base64
> literals the same way as the pluggables; this was just an obvious
> simplifying step.
> Actually, the various text-based representations for binary data do make
> great pluggables — there really is no good reason to be limited to
> base16/32/64, and the clear separation even allows us to cleanly defer
> defining base32 because we currently have no implementation experience.
>
> Again, this clean model can be broken up and special-cased and be made
> more complicated in the document, but there would need to be a really good
> reason to do so.
>
> The objective of including ABNF was being able to explain the syntax, and,
> if possible, even to generate code from that.
> (Parser combinators (nom) were mentioned; I don't see a reason why nom
> cannot be used for directly implementing the two layer approach, by the
> way.)
> Doing code generation directly from the ABNF is not the single, normative
> way to do an implementation.
> There is nothing wrong with an implementer who comes up with their own
> grammar, and I wouldn’t mind cultivating a single-level ABNF as a separate
> project.
> Putting this into the main document (and replacing the clean syntax) is
> just premature optimization, like as if TCP and IP had been described in a
> single document — this was certainly possible (and was done this way in
> 1974), but the invention of layers and the separation of functionalities
> was what made IP so powerful.
>
> EDN-literals as defined today has a clear mental model for the pluggable
> part.
> It is eminently easy to check that the syntax makes sense.
> I'd say that swapping out a major part of the grammar is a rather risky
> late change.
> There are a few other CBOR and CDDL drafts that really wait for our
> attention, and I really don’t want to waste more time on this subject now
> that we have a stable basis.
>
> If there were any merit to doing this change, I might be more open to it.
> But the upside is very limited, and it also would be a big regression.
>
> Of course, what we really want is for ABNF itself to specify how the
> two-layer approach works (using the first layer to transform, then parse
> using the second layer).  Stay tuned...
>
>