[Tools-discuss] Re: [art] Re: Exploring ABNF extracts from RFCs

[Dominique Hazael-Massieux <[e-mail]>]

Thanks Carsten for the pointers to the CDDL work and the existing 
extractions you've built!

Reactions of my own to Paul's questions based on my automated ABNF 
consolidation experience:

Le 26/07/2024 à 18:09, Paul Kyzivat a écrit :
> There are interesting open issues around abnf dependencies:
> 
> * If I want to use a rule defined in another document, do I add to my 
> abnf *all* of the foreign abnf that contains the rule definitions I 
> need? Or do I only import the specific rules I need and any other rules 
> those depend on?

The consolidater takes the approach of importing only the specific rules 
needed and their dependencies.

> * What to do if the imported abnf has rule names that duplicate ones I 
> have defined in my own abnf? Should the abnf of each document live in a 
> distinct namespace, linked only for the cross referencing rule names? Or 
> should the writer of abnf with dependencies be responsible for avoiding 
> naming conflicts?

The consolidater assumes that each ABNF lives in its own namespace, and 
renames conflicting imported rules (by prefixing them with the rfc name 
they're imported from).

(when I started the project, I assumed name conflicts would be 
considered a bug; but there were too many instances of it to make that 
assumption sustainable)

> * How to deal with the different ways abnf is used in documents. E.g.,
> 
> - the simple case: the document contains a single block of abnf defining 
> the syntax for the protocol that is the subject of the document.
> 
> - the doc has several blocks of abnf, each defining a separate syntax. 
> Rule names may be reused, with different syntax, in each.
> 
> - the doc contains (possibly incomplete) fragments of abnf scattered 
> through the doc text, plus a complete consolidated abnf syntax that 
> likely duplicates some or all of the fragments.
> 
> - the doc contains a complete abnf syntax that is broken into fragments 
> interspersed with text descriptions.

Another case that I had to consider for my tool: RFC 9394 extends two 
versions of IMAP (RFC 3501 and 9051), and does so by extending rules 
that are present ine one version but not the other - I'm treating these 
as profiles in the ABNF (which I think would work to deal with the other 
cases you mention).

> My interest in this came from working on sip for many years. It has a 
> root document RFC3261, and many extension & revision documents. There 
> are extensive dependencies among these documents. The manner of 
> expressing dependencies evolved over time. E.g.,
> 
> - Simple text in the doc prior to the abnf. E.g.,: "The rules foo, bar, 
> and baz are defined in RFCnnn".
> 
> - Saying (or simply assuming) that all the Core Rules from RFC5234 are 
> available.
> 
> - Stating the dependency in comment in the abnf
> 
> - Stating the dependency in an abnf prose-val. E.g.,
>      foo = <foo as defined in RFCnnn>

The consolidater tool as it tries to determine whether a given ABNF can 
be consolidated identifies not only rules that are undefined, but also 
rules that are defined only as prose as indeed they often contain 
references to other RFCs.

> The latter is now my recommended best practice. It results in an abnf 
> that can be fully syntax checked. Ideally we would formalize the syntax 
> within the prose-val so it can be mechanically processed. Or define a 
> new syntax for import, and possibly for namespaces.

An import and a namespace syntax would sure have helped :)

Dom