[Tools-discuss] Exploring ABNF extracts from RFCs

[Dominique Hazael-Massieux <[e-mail]>]

Hi,

(I hope this list is the appropriate place to bring this project up; let 
me know if there is a better place)

Inspired by some of the work happening in the W3C community around 
extracting, curating and publishing reusable data and code from Web 
specifications [1], I have been exploring what this might look like for 
RFCs.

I had initially planned to present and continue that exploration during 
the IETF 120 hackathon [2], but didn't realize the hackathon start was 
moved to Saturday, so ended up working on the project on my own.

The current state of my exploration is visible in two github repos:
   https://github.com/dontcallmedom/rfcref/tree/main/abnf
   https://github.com/dontcallmedom/ABNFary

I have in particular focused on extracting ABNF from (some) RFCs 
(re-using aex on pre-XML RFCs, and a simpler XML-extraction script on 
XML RFCs), and crucially, identifying the dependencies that exist 
between ABNF fragments (where e.g. the URI production defined in RFC3986 
gets "imported" into e.g. RFC9110):
   https://github.com/dontcallmedom/rfcref/tree/main/abnf/dependencies

With these dependencies semi-manually detected and documented, a 
(node.js) tool I wrote allows to build consolidated ABNF files that are 
guaranteed to have all their rules defined:
   https://github.com/dontcallmedom/rfcref/tree/main/abnf/consolidated

(I identified a couple of errata in the process of consolidating some of 
these RFCs [3])

Finally, a built a simple Web interface on top of the said consolidated 
ABNF that allows to see interlinked ABNF rules, with railroad diagrams 
illustrating the "endemic" rules of a given RFC:
   https://dontcallmedom.github.io/ABNFary/?num=9110

The broader motivation for that project is to enable easier and broader 
usage of ABNF definitions (and more generally, data/code provided in 
RFCs); the similar effort in W3C I mentioned [1] has found impactful 
re-use in implementation conformance and interop testing, developer 
documentation, IDEs, libraries, fuzz-testing, spec authoring tools, spec 
validation.

Although the errata I filed show potential, I'm seeking input and 
feedback on whether this project (and in particular its current focus on 
ABNF) is likely to provide opportunities for similar re-use.

And if so, I'm also interested to hear what would be the best way to 
make this a collaborative effort that would align best with the IETF 
community expectations and processes.

(I'm about to disappear for a few weeks, but thought I would start this 
discussion while the project is still fresh in my mind)

Dom

1. https://github.com/w3c/webref/
2. 
https://wiki.ietf.org/en/meeting/120/hackathon#collecting-consolidated-abnf-from-rfcs
3. https://www.rfc-editor.org/errata/eid8040 
https://www.rfc-editor.org/errata/eid8039