[Ietf-and-github] FYI: W3C and GitHub tooling

[Mark Nottingham <[e-mail]>]

I spent some time at W3C TPAC (their yearly plenary event) talking to folks about the tooling that they use on GitHub, to see if there was anything we could leverage (ideas or code), or even collaborate on (everything referenced is Open Source, and they're very amenable to collaboration where it makes sense). This is a bit of a brain dump, but there's lots of interesting potential here, I think; I'll explore what we might do in a followup.

At a high level, their philosophy is that data should be public wherever feasible, and maintained as close as possible to its source. Since they use GitHub for all of their work, that means keeping state in checked-in files wherever possible.

For example, they have a `w3c.json` file that WGs are required to check into their repos; see documentation at:
  https://w3c.github.io/w3c.json.html

This, in turn, is used by several of their tools to track and manage their process. The validate-repos tool:
  https://github.com/w3c/validate-repos
spiders their repos regularly to extract metadata and assure that they are conforming to best practices; see report at:
  https://w3c.github.io/validate-repos/report.html

They also have a repository manager:
   https://github.com/w3c/ash-nazg/
... that helps them create and interact with repos:
  https://labs.w3.org/repo-manager/repos

For example, it validates that a contributor has signed the IPR policy for the appropriate WG, as seen in the 'checks' for this PR:
  https://github.com/w3c/webvtt/pull/470

See attached screenshot for a creation view.

They also have a beta of an exploration API up:
  https://github.com/w3c/gargantua
For example:
  https://www.w3.org/PM/Groups/browse.html?gid=107714
Note that this is pulling all of the data (some of it aggregated into static files) and presenting it in the client; there's no back-end database, per se, for much of it.	

Where there is a backend db, there are also APIs, documented at:
  https://api.w3.org/doc
And
  https://github.com/w3c/w3c-api

They manage spec transitions (roughly, the IESG discussion/approval process) using a dedicated Github repo:
  https://github.com/w3c/transitions
... (see the issues list), and are talking about developing some GitHub-centric tooling to improve that.

Some groups also manage their review process (roughly analogous to our directorates) using tools built on top of GitHub; e.g.,
  http://w3c.github.io/i18n-activity/reviews/
  http://w3c.github.io/i18n-activity/projects/

Repos are spidered for lots of document checks, e.g., for reference quality and statistics:
  https://github.com/tidoust/reffy-service

Repos can also trigger e-mails for notifications of things like document state transitions:
  https://github.com/plehegar/transition-notifier

For more general info, see:
   https://w3c.github.io
... and their labs page:
  https://www.w3.org/2015/labs/

I asked how concerned they were about taking such a dependency upon GitHub; I'd characterise their position as accepting that risk (re-tooling on something like GitLab would incur a significant delay) in return for the capabilities, reach, scalability and reliability they get from GitHub. They do take backups, etc. I didn't ask about whether they're looking at GitHub Actions, but that seems like an interesting area to explore.

Cheers,



--
Mark Nottingham   https://www.mnot.net/