[Ietf-languages] ISO 639 changes (was: RE: Latin Sub tags)

[Doug Ewell <[e-mail]>]

Hugh Paterson III wrote:

> As a matter of practicality, it would be helpful for my processes if
> the format of the data files would remain tab delimited or CSV.

In my perfect Barbie world, the data files would look identical to the current 639-3 files, as I have built many tools that assume that format. The main one takes the 639-3 files as input and builds a mock, partial LSR, from which I extract proposed records that are used in the manual process of creating registration forms. Much of my overall process is manual, which is why it takes me days or weeks to get the BCP 47 changes onto this list, but the tools help a lot.

Of course, we are not in a perfect world, but software can always be updated or rewritten to accommodate new input formats. The important thing is to know in detail what changes will be made, if any, and when, so that we can continue to propagate data from the core standards into the LSR accurately and efficiently.

> Sebastian mentions a "database format". I can imagine that as an SQL
> dump or JSON file, which are valid and in some cases useful formats.

I didn’t see anything about a “database format” — perhaps I missed that — but he did mention databases several times. That doesn’t necessarily mean the publicly available format has to reflect the database architecture. I don’t know what database 639-3/RA is currently using, and thankfully I don’t have to.

> I am curious how the MA will approach the matter of data versioning.
> This is something that some of my colleagues and I have often desired
> from downloaded data (from various sources) in the BCP-47 space.
> Versions allow us to quickly look at the state of our systems and
> determine if we need to update data.

The first line of the LSR is the “File-Date” record, which is intended as a version number of sorts, and should make it clear whether you are up to date. This value has been duplicated eight times since 2006, when there have been errors in the LSR that required an immediate fix. Possibly a “Version” record could be added to further disambiguate, but with only eight such short-lived anomalies out of the 159 versions of the LSR generated since October 2005, this has not been reported to cause any problems.

I do have a local copy of all 159 versions, and all four versions of the LTER, but I almost never refer to them except to look up “fun facts” as in this post.

--
Doug Ewell, CC, ALB | Lakewood, CO, US | ewellic.org