[Tools-discuss] idnits 2.16.02 counts line length using bytes and not characters (but xml2rfc 2.34.0 line-wraps counting chars, not bytes)

[Daniel Kahn Gillmor <[e-mail]>]

i tried running idnits 2.16.02 on a local copy of an intermediate build
of draft-autocrypt-lamps-protected-headers (something between -00 and
-01).

idnits complained that one of my lines was > 72 chars long.

It was correct that the line was > 72 *bytes* long, but because the txt
was using UTF-8 (and not all characters are 7-bit clean), the line itself
was <= 72 characters.

the line in question was:

----
   In the diagrams below, "↧" (DOWNWARDS ARROW FROM BAR, U+21A7) is used
----

This .txt file was generated from xml2rfc 2.34.0-1 (as shipped in debian).

For draft -01 i've worked around it by adjusting the text so that it
reflows to have both the chars and bytes <= 72, so i'm not blocked on
the error, but i wanted to document it here (happy to also report it on
a bugtracker if you prefer, let me know).

My concern is that if xml2rfc is line-wrapping its txt output based on
the number of characters, then idnits should *also* be doing its
line-length-counting on the basis of characters, not the number of
bytes.

So i think the two tools should probably be better aligned if we want to
support non-ASCII characters in the published txt documents.

        --dkg