Re: [xml2rfc] idnits 2.16.05 finds non-ascii character —

[Carsten Bormann <[e-mail]>]

> On 2021-02-18, at 21:57, Roger Price <[e-mail]> wrote:
> 
> My XML markup includes an &mdash; in the <title>.  

Which is rather surprising to me, as it should not be possible according to the currently prevailing restrictive interpretation of RFC 7997.

> This looks fine in HTML, but idnits 2.16.05 finds non-ascii utf-8 e28084 in the txt file.

This is intentional as long as we stick to the above restrictive interpretation.

> I fixed the problem by adding the line
> 
>  text = text.replace(u'\u2014', u'-')  # Replace &mdash; EM DASH with HYPHEN-MINUS
> 
> to file $PYTHON/site-packages/xml2rfc/writers/text.py immediately before line
> 
>  text = text.replace(u'\u00A0', u' ')
> 
> Could this be made a permanent change?

I hope not.  Em-dashes look OK in plaintext.   They might be harder to distinguish from en-dashes or neutral hyphen minus in monospaced fonts. 
If we do decide we need a surrogate for em-dashes in monospaced plaintext, that should be ---.
(V2 used -- for its ASCIIification, IIRC.)

> Perhaps there are other character entities which could profit from such replacement.

There are not too many characters that suffer much from a monospaced font.

Grüße, Carsten