Re: [xml2rfc] Unable to run xml2rfc --help

That appears to be a rather funny bug.
Apparently, the ASCII neutral minus hyphen is in the list of keys. Depending on how the Python hashing random number generator is seeded, that hyphen is going to a position where it can’t be in the regexp or where it can. True Heisenbug. 
(The hyphen needs to be protected to go into a character class, even when the resulting character range seems to be syntactically acceptable.)

Sent from mobile, sorry for terse

> On 29. Dec 2020, at 11:22, Roger Price <roger@rogerprice.org> wrote:
> 
> On Mon, 28 Dec 2020, Carsten Bormann wrote:
> 
>>> On 2020-12-25, at 15:24, Roger Price <roger@rogerprice.org> wrote:
>>> python3.5
>> but this got my attention: Python 3.5 is end-of-life; did you try a newer Python?
> 
> I changed the xml2rfc shebang to #!/usr/bin/python3.8 and ran Python 3.8.1 but it made no difference.
> 
>> I’m out of educated guesses (at least until you are showing more of the traceback),
> 
> The full Python error message is:
> 
> Traceback (most recent call last):
>  File "/usr/bin/xml2rfc", line 7, in <module>
>    from xml2rfc.run import main
>  File "/mnt/home/rprice/.local/lib/python3.5/site-packages/xml2rfc/__init__.py", line
>    14, in <module>
>    from xml2rfc.parser import  XmlRfcError, CachingResolver, XmlRfcParser, XmlRfc
>  File "/mnt/home/rprice/.local/lib/python3.5/site-packages/xml2rfc/parser.py",
>    line 20, in <module> from xml2rfc.writers import base
>  File "/mnt/home/rprice/.local/lib/python3.5/site-packages/xml2rfc/writers/__init__.py",
>    line 2, in <module> from xml2rfc.writers.base import RfcWriterError
>  File "/mnt/home/rprice/.local/lib/python3.5/site-packages/xml2rfc/writers/base.py",
>    line 30, in <module>
>    from xml2rfc.util.unicode import ( punctuation, unicode_replacements,
>    unicode_content_tags, bare_unicode_tags,
>  File "/mnt/home/rprice/.local/lib/python3.5/site-packages/xml2rfc/util/unicode.py",
>    line 260, in <module>
>    punctuation_re = re.compile(r'[%s]'%''.join(list(punctuation.keys())))
>  File "/usr/lib/python3.5/re.py", line 224, in compile
>    return _compile(pattern, flags)
>  File "/usr/lib/python3.5/re.py", line 293, in _compile
>    p = sre_compile.compile(pattern, flags)
>  File "/usr/lib/python3.5/sre_compile.py", line 536, in compile
>    p = sre_parse.parse(p, flags)
>  File "/usr/lib/python3.5/sre_parse.py", line 829, in parse
>    p = _parse_sub(source, pattern, 0)
>  File "/usr/lib/python3.5/sre_parse.py", line 437, in _parse_sub
>    itemsappend(_parse(source, state))
>  File "/usr/lib/python3.5/sre_parse.py", line 575, in _parse
>    raise source.error(msg, len(this) + 1 + len(that))
> sre_constants.error: bad character range …-‚ at position 3
> 
> In file unicode.py at line 260
> 
> punctuation_re = re.compile(r'[%s]'%''.join(list(punctuation.keys())))
> 
> I added the two lines
> 
> import sys
> print("unicode.py: list(punctuation.keys())) = {}".format(list(punctuation.keys())))
> 
> On runs when Python crashes I see outputs:
> 
> unicode.py: list(punctuation.keys()) = ['−', '‐', '′', '\u2002', '’', '´',
>                 '\u2003', '‚', '–', '\u2009', '”', '-', '“', '‘', '…', '„', '—']
> unicode.py: list(punctuation.keys()) = ['„', '”', '—', '–', '…', '-', '\u2009',
>                 '‘', '’', '\u2003', '−', '´', '\u2002', '‚', '′', '‐', '“']
> 
> When the runs succeed I see outputs:
> 
> unicode.py: list(punctuation.keys()) = ['−', '\u2003', '\u2009', '‘', '“', '‚',
>                 '-', '”', '—', '’', '′', '\u2002', '‐', '–', '…', '´', '„']
> unicode.py: list(punctuation.keys()) = ['\u2002', '‐', '′', '\u2003', '−', '´',
>                 '“', '”', '„', '–', '-', '‘', '’', '…', '‚', '\u2009', '—']
> 
> I do not understand why such a list of constants has to be so random. I'm not sure where to look next, but if you would like to see further traces of other functions, just ask. Roger_______________________________________________
> xml2rfc mailing list
> xml2rfc@ietf.org
> https://www.ietf.org/mailman/listinfo/xml2rfc