[OPSEC] [IETF Successes and Failures] #3 (component1): xml2rfc: hyphen not escaped in unicode.py

[opsec issue tracker <[e-mail]>]

#3: xml2rfc: hyphen not escaped in unicode.py

 Debian Stretch, uname -a reports:
 Linux maria 4.9.0-4-amd64 #1 SMP Debian 4.9.65-3+deb9u1 (2017-12-23)
 x86_64 GNU/Linux
 Command python3 reports:
 Python 3.8.1 (default, Feb 22 2020, 11:56:23) [GCC 6.3.0 20170516] on
 linux

 When I enter command "xml2rfc -h" repeatedly, it fails half the time with
 error message:

 Traceback (most recent call last):
 File "/usr/bin/xml2rfc", line 7, in <module>
 from xml2rfc.run import main
 File "/mnt/home/rprice/.local/lib/python3.5/site-
 packages/xml2rfc/__init__.py", line 14, in <module>
 from xml2rfc.parser import  XmlRfcError, CachingResolver, XmlRfcParser,
 XmlRfc
 File "/mnt/home/rprice/.local/lib/python3.5/site-
 packages/xml2rfc/parser.py", line 20, in <module>
 from xml2rfc.writers import base
 File "/mnt/home/rprice/.local/lib/python3.5/site-
 packages/xml2rfc/writers/__init__.py", line 2, in <module>
 from xml2rfc.writers.base import RfcWriterError
 File "/mnt/home/rprice/.local/lib/python3.5/site-
 packages/xml2rfc/writers/base.py", line 30, in <module>
 from xml2rfc.util.unicode import ( punctuation, unicode_replacements,
 unicode_content_tags, bare_unicode_tags,
 File "/mnt/home/rprice/.local/lib/python3.5/site-
 packages/xml2rfc/util/unicode.py", line 260, in <module>
 punctuation_re = re.compile(r'[%s]'%''.join(list(punctuation.keys())))
 File "/usr/lib/python3.5/re.py", line 224, in compile
 return _compile(pattern, flags)
 File "/usr/lib/python3.5/re.py", line 293, in _compile
 p = sre_compile.compile(pattern, flags)
 File "/usr/lib/python3.5/sre_compile.py", line 536, in compile
 p = sre_parse.parse(p, flags)
 File "/usr/lib/python3.5/sre_parse.py", line 829, in parse
 p = _parse_sub(source, pattern, 0)
 File "/usr/lib/python3.5/sre_parse.py", line 437, in _parse_sub
 itemsappend(_parse(source, state))
 File "/usr/lib/python3.5/sre_parse.py", line 575, in _parse
 raise source.error(msg, len(this) + 1 + len(that))
 sre_constants.error: bad character range −-“ at position 3

 At line 260 in .../xml2rfc/util/unicode.py I inserted two lines to display
 the value of punctuation.keys()

 259-punctuation.update(unicode_quote_replacements)
 260-import sys
 261-print("unicode.py: list(punctuation.keys()) {}"
 .format(list(punctuation.keys())),file=sys.stderr)
 262-punctuation_re = re.compile(r'[%s]'%''.join(list(punctuation.keys())))

 When xml2rfc succeeded, I saw

 unicode.py: list(punctuation.keys()) = ['\u2002', '-', '‐', '′', '–', '´',
 '…', '’',
 '−', '„', '—', '\u2009', '‚', '‘', '”', '“', '\u2003']
 unicode.py: list(punctuation.keys()) = ['´', '„', '\u2003', '‚', '−', '“',
 '’', '‘', '-', '…',
 '\u2009', '—', '–', '”', '′', '‐', '\u2002']

 When xml2rfc failed, I saw

 unicode.py: list(punctuation.keys()) = ['´', '\u2002', '−', '-', '“', '„',
 '‘', '′',
 '‚', '–', '…', '’', '‐', '”', '—', '\u2009', '\u2003']
 unicode.py: list(punctuation.keys()) = ['‐', '\u2003', '„', '\u2002',
 '\u2009', '‚',
 '—', '’', '−', '…', '‘', '′', '-', '“', '”', '´', '–']

 It looks as if the character "-" is being wrongly interpreted by re as a
 range indicator.
 Perhaps it should be escaped.

 My apologies for the wretched formatting of this message.
 Roger

-- 
----------------------------------+----------------------
 Reporter:  [e-mail]  |      Owner:  somebody
     Type:  defect                |     Status:  new
 Priority:  major                 |  Milestone:
Component:  component1            |    Version:
 Keywords:  re escape hyphen      |
----------------------------------+----------------------

Ticket URL: <https://trac.tools.ietf.org/misc/outcomes/ticket/3>
IETF Successes and Failures <http://tools.ietf.org/misc/outcomes/>
IETF Successes and Failures