Re: [xml2rfc-dev] New xml2rfc release: v2.37.1

["Martin Thomson" <[e-mail]>]

Sorry, I foolishly left that out.

The code crashes if you have unicode punctuation because of this:

def downcode_punctuation(str):
    while True:
        match = re.search(punctuation_re, str)
        if not match:
            return str
        str = re.sub(match.group(1), punctuation[match.group(1)], str)

The last line expects a grouping, but the regular expression lacks that grouping.  That's all.

On Fri, Dec 13, 2019, at 12:01, Henrik Levkowetz wrote:
> Hi Martin,
> 
> Please provide the reason for this change.
> 
>    Henrik
> 
> > On 13 Dec 2019, at 01:25, Martin Thomson <[e-mail]> wrote:
> > 
> > Hi Henrik,
> > 
> > One small fix is required for downcode_punctuation():
> > 
> > diff --git a/xml2rfc/util/unicode.py b/xml2rfc/util/unicode.py
> > index 51ffbb5..6f59e6e 100644
> > --- a/xml2rfc/util/unicode.py
> > +++ b/xml2rfc/util/unicode.py
> > @@ -218,21 +218,21 @@ unicode_quote_replacements = {
> >     u'\u201e': '"',
> >     u'\u2032': "'",
> > }
> > 
> > punctuation = {
> >     u'\u2026': '...',
> > }
> > punctuation.update(unicode_space_replacements)
> > punctuation.update(unicode_dash_replacements)
> > punctuation.update(unicode_quote_replacements)
> > -punctuation_re = re.compile(r'[%s]'%''.join(list(punctuation.keys())))
> > +punctuation_re = re.compile(r'([%s])'%''.join(list(punctuation.keys())))
> > 
> > unicode_replacements = {
> >     # Unicode code points corresponding to (x)html entities, also in
> >     # rfc2629-xhtml.ent
> >     u'\x09': ' ',
> >     u'\xa0': ' ',
> >     u'\xa1': '!',
> >     u'\xa2': '[cents]',
> >     u'\xa3': 'GBP',
> >     u'\xa4': '[currency units]',
> > 
> > Aside from that, this meets my immediate needs.
> > 
> >> On Fri, Dec 13, 2019, at 00:23, Henrik Levkowetz wrote:
> >> 
> >> Hi,
> >> 
> >> This is an automatic notification about a new xml2rfc release, 
> >> v2.37.1, generated when running the mkrelease script.
> >> 
> >> Release notes:
> >> 
> >> xml2rfc (2.37.1) ietf; urgency=medium
> >> 
> >>  * Fixed a bug in the text formatter pagination code where it incorrectly 
> >>    tried to annotate Comment and PI nodes with page number information.
> >> 
> >>  * Updated the v2v3 converter to do essentially what it did before v2.37
> >>    with respect to unicode downcoding, but with more explicit calls.
> >> 
> >>  * Added a base writer method to downcode reference punctuation.
> >> 
> >>  * Moved the list of (tag, attr) combinations that permit unicode values 
> >>    into util.unicode.  Rewrote docwncode_punctuation() to only touch 
> >>    punctuation.
> >> 
> >>  * Restored lost trailing text after <contact> in <t> context for text 
> >>    output.
> >> 
> >> -- Henrik Levkowetz <[e-mail]>  12 Dec 2019 12:42:15 +0000
> >> 
> >> The preferred way to install xml2rfc is by doing 'pip install xml2rfc',
> >> and 'pip install --upgrade xml2rfc' to upgrade.  If there are system-
> >> installed python modules which pip will not upgrade, you may have to
> >> use 'pip install --upgrade --no-deps xml2rfc' and install dependencies
> >> manually.
> >> 
> >> The new version is also available through SVN checkout, with
> >>  'svn checkout http://svn.tools.ietf.org/svn/tools/xml2rfc/tags/cli/2.37.1'
> >> 
> >> Regards,
> >> 
> >>    Henrik
> >>    (via the mkrelease script)
> >> 
> >> _______________________________________________
> >> xml2rfc-dev mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/xml2rfc-dev
> >> 
> > 
> > _______________________________________________
> > xml2rfc-dev mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/xml2rfc-dev
> > 
> 
>