Re: [xml2rfc-dev] New xml2rfc release: v2.37.1

[Henrik Levkowetz <[e-mail]>]

Hi Martin,

On 2019-12-13 02:11, Martin Thomson wrote:
> Sorry, I foolishly left that out.
> 
> The code crashes if you have unicode punctuation because of this:

Huh.  I guess this is a case of some Python 3 versions being stricter
than 2.7.  I did put in a test case for this, but in my rush to get the
fix out, and having to rush for an airport, I didn't run the full tox
suite.  Which Python version did you use?


	Henrik

> def downcode_punctuation(str):
>     while True:
>         match = re.search(punctuation_re, str)
>         if not match:
>             return str
>         str = re.sub(match.group(1), punctuation[match.group(1)], str)
> 
> The last line expects a grouping, but the regular expression lacks that grouping.  That's all.
> 
> On Fri, Dec 13, 2019, at 12:01, Henrik Levkowetz wrote:
>> Hi Martin,
>> 
>> Please provide the reason for this change.
>> 
>>    Henrik
>> 
>> > On 13 Dec 2019, at 01:25, Martin Thomson <[e-mail]> wrote:
>> > 
>> > Hi Henrik,
>> > 
>> > One small fix is required for downcode_punctuation():
>> > 
>> > diff --git a/xml2rfc/util/unicode.py b/xml2rfc/util/unicode.py
>> > index 51ffbb5..6f59e6e 100644
>> > --- a/xml2rfc/util/unicode.py
>> > +++ b/xml2rfc/util/unicode.py
>> > @@ -218,21 +218,21 @@ unicode_quote_replacements = {
>> >     u'\u201e': '"',
>> >     u'\u2032': "'",
>> > }
>> > 
>> > punctuation = {
>> >     u'\u2026': '...',
>> > }
>> > punctuation.update(unicode_space_replacements)
>> > punctuation.update(unicode_dash_replacements)
>> > punctuation.update(unicode_quote_replacements)
>> > -punctuation_re = re.compile(r'[%s]'%''.join(list(punctuation.keys())))
>> > +punctuation_re = re.compile(r'([%s])'%''.join(list(punctuation.keys())))
>> > 
>> > unicode_replacements = {
>> >     # Unicode code points corresponding to (x)html entities, also in
>> >     # rfc2629-xhtml.ent
>> >     u'\x09': ' ',
>> >     u'\xa0': ' ',
>> >     u'\xa1': '!',
>> >     u'\xa2': '[cents]',
>> >     u'\xa3': 'GBP',
>> >     u'\xa4': '[currency units]',
>> > 
>> > Aside from that, this meets my immediate needs.
>> > 
>> >> On Fri, Dec 13, 2019, at 00:23, Henrik Levkowetz wrote:
>> >> 
>> >> Hi,
>> >> 
>> >> This is an automatic notification about a new xml2rfc release, 
>> >> v2.37.1, generated when running the mkrelease script.
>> >> 
>> >> Release notes:
>> >> 
>> >> xml2rfc (2.37.1) ietf; urgency=medium
>> >> 
>> >>  * Fixed a bug in the text formatter pagination code where it incorrectly 
>> >>    tried to annotate Comment and PI nodes with page number information.
>> >> 
>> >>  * Updated the v2v3 converter to do essentially what it did before v2.37
>> >>    with respect to unicode downcoding, but with more explicit calls.
>> >> 
>> >>  * Added a base writer method to downcode reference punctuation.
>> >> 
>> >>  * Moved the list of (tag, attr) combinations that permit unicode values 
>> >>    into util.unicode.  Rewrote docwncode_punctuation() to only touch 
>> >>    punctuation.
>> >> 
>> >>  * Restored lost trailing text after <contact> in <t> context for text 
>> >>    output.
>> >> 
>> >> -- Henrik Levkowetz <[e-mail]>  12 Dec 2019 12:42:15 +0000
>> >> 
>> >> The preferred way to install xml2rfc is by doing 'pip install xml2rfc',
>> >> and 'pip install --upgrade xml2rfc' to upgrade.  If there are system-
>> >> installed python modules which pip will not upgrade, you may have to
>> >> use 'pip install --upgrade --no-deps xml2rfc' and install dependencies
>> >> manually.
>> >> 
>> >> The new version is also available through SVN checkout, with
>> >>  'svn checkout http://svn.tools.ietf.org/svn/tools/xml2rfc/tags/cli/2.37.1'
>> >> 
>> >> Regards,
>> >> 
>> >>    Henrik
>> >>    (via the mkrelease script)
>> >> 
>> >> _______________________________________________
>> >> xml2rfc-dev mailing list
>> >> [e-mail]
>> >> https://www.ietf.org/mailman/listinfo/xml2rfc-dev
>> >> 
>> > 
>> > _______________________________________________
>> > xml2rfc-dev mailing list
>> > [e-mail]
>> > https://www.ietf.org/mailman/listinfo/xml2rfc-dev
>> > 
>> 
>>
>