Re: [Rfc-markdown] [xml2rfc] [irsg] character sets, was UPDATE regarding <u>

On 2023-03-06, at 00:42, Martin Thomson <mt@lowentropy.net> wrote:
> 
> Carsten, have you considered how the report might differ for <artwork>?  I see a few documents with a lot of symbols in artwork, which might make the report noisier than is ideal.

I’m not sure — in the end these characters also must be in xml2rfc's PDF repertoire.

> That would maybe imply working from XML, which has some interesting implications for those characters that xml2rfc treats specially, like non-breaking spaces and zero-width spaces of different types.

I think it would be interesting to create a variety of targeted text extractors for RFCXML.

(Note that RFCXML does store a lot of text in attributes, which therefore also need to be extracted and sorted into the buckets we want to have.)

Grüße, Carsten

> 
> On Mon, Mar 6, 2023, at 00:19, Carsten Bormann wrote:
>> On 2023-03-04, at 20:03, John Levine <johnl@taugh.com> wrote:
>>> 
>>> One issue is our policy about where any non-ASCII goes, but a separate
>>> issue that Carsten has run into is exotic characters beyond the 3000
>>> or so that are in the fonts we normally use.
>> 
>> Kramdown-rfc 1.6.26 now has an `echars` utility that is more talkative 
>> about unicode blocks and unicode scripts.  Examples below.  I hope this 
>> is useful in our quest to rescue these Unicode arcana from being held 
>> captive by Unicode super-geeks.
>> 
>> Grüße, Carsten
>> 
>> 
>> This is the RFC-to-be where RPC happened to pick a character from the 
>> Dingbats block that started this discussion:
>> 
>> $ echars rfc/authors/rfc9340.txt
>> *** Latin-1 Supplement (Latin)
>> ß: U+00DF    1 LATIN SMALL LETTER SHARP S
>> á: U+00E1    3 LATIN SMALL LETTER A WITH ACUTE
>> ä: U+00E4    1 LATIN SMALL LETTER A WITH DIAERESIS
>> é: U+00E9    5 LATIN SMALL LETTER E WITH ACUTE
>> ó: U+00F3    1 LATIN SMALL LETTER O WITH ACUTE
>> ø: U+00F8    1 LATIN SMALL LETTER O WITH STROKE
>> ü: U+00FC    7 LATIN SMALL LETTER U WITH DIAERESIS
>> *** Latin Extended-A (Latin)
>> ć: U+0107    2 LATIN SMALL LETTER C WITH ACUTE
>> č: U+010D    2 LATIN SMALL LETTER C WITH CARON
>> ę: U+0119    1 LATIN SMALL LETTER E WITH OGONEK
>> ł: U+0142    2 LATIN SMALL LETTER L WITH STROKE
>> š: U+0161    1 LATIN SMALL LETTER S WITH CARON
>> *** General Punctuation (Common)
>> –: U+2013    1 EN DASH
>> *** Dingbats (Common)
>> ➔: U+2794    2 HEAVY WIDE-HEADED RIGHTWARDS ARROW
>> *** Miscellaneous Mathematical Symbols-A (Common)
>> ⟩: U+27E9   61 MATHEMATICAL RIGHT ANGLE BRACKET
>> *** Arabic Presentation Forms-B (Common)
>> : U+FEFF    1 ZERO WIDTH NO-BREAK SPACE
>> 
>> 
>> For comparison, an RFC out of the pre-v3 times:
>> 
>> $ echars rfc/rfc8265.txt
>> *** Basic Latin (Common)
>> "\f": U+000C   25 <control-000C>
>> *** Latin-1 Supplement
>> ¹: U+00B9    1 SUPERSCRIPT ONE (Common)
>> ß: U+00DF    3 LATIN SMALL LETTER SHARP S (Latin)
>> å: U+00E5    1 LATIN SMALL LETTER A WITH RING ABOVE (Latin)
>> *** Latin Extended-A (Latin)
>> ſ: U+017F    2 LATIN SMALL LETTER LONG S
>> *** Greek and Coptic (Greek)
>> Σ: U+03A3    2 GREEK CAPITAL LETTER SIGMA
>> π: U+03C0    2 GREEK SMALL LETTER PI
>> ς: U+03C2    2 GREEK SMALL LETTER FINAL SIGMA
>> σ: U+03C3    2 GREEK SMALL LETTER SIGMA
>> *** Ogham (Ogham)
>>  : U+1680    1 OGHAM SPACE MARK
>> *** Number Forms (Latin)
>> Ⅳ: U+2163    6 ROMAN NUMERAL FOUR
>> *** Mathematical Operators (Common)
>> ∞: U+221E    2 INFINITY
>> *** Miscellaneous Symbols (Common)
>> ♦: U+2666    1 BLACK DIAMOND SUIT
>> *** Alphabetic Presentation Forms (Latin)
>> ﬁ: U+FB01    2 LATIN SMALL LIGATURE FI
>> *** Arabic Presentation Forms-B (Common)
>> : U+FEFF    1 ZERO WIDTH NO-BREAK SPACE
>> 
>> You can see the form feeds we used before RFC8650, as well as the 
>> dreadful BOM already (which is in the Arabic Presentation Forms-B 
>> block, in case you didn’t know that).
>> 
>> _______________________________________________
>> Rfc-markdown mailing list
>> Rfc-markdown@ietf.org
>> https://www.ietf.org/mailman/listinfo/rfc-markdown
> 
> _______________________________________________
> Rfc-markdown mailing list
> Rfc-markdown@ietf.org
> https://www.ietf.org/mailman/listinfo/rfc-markdown