Re: Syntax

Julian Reschke said:
> The reason why XML's production is complex is that it excludes
> characters that do not belong into identifiers.
>
> The escaping rule quoted above doesn't solve that problem at all; it's
> just an escaping rule.

Right.

This sort of exclusion is not easy to do in syntax. It's easier to do in a
separate restriction (in the C Standard, the syntax simply says you have a
"universal character name" and then there's a separate requirement:

       [#3] Each universal character name in  an  identifier  shall
       designate  a character whose encoding in ISO/IEC 10646 falls
       into one of the ranges specified in annex D.
).

>> Hmmm, how many IETF formats are in Unicode? (Apart from those based
>> only on XML, like Atom in RFC 4287.) ABNF is not, for instance (right,
>> it is not a few format, the RFC is recent but it derives from an older
>> format.)
> I just tried to understand how RFC4234 works with non-ASCII characters,
> and it's not obvious at all. Section 2.4 seems to deal with it but
> really sounds a bit like hand-waving.

I take it as saying that you have to write your grammar to show the
encoding that you want to use, and may want to have alternative grammars
for different contexts. It doesn't let you use non-ASCII characters in
grammars.

In RFC3977 we wrote the following, which seems to be acceptable to IETF:

    UTF8-non-ascii = UTF8-2 / UTF8-3 / UTF8-4
    UTF8-2    = %xC2-DF UTF8-tail
    UTF8-3    = %xE0 %xA0-BF UTF8-tail / %xE1-EC 2UTF8-tail /
                %xED %x80-9F UTF8-tail / %xEE-EF 2UTF8-tail
    UTF8-4    = %xF0 %x90-BF 2UTF8-tail / %xF1-F3 3UTF8-tail /
                %xF4 %x80-8F 2UTF8-tail
    UTF8-tail = %x80-BF

So, putting it together, one approach would be something like this:

    name       = identifier / quoted-name

    identifier = id-initial *(["-"] 1*id-char)
    id-char    = id-initial / DIGIT
    id-initial = ALPHA / UTF8-non-ascii
        ; Each UTF8-non-ascii in an identifier shall designate a character
        ; whose encoding in ISO/IEC 10646 falls into one of the ranges
        ; specified in appendix X.

    quoted-name = DQUOTE 1*q-char DQUOTE
    q-char      = %x21 / %x23-5B / %x5D-7E / UTF-8-non-ascii / q-escape
        ; excludes DQUOTE and BACKSLASH from ASCII
    q-escape    = %x5C.75 4HEXDIG / %x5C.55 8HEXDIG

-- 
Clive D.W. Feather  | Work:  <clive@demon.net>   | Tel:    +44 20 8495 6138
Internet Expert     | Home:  <clive@davros.org>  | Fax:    +44 870 051 9937
Demon Internet      | WWW: http://www.davros.org | Mobile: +44 7973 377646
THUS plc            |                            |

_______________________________________________
Cosmogol mailing list
Cosmogol@ietf.org
https://www1.ietf.org/mailman/listinfo/cosmogol