Re: [Emailcore] A/S outstanding issue #51 (email addresses in HTML forms)

[John C Klensin <[e-mail]>]

--On Monday, 17 October, 2022 14:35 +0100 Alexey Melnikov
<[e-mail]> wrote:

> Hi John,
> 
> On Mon, Oct 17, 2022, at 2:25 PM, John C Klensin wrote:
>> As participant only...
> 
> Likewise.
> 
>> --On Monday, 17 October, 2022 14:00 +0100 Alexey Melnikov
>> <[e-mail]> wrote:
>> 
>>> Hi John,
>>> I agree with you that we should say a bit more about
>>> problematic cases. Possible add something like your text
>>> after the paragraph that Ken suggested.
>>> 
>>> Some specific comments below:
>>> 
>>> On Fri, Oct 7, 2022, at 9:39 PM, John Levine wrote:
>>>> It appears that Ken Murchison  <[e-mail]> said:
>>>>> I have crafted the following text for this issue:
>>> ...
>>>> If we are going to stick our foot into this swamp at all, I
>>>> think we should dive in and describe the popular ways that
>>>> non-mail systems screw up mail addresses such as
>>>> 
>>>> * Everyone assumes ASCII upper and lower case are
>>>> equivalent. Many turn addresses into all upper or all lower
>>>> before sending
>>> 
>>> Yes, I think we should this.
>> 
>> Agreed, but "everyone" is too strong and therein lies the
>> problem.  A bit more needs to be said to discourage the
>> practices and/or to predict occasional problems when those
>> transformations are made.
> 
> I think enough systems assume ASCII case-insensitivity that
> insisting that they are not is not going to work in many
> cases. I am afraid the boat has sailed on enforcing this one.

Then someone should be proposing that we change 5321bis, not
just make a comment in the A/S.  Either way, this increases my
concern about excluding SMTPUTF8 comments/advice from the A/S.
Based on the "case sensitive local parts" requirement, the EAI
WG decided that it did not need to explicitly insist on that.
However, if we say something equivalent to "it is ok to assume
that local-parts of addresses are case-insensitive because
everyone else does", then we probably need to be clear that, in
general, that does not apply to non-ASCII addresses in either
the local-part or, if expressed in UTF-8 rather than Punycode
encoding, the domain part. The A/S already steps rather far into
that swamp by saying that Internationalized Email SHOULD be
supported in Section 2.4 (incidentally the citation there is
wrong).  And then we probably need to figure out whether those
who assume case insensitivity for ASCII also assume it for
non-ASCII Latin script strings.  A reasonable, but naive,
assumption is that it should ("after all, what difference does a
diacritical make?") but the reality is that it does not work for
many cases.  

(( Example for those who have avoided immersion in the i18n
swamp: for some languages, in some localities, the upper case of
"á" (U+00E1) is "A" (U+0041).   Now, in a context in which
SMTPUTF8 addresses are allowed, what is the lower case of
"ABC@EFG".  If one assumes, a priori, that is an ASCII string,
then "abc@efg" is a reasonable (and correct and unique) answer.
But what if the "real" address was "ábc@éfg" and someone got
"ABC@EFG" by applying a "drop the diacritical marks when going
to upper case" rule?   The Unicode Case Mapping and Case Folding
rules prevent doing that, but the SMTPUTF8 specs don't reference
them as useful operations.   And, at the risk of invoking an
issue that brought about conflicting standards in the IDN world,
the character "ß" (U+00DF) does not have a distinct upper case
form... except when it does.  Those are just example that should
be at least mostly understandable to those reading this: there
are cases that are arguably much worse.  ))

So, if we are going to say something in the A/S that essentially
changes the requirement, we'd better write it very carefully --
and probably explicitly include RFC 6530ff in its scope.
 
>>> ...

More generally, as non-ASCII email addresses (even ASCII local
parts with IDNs expressed in UTF-8 not Punycode) become more
prevalent and especially if the A/S is going to put a SHOULD on
Internationalized Address support, I am becoming convinced that
we would be performing a real disservice to the international
email community, as well as nearly contradicting ourselves, by
pretending that issues like the above by ignoring the i18n
issues and, in particular, saying "ASCII addresses" and assuming
the reader will understand all of those subtleties .

(A/S co-author hat momentarily back on.)
Ken, unless someone sees a way to avoid the i18n issues that I
don't and can quickly get what appears to be WG consensus behind
it, I believe the next draft should include (at least) a
placeholder section after the current Section 4 (" MIME and Its
Implications") called "Internationalization of Addresses and
Headers and Its Implications" or words to that effect.

And I hope that at least some of those who are actively
promoting the use of SMTPUTF8 addresses and also following this
list will do some writing rather than either expecting me to do
it or assuming the correct text will magically appear.

best,
    john


   best,
     john