Re: [Emailcore] A/S outstanding issue #51 (email addresses in HTML forms)

[Barry Leiba <[e-mail]>]

> I look forward to your proposed text.  If you can post it and/or send it
> to me, I can get it in the A/S update that I intend to post before the
> deadline on Monday.

Here is text that I proposed adding after the paragraph Ken proposed
for the new Section 3.2:

ADD

In particular, SMTP specifies that the local-part of an email address
is case-sensitive (see Section 2.4 of
[I-D.ietf-emailcore-rfc5322bis]):

   The local-part of a mailbox MUST BE treated as case sensitive.
   Therefore, SMTP implementations MUST take care to preserve the case
   of mailbox local-parts.  In particular, for some hosts, the user
   "smith" is different from the user "Smith".  However, exploiting the
   case sensitivity of mailbox local-parts impedes interoperability and
   is discouraged.

While case-sensitivity is specified as an absolute requirement, it is
important to stress that most implementations do not make case
distinctions in local parts (most treat “smith”, “Smith”, and “SMITH”
as the same), and most implementations do preserve the case that is
received (from SMTP or HTTP, from address books, or from user input).
Maximum interoperability will be achieved by keeping local-parts
unchanged (and especially making no attempt to change their case in
any way) and by assuming that local-parts that differ only in their
case probably refer to the same mailbox.  This is particularly
important for software that validates user-input fields, where case
changes are tempting, but must be avoided.

It is also important to note, as we encounter non-ASCII local-parts
over time, that case changes are both character-set dependent and
language dependent, and attempts to change case without having the
full context necessary are likely to be wrong often enough to matter.

END

I also wonder if, somewhere, we should say that new implementations
SHOULD make local-parts that differ only in their case refer to the
same mailbox, thus strengthening the "is discouraged" from the SMTP
spec.  We might also be able to get away with moving from "is
discouraged" to some SHOULD NOT wording in SMTP, while still moving to
Internet Standard.

Barry

> On 10/17/22 10:53 AM, Barry Leiba wrote:
> > Process: I think that it we change the case-sensitivity of local-part,
> > we are no longer in an Internet Standard path, but would have to go
> > back to Proposed Standard.
> >
> > I think the best approach for us now is to leave the text in 5321bis
> > that's in Section 2.4, which discourages case-sensitivity, to put very
> > clear text in the AS that actually using case-sensitive local-part is
> > bad for interoperability and will break with a lot of current software
> > that assume insensitivity, however incorrectly, and to thus have the
> > AS highlight that discouragement.
> >
> > The result would be that the formal grammar would still allow
> > case-sensitive local-part and SMTP would still normatively say, "The
> > local-part of a mailbox MUST BE treated as case sensitive.  Therefore,
> > SMTP implementations MUST take care to preserve the case of mailbox
> > local-parts."  (Except that the "BE" should be in lower case... JCK
> > please note.)  But it also would still say, "However, exploiting the
> > case sensitivity of mailbox local-parts impedes interoperability and
> > is discouraged," and the AS would follow up on that part.
> >
> > I'm working on some text to propose for the AS in line with what I'm suggesting.
> >
> > Barry
> >
> > On Mon, Oct 17, 2022 at 10:32 AM John C Klensin <[e-mail]> wrote:
> >>
> >>
> >> --On Monday, 17 October, 2022 14:35 +0100 Alexey Melnikov
> >> <[e-mail]> wrote:
> >>
> >>> Hi John,
> >>>
> >>> On Mon, Oct 17, 2022, at 2:25 PM, John C Klensin wrote:
> >>>> As participant only...
> >>> Likewise.
> >>>
> >>>> --On Monday, 17 October, 2022 14:00 +0100 Alexey Melnikov
> >>>> <[e-mail]> wrote:
> >>>>
> >>>>> Hi John,
> >>>>> I agree with you that we should say a bit more about
> >>>>> problematic cases. Possible add something like your text
> >>>>> after the paragraph that Ken suggested.
> >>>>>
> >>>>> Some specific comments below:
> >>>>>
> >>>>> On Fri, Oct 7, 2022, at 9:39 PM, John Levine wrote:
> >>>>>> It appears that Ken Murchison  <[e-mail]> said:
> >>>>>>> I have crafted the following text for this issue:
> >>>>> ...
> >>>>>> If we are going to stick our foot into this swamp at all, I
> >>>>>> think we should dive in and describe the popular ways that
> >>>>>> non-mail systems screw up mail addresses such as
> >>>>>>
> >>>>>> * Everyone assumes ASCII upper and lower case are
> >>>>>> equivalent. Many turn addresses into all upper or all lower
> >>>>>> before sending
> >>>>> Yes, I think we should this.
> >>>> Agreed, but "everyone" is too strong and therein lies the
> >>>> problem.  A bit more needs to be said to discourage the
> >>>> practices and/or to predict occasional problems when those
> >>>> transformations are made.
> >>> I think enough systems assume ASCII case-insensitivity that
> >>> insisting that they are not is not going to work in many
> >>> cases. I am afraid the boat has sailed on enforcing this one.
> >> Then someone should be proposing that we change 5321bis, not
> >> just make a comment in the A/S.  Either way, this increases my
> >> concern about excluding SMTPUTF8 comments/advice from the A/S.
> >> Based on the "case sensitive local parts" requirement, the EAI
> >> WG decided that it did not need to explicitly insist on that.
> >> However, if we say something equivalent to "it is ok to assume
> >> that local-parts of addresses are case-insensitive because
> >> everyone else does", then we probably need to be clear that, in
> >> general, that does not apply to non-ASCII addresses in either
> >> the local-part or, if expressed in UTF-8 rather than Punycode
> >> encoding, the domain part. The A/S already steps rather far into
> >> that swamp by saying that Internationalized Email SHOULD be
> >> supported in Section 2.4 (incidentally the citation there is
> >> wrong).  And then we probably need to figure out whether those
> >> who assume case insensitivity for ASCII also assume it for
> >> non-ASCII Latin script strings.  A reasonable, but naive,
> >> assumption is that it should ("after all, what difference does a
> >> diacritical make?") but the reality is that it does not work for
> >> many cases.
> >>
> >> (( Example for those who have avoided immersion in the i18n
> >> swamp: for some languages, in some localities, the upper case of
> >> "á" (U+00E1) is "A" (U+0041).   Now, in a context in which
> >> SMTPUTF8 addresses are allowed, what is the lower case of
> >> "ABC@EFG".  If one assumes, a priori, that is an ASCII string,
> >> then "abc@efg" is a reasonable (and correct and unique) answer.
> >> But what if the "real" address was "ábc@éfg" and someone got
> >> "ABC@EFG" by applying a "drop the diacritical marks when going
> >> to upper case" rule?   The Unicode Case Mapping and Case Folding
> >> rules prevent doing that, but the SMTPUTF8 specs don't reference
> >> them as useful operations.   And, at the risk of invoking an
> >> issue that brought about conflicting standards in the IDN world,
> >> the character "ß" (U+00DF) does not have a distinct upper case
> >> form... except when it does.  Those are just example that should
> >> be at least mostly understandable to those reading this: there
> >> are cases that are arguably much worse.  ))
> >>
> >> So, if we are going to say something in the A/S that essentially
> >> changes the requirement, we'd better write it very carefully --
> >> and probably explicitly include RFC 6530ff in its scope.
> >>
> >>>>> ...
> >> More generally, as non-ASCII email addresses (even ASCII local
> >> parts with IDNs expressed in UTF-8 not Punycode) become more
> >> prevalent and especially if the A/S is going to put a SHOULD on
> >> Internationalized Address support, I am becoming convinced that
> >> we would be performing a real disservice to the international
> >> email community, as well as nearly contradicting ourselves, by
> >> pretending that issues like the above by ignoring the i18n
> >> issues and, in particular, saying "ASCII addresses" and assuming
> >> the reader will understand all of those subtleties .
> >>
> >> (A/S co-author hat momentarily back on.)
> >> Ken, unless someone sees a way to avoid the i18n issues that I
> >> don't and can quickly get what appears to be WG consensus behind
> >> it, I believe the next draft should include (at least) a
> >> placeholder section after the current Section 4 (" MIME and Its
> >> Implications") called "Internationalization of Addresses and
> >> Headers and Its Implications" or words to that effect.
> >>
> >> And I hope that at least some of those who are actively
> >> promoting the use of SMTPUTF8 addresses and also following this
> >> list will do some writing rather than either expecting me to do
> >> it or assuming the correct text will magically appear.
> >>
> >> best,
> >>      john
> >>
> >>
> >>     best,
> >>       john
> >>
> >> --
> >> Emailcore mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/emailcore