Re: [Emailcore] A/S outstanding issue #51 (email addresses in HTML forms)

[Barry Leiba <[e-mail]>]

I certainly think that all this is more than just HTML forms.

b

On Thu, Oct 20, 2022 at 2:43 PM Ken Murchison <[e-mail]> wrote:
>
> Thanks Barry,
>
> Are we intending for this new text to apply only within the context of
> Section 3.2 (HTML Forms) or should this be considered more general guidance?
>
> I'm also wondering if the point regarding "plus" addressing is more
> general guidance rather than limited to just HTML forms.
>
>
>
> On 10/20/22 2:34 PM, Barry Leiba wrote:
> >> I look forward to your proposed text.  If you can post it and/or send it
> >> to me, I can get it in the A/S update that I intend to post before the
> >> deadline on Monday.
> > Here is text that I proposed adding after the paragraph Ken proposed
> > for the new Section 3.2:
> >
> > ADD
> >
> > In particular, SMTP specifies that the local-part of an email address
> > is case-sensitive (see Section 2.4 of
> > [I-D.ietf-emailcore-rfc5322bis]):
> >
> >     The local-part of a mailbox MUST BE treated as case sensitive.
> >     Therefore, SMTP implementations MUST take care to preserve the case
> >     of mailbox local-parts.  In particular, for some hosts, the user
> >     "smith" is different from the user "Smith".  However, exploiting the
> >     case sensitivity of mailbox local-parts impedes interoperability and
> >     is discouraged.
> >
> > While case-sensitivity is specified as an absolute requirement, it is
> > important to stress that most implementations do not make case
> > distinctions in local parts (most treat “smith”, “Smith”, and “SMITH”
> > as the same), and most implementations do preserve the case that is
> > received (from SMTP or HTTP, from address books, or from user input).
> > Maximum interoperability will be achieved by keeping local-parts
> > unchanged (and especially making no attempt to change their case in
> > any way) and by assuming that local-parts that differ only in their
> > case probably refer to the same mailbox.  This is particularly
> > important for software that validates user-input fields, where case
> > changes are tempting, but must be avoided.
> >
> > It is also important to note, as we encounter non-ASCII local-parts
> > over time, that case changes are both character-set dependent and
> > language dependent, and attempts to change case without having the
> > full context necessary are likely to be wrong often enough to matter.
> >
> > END
> >
> > I also wonder if, somewhere, we should say that new implementations
> > SHOULD make local-parts that differ only in their case refer to the
> > same mailbox, thus strengthening the "is discouraged" from the SMTP
> > spec.  We might also be able to get away with moving from "is
> > discouraged" to some SHOULD NOT wording in SMTP, while still moving to
> > Internet Standard.
> >
> > Barry
> >
> >> On 10/17/22 10:53 AM, Barry Leiba wrote:
> >>> Process: I think that it we change the case-sensitivity of local-part,
> >>> we are no longer in an Internet Standard path, but would have to go
> >>> back to Proposed Standard.
> >>>
> >>> I think the best approach for us now is to leave the text in 5321bis
> >>> that's in Section 2.4, which discourages case-sensitivity, to put very
> >>> clear text in the AS that actually using case-sensitive local-part is
> >>> bad for interoperability and will break with a lot of current software
> >>> that assume insensitivity, however incorrectly, and to thus have the
> >>> AS highlight that discouragement.
> >>>
> >>> The result would be that the formal grammar would still allow
> >>> case-sensitive local-part and SMTP would still normatively say, "The
> >>> local-part of a mailbox MUST BE treated as case sensitive.  Therefore,
> >>> SMTP implementations MUST take care to preserve the case of mailbox
> >>> local-parts."  (Except that the "BE" should be in lower case... JCK
> >>> please note.)  But it also would still say, "However, exploiting the
> >>> case sensitivity of mailbox local-parts impedes interoperability and
> >>> is discouraged," and the AS would follow up on that part.
> >>>
> >>> I'm working on some text to propose for the AS in line with what I'm suggesting.
> >>>
> >>> Barry
> >>>
> >>> On Mon, Oct 17, 2022 at 10:32 AM John C Klensin <[e-mail]> wrote:
> >>>>
> >>>> --On Monday, 17 October, 2022 14:35 +0100 Alexey Melnikov
> >>>> <[e-mail]> wrote:
> >>>>
> >>>>> Hi John,
> >>>>>
> >>>>> On Mon, Oct 17, 2022, at 2:25 PM, John C Klensin wrote:
> >>>>>> As participant only...
> >>>>> Likewise.
> >>>>>
> >>>>>> --On Monday, 17 October, 2022 14:00 +0100 Alexey Melnikov
> >>>>>> <[e-mail]> wrote:
> >>>>>>
> >>>>>>> Hi John,
> >>>>>>> I agree with you that we should say a bit more about
> >>>>>>> problematic cases. Possible add something like your text
> >>>>>>> after the paragraph that Ken suggested.
> >>>>>>>
> >>>>>>> Some specific comments below:
> >>>>>>>
> >>>>>>> On Fri, Oct 7, 2022, at 9:39 PM, John Levine wrote:
> >>>>>>>> It appears that Ken Murchison  <[e-mail]> said:
> >>>>>>>>> I have crafted the following text for this issue:
> >>>>>>> ...
> >>>>>>>> If we are going to stick our foot into this swamp at all, I
> >>>>>>>> think we should dive in and describe the popular ways that
> >>>>>>>> non-mail systems screw up mail addresses such as
> >>>>>>>>
> >>>>>>>> * Everyone assumes ASCII upper and lower case are
> >>>>>>>> equivalent. Many turn addresses into all upper or all lower
> >>>>>>>> before sending
> >>>>>>> Yes, I think we should this.
> >>>>>> Agreed, but "everyone" is too strong and therein lies the
> >>>>>> problem.  A bit more needs to be said to discourage the
> >>>>>> practices and/or to predict occasional problems when those
> >>>>>> transformations are made.
> >>>>> I think enough systems assume ASCII case-insensitivity that
> >>>>> insisting that they are not is not going to work in many
> >>>>> cases. I am afraid the boat has sailed on enforcing this one.
> >>>> Then someone should be proposing that we change 5321bis, not
> >>>> just make a comment in the A/S.  Either way, this increases my
> >>>> concern about excluding SMTPUTF8 comments/advice from the A/S.
> >>>> Based on the "case sensitive local parts" requirement, the EAI
> >>>> WG decided that it did not need to explicitly insist on that.
> >>>> However, if we say something equivalent to "it is ok to assume
> >>>> that local-parts of addresses are case-insensitive because
> >>>> everyone else does", then we probably need to be clear that, in
> >>>> general, that does not apply to non-ASCII addresses in either
> >>>> the local-part or, if expressed in UTF-8 rather than Punycode
> >>>> encoding, the domain part. The A/S already steps rather far into
> >>>> that swamp by saying that Internationalized Email SHOULD be
> >>>> supported in Section 2.4 (incidentally the citation there is
> >>>> wrong).  And then we probably need to figure out whether those
> >>>> who assume case insensitivity for ASCII also assume it for
> >>>> non-ASCII Latin script strings.  A reasonable, but naive,
> >>>> assumption is that it should ("after all, what difference does a
> >>>> diacritical make?") but the reality is that it does not work for
> >>>> many cases.
> >>>>
> >>>> (( Example for those who have avoided immersion in the i18n
> >>>> swamp: for some languages, in some localities, the upper case of
> >>>> "á" (U+00E1) is "A" (U+0041).   Now, in a context in which
> >>>> SMTPUTF8 addresses are allowed, what is the lower case of
> >>>> "ABC@EFG".  If one assumes, a priori, that is an ASCII string,
> >>>> then "abc@efg" is a reasonable (and correct and unique) answer.
> >>>> But what if the "real" address was "ábc@éfg" and someone got
> >>>> "ABC@EFG" by applying a "drop the diacritical marks when going
> >>>> to upper case" rule?   The Unicode Case Mapping and Case Folding
> >>>> rules prevent doing that, but the SMTPUTF8 specs don't reference
> >>>> them as useful operations.   And, at the risk of invoking an
> >>>> issue that brought about conflicting standards in the IDN world,
> >>>> the character "ß" (U+00DF) does not have a distinct upper case
> >>>> form... except when it does.  Those are just example that should
> >>>> be at least mostly understandable to those reading this: there
> >>>> are cases that are arguably much worse.  ))
> >>>>
> >>>> So, if we are going to say something in the A/S that essentially
> >>>> changes the requirement, we'd better write it very carefully --
> >>>> and probably explicitly include RFC 6530ff in its scope.
> >>>>
> >>>>>>> ...
> >>>> More generally, as non-ASCII email addresses (even ASCII local
> >>>> parts with IDNs expressed in UTF-8 not Punycode) become more
> >>>> prevalent and especially if the A/S is going to put a SHOULD on
> >>>> Internationalized Address support, I am becoming convinced that
> >>>> we would be performing a real disservice to the international
> >>>> email community, as well as nearly contradicting ourselves, by
> >>>> pretending that issues like the above by ignoring the i18n
> >>>> issues and, in particular, saying "ASCII addresses" and assuming
> >>>> the reader will understand all of those subtleties .
> >>>>
> >>>> (A/S co-author hat momentarily back on.)
> >>>> Ken, unless someone sees a way to avoid the i18n issues that I
> >>>> don't and can quickly get what appears to be WG consensus behind
> >>>> it, I believe the next draft should include (at least) a
> >>>> placeholder section after the current Section 4 (" MIME and Its
> >>>> Implications") called "Internationalization of Addresses and
> >>>> Headers and Its Implications" or words to that effect.
> >>>>
> >>>> And I hope that at least some of those who are actively
> >>>> promoting the use of SMTPUTF8 addresses and also following this
> >>>> list will do some writing rather than either expecting me to do
> >>>> it or assuming the correct text will magically appear.
> >>>>
> >>>> best,
> >>>>       john
> >>>>
> >>>>
> >>>>      best,
> >>>>        john
> >>>>
> >>>> --
> >>>> Emailcore mailing list
> >>>> [e-mail]
> >>>> https://www.ietf.org/mailman/listinfo/emailcore
>
> --
> Kenneth Murchison
> Senior Software Developer
> Fastmail US LLC
>