Re: [dmarc-ietf] Example of Indirect Mail Flow Breakage with p=reject?

[Douglas Foster <[e-mail]>]

Yes, imperfections will always be with us.    That is my point.

Why should we expect that millions of organizations, operating
independently, will produce a result where the good guys always have
perfectly correct information?

My implementation expects problems.    Separating the harmless mistakes
from the spammer is part of my job.   Senders provide me with usable
information 93% of the time.  My job is to make sense of the other
7%, which I do.  Not an unreasonable burden.  As I said previously, for
high-end filtering problems, a really good user quarantine module could
avoid an excess of system administrator labor.

When my problem set is large, I let the unauthenticated messages through,
and hope that my content filtering will catch the bad stuff, which it
usually does.   I review the data after the fact and tune my reputation
database.

As the reputation database grows, the problem gets easier, because:

   - The volume of unauthenticated messages goes down.
   - The acceptable subset of unauthenticated messages become progressively
   less important to the recipient.
   - The acceptable percentage of unauthenticated messages becomes smaller
   and smaller.

All of this means that quarantine delays become an acceptable burden on the
filtering process.

My frustration is that RFC 7208 and RFC 7489 are written as if all data
will be perfect, so no discussion of exceptions is necessary.   Then
product developers implement the same assumption.   The result is "Check
box 1 to block on SPF Fail," and "Check box 2 to block on DMARC Fail."
 Worse yet, the administrator often has to start blocking messages to find
out how much damage results, because there is no option to gather data
before acting on it.   When he does find problems, there is a good chance
that whitelisting around the problem will create security holes that
whitelist potential spam.   We have a global problem of mis-set
expectations and defective products.   The current draft of DMARCbis does
nothing to fix the misunderstanding.

When evaluators implement SPF and DMARC in a way that expects problems,
legitimate senders will be a lot happier, and illegitimate senders will
have to work harder.   Our document(s) need to move the world in that
direction.

Doug Foster


On Wed, Apr 5, 2023 at 7:18 AM Neil Anuskiewicz <[e-mail]>
wrote:

>
>
> On Apr 5, 2023, at 3:58 AM, Douglas Foster <
> [e-mail]> wrote:
>
> 
> The sad thing is that there is no need to do a bandage pull if evaluators
> can learn how to serve the interests of their users properly.   I don't
> throw away any mail based on Sender Authentication failure alone.   But I
> also don't tolerate the idea that I have to accept malicious impersonation
> in order to accept all of the mail that my users should receive.
>
> In this group, I have been arguing a losing cause that DMARC
> authentication can wisely be applied whether a DMARC policy exists or not.
>  After discarding blacklisted message sources, here are my results from
> applying DMARC-like rules to all of my mail:
>
>
> Ideally, what you’re saying is true but many larger organizations had no
> real controls in place, which of course leads to the proliferation of
> shadow IT, including ESP’s sending bulk mail right from the org domain.
> It’s like herding gerbils for the system and security folks to influence
> the account admins. Often an ESP admin is a marketing person which isn’t
> inherently bad as there are some extraordinary marketers. But they often
> feel uncomfortable with techie stuff so choose to pretend that IT and
> security haven’t been asking for help with the offer to the marketing folks
> of screen shares, etc., yet often these account admins don’t show at
> meetings. Then I see tech execs talking more and more about how they’ve
> really tried to go this the right way but unfortunately we need to piss
> some people off to get their attention. Often the right question can help
> though that right question is essentially please play ball. It’s coming
> from the top, we need to have this done within a couple months so I’m going
> to need your cooperation.
>
>
> 61% are aligned with both SPF PASS and DKIM PASS
> 16% are aligned with SPF only
> 16% are aligned with DKIM only
> ----
> 93% aligned with DMARC-like logic
>   4% are authenticated using local policy that allows non-standard
> alignment or overrides a non-PASS SPF result.
> ----
> 97% of all FROM address are verified to my satisfaction
>
> Clearly, I am within reach of 100% verification of the RFC5322.From domain.
>
> I don't know that I receive any mailing list traffic, but this is how it
> would fit into my model:
> - Failure to verify causes the message to be flagged for review.
> - Review indicates that the message is from a mailing list
> - Research determines that the MLM provides reliable Sender Authentication
> and best effort spam filtering, so I do not need to repeat sender
> authentication
> - I create a local policy that accepts any From domain when the SMTP and
> Server information identify the mailing list.
> - Mailing list messages are forwarded to content filtering for normal
> acceptance processing.
>
> So my "stream info" proposal is trying to solve the "research" entry on
> the list above.
>
> If the mailing list cannot be trusted to perform Sender Authentication,
> then I need to implement code which parses the entire set of Received
> headers, ARC headers, and possibly other headers.  I am probably not
> willing put that processing burden on every message to solve a problem for
> a poorly-managed list    I will be easier to refuse the accommodation and
> tell the user to join the list with a Google account.
>
> So I think we need a document that tells evaluators how to "not be
> stupid".    I may be the only one present who can write it, and I could do
> so if the scope is willing to move in that direction.
>
> But as long as there are stupid evaluators, senders have to cope with the
> reality in front of them.
>
> Yes sir, with well written documentation that’s prompted as important
> could help. But at the end of the day, when there are so many people and
> personalities a small percentage of them seem to want to do everything the
> hard way. To each their own.
>
>
> N
>
>
>
>
> On Wed, Apr 5, 2023 at 4:14 AM Neil Anuskiewicz <[e-mail]>
> wrote:
>
>> I’m with Doug on this one. The bandage should be pulled off quickly and
>> sympathy expressed to those who miss backward compatibility. I wouldn’t say
>> utilitarianism is the right frame but here why wouldn’t it be morally right
>> not to mention technically sound to inconvenience and anger the few to
>> create positive downstream effects for the majority.
>>
>> If mailing list managers were also flogged then my opinion would shift
>> right back in favor of the manager. Regardless, I feel gratitude for the
>> work of ml managers but I’m sorry maybe the day has come to discuss making
>> decisions for the community as a whole even if it inconveniences (granted
>> not trivially).
>>
>> I had trouble sleeping so I hope my response holds up as being reasonable
>> when I re read it in the morning. Thanks.
>>
>> Neil
>>
>> On Mar 29, 2023, at 7:01 PM, Douglas Foster <
>> [e-mail]> wrote:
>>
>> 
>> If my cigarette smoke inconveniences 100 people on my plane flight,
>> should I come prepared to go smokeless or should they come prepared with
>> masks?    The mailing list problem is created by mailing list practices,
>> and it is the mailing lists problem to solve the problem they created.
>>
>> We actually have an abundance of options for them right now:
>>
>>    - Do not alter content.
>>    - Mung the From address
>>    - Use ARC to indicate that the list is forwarding and modifying
>>    traffic from someone else.
>>    - As part of the subscription process, require subscribers to get a
>>    filtering exception implemented for the list traffic.
>>    - Use test messages to determine whether a recipient domain blocks on
>>    p=reject, and do conditional munging for only those recipient domains that
>>    need it.
>>
>> In the case of this IETF list, probably zero participants need From
>> munging, but IETF mungs everything from Comcast because they are unwilling
>> to attempt conditional munging.   So Comcast should allow its domain to be
>> impersonated so that IETF does not have to collect and use
>> subscriber-specific information?
>>
>> For any mailing list trust system to work, the mailing list would have to
>> be trustworthy.    Most mailing lists should be able to require 100% SPF
>> PASS on posts, as well as exact match between MailFrom domain and From
>> domain, because posts should be individual contributions.   However, the
>> testimony of knowledgeable people in this group is that most mailing lists
>> cannot be bothered to enforce sender authentication at all.    These lists
>> aren't protecting their subscribers from impersonation fraud, yet they are
>> complaining that evaluators are suspecting them of forwarding impersonation
>> fraud.  That is hypocrisy.
>>
>> Someone please explain to me why everyone should make themselves more
>> vulnerable to ransomware and other attacks so that mailing lists can avoid
>> being inconvenienced and avoid having secure operating practices.
>>
>> Doug Foster
>>
>>
>>
>>
>>
>> On Wed, Mar 29, 2023 at 7:56 PM Barry Leiba <[e-mail]>
>> wrote:
>>
>>> 1. IETF has installed a very ugly workaround to the problem, rewriting
>>> the "from" header field.  It's absolutely a workaround, and not a proper
>>> solution.
>>>
>>> 2. Without the workaround, the real pain is not that a message from
>>> Comcast posted to the list doesn't get to you (though that's true): the
>>> real pain is that if Valimail rejects (bounces) those messages, the Mailman
>>> software will eventually unsubscribe you -- YOU, not the Comcast user --
>>> from the list for exceeding the bounce threshold.
>>>
>>> 3. Even with the workaround, I see, as a list owner, several unsubscribe
>>> notifications a week due to excessive bounces.
>>>
>>> The damage to mail list operations is real, and expecting every mailing
>>> list manager to install an ugly workaround is not the right answer.
>>> Telling those deploying DMARC what interoperability problems an
>>> inappropriate choice of p=reject causes, and telling them not to do that...
>>> is the right answer.
>>>
>>> And, as I said, when they decide that their needs are more important
>>> than those interoperability problems, they have that right, and at least
>>> they will now be making an informed decision.  The standard needs to say
>>> this.
>>>
>>> Barry
>>>
>>>
>>> On Thu, Mar 30, 2023 at 6:41 AM Todd Herr <todd.herr=
>>> [e-mail]> wrote:
>>>
>>>> Colleagues,
>>>>
>>>> Can someone please point me to a mailing list server or other indirect
>>>> mail flow that I might somehow engage with so that I can experience the
>>>> pain of not having a message reach its destination when sent with a policy
>>>> of p=reject?
>>>>
>>>> I post to various IETF mailing lists from my work address, and my
>>>> employer, like Mr. Brotman's, publishes a DMARC record with p=reject.
>>>> Thanks to the work of the folks who manage the IETF mailing list software,
>>>> my participation in these discussions is not hindered in any way; I can
>>>> post to lists, people can reply directly to me if they choose, and I can
>>>> reply to the list and/or to the author of any post without any extra work
>>>> on my part.
>>>>
>>>> This leaves me in a position where I do not appreciate how a DMARC
>>>> policy of p=reject can harm interoperability, or perhaps better stated, I
>>>> do not appreciate that it does harm interoperability. I understand that it
>>>> can, because SPF can fail when mail transits intermediaries and DKIM can
>>>> fail if the intermediary alters the content of the message. That said, I
>>>> cannot recall seeing a bounce attributable to a DMARC failure in the three
>>>> years that I've worked here (nor for the year or two prior when my previous
>>>> employer deployed p=reject) and so I want to be able to send a message that
>>>> would result in such a bounce.
>>>>
>>>> Can anyone help me?
>>>>
>>>> Thanks.
>>>>
>>>> --
>>>>
>>>> *Todd Herr * | Technical Director, Standards and Ecosystem
>>>> *e:* [e-mail]
>>>> *m:* 703.220.4153
>>>>
>>>> This email and all data transmitted with it contains confidential
>>>> and/or proprietary information intended solely for the use of individual(s)
>>>> authorized to receive it. If you are not an intended and authorized
>>>> recipient you are hereby notified of any use, disclosure, copying or
>>>> distribution of the information included in this transmission is prohibited
>>>> and may be unlawful. Please immediately notify the sender by replying to
>>>> this email and then delete it from your system.
>>>> _______________________________________________
>>>> dmarc mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/dmarc
>>>>
>>> _______________________________________________
>>> dmarc mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/dmarc
>>>
>> _______________________________________________
>> dmarc mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/dmarc
>>
>> _______________________________________________
> dmarc mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/dmarc
>
>