Re: [dmarc-ietf] p=quarantine

[Douglas Foster <[e-mail]>]

Like any security problem, we need to minimize false positives (desired
mail being blocked) and false negatives (unwanted or malicious mail being
allowed).  ARC will hopefully address the false positives, but the false
negative issue remains.   The situation does not seem hopeless, but
the topic does appear to be underdeveloped.   Here are some observations:

SPF PASS means the message was received directly from the stated domain,
but it does not indicate whether or not the stated domain was the
originator, since the SMTP address may have been rewritten.   Similarly,
SPF FAIL indicates that the message was not received directly from the
stated domain, but it cannot distinguish between a message that originated
from an unauthorized source and a message that originated from an
authorized source but was then forwarded without SMTP rewrite.    The FAIL
could also be the result of a DNS configuration error, such as a domain
that changed hosting services without changing their SPF policy record.
The issues with failed DKIM signatures and From rewrite are similar.

A forwarded message may have:
- no address rewrite causing SPF FAIL but no masking of the
originator address,
- transparent address rewrite using SRS or ARC, or
- non-transparent replacement of prior SMTP and/or From addresses, without
SRS or ARC clues being included (as this list).

Consequently, the only reliable way to detect an untrusted source behind a
forwarder is to evaluate the entire message header chain.

Blacklisting
---------------
For purposes of filtering unwanted messages, verifiable header data is not
required.   (If someone wants to spoof an untrusted sender, I am happy to
consider that spoofer untrusted also.)   The Received chain can be checked
for IP addresses and DNS names with negative reputation.   Similarly, any
Received-SPF, Authentication-Results, ARC-Authenticated-Results,
Authenticated-As or similar headers can be checked for email domain names
with negative reputation.   All of this can and should be done regardless
of the SPF and DMARC status, because SPF and DMARC are designed around
direct mail flows, and SPF and DMARC pass can be achieved by pretending to
be a direct mail flow.

An additional test is to count the number of organizations that handled the
message along the delivery path.    First, consolidate out the
Received headers with private IPs or loopback IPs, then (probably) drop any
messages transferred over LMTP, and any transfers where the ReverseDNS and
HELO domains are unchanged in the handoff.   What should be left is a small
number of entries where the FROM information indicates the IP and hostnames
of the sending organization and the HELO name should be an MX of the
receiving organization.   (SPF checks and Forward-confirmed DNS checks can
then be performed on this data.)

For direct messages, I assume that the maximum number of organization
boundaries should be three:
- Cloud-based SMTP submitting client to mailstore organization,
- mailstore organization to cloud-based outbound filtering service,
- then outbound filtering service to destination domain.
Additionally, the cloud-based outbound filtering services are relatively
few and well known, which should simplify determining whether an
intermediate organization is a filtering service or a forwarding service.

It seems axiomatic that a forwarder will have an unreliable reputation with
recipients.   If the forwarder blocks a message, the recipient system will
not directly know that it was blocked, whether the blocked message was
wanted or unwanted.   But if the forwarder allows a message that the
recipient does not want, the forwarder takes a hit on its reputation.   An
automated forwarding service has an incentive to avoid false positives, so
it is almost certain to have a spam filtering policy which is weaker than
the final recipient system.    Rather than blaming the forwarding service
for unwanted messages getting through, the evaluator should look at the
entire received chain.

Whitelisting
---------------
It is much riskier to perform whitelisting based on the header data,
because one must allow for the possibility that a nation-state attacker
will construct a convincing but fraudulent header chain, if they perceive
that doing so will succeed.   But I cannot imagine whitelisting a message
from a random forwarder based on header data alone.  I would only do that
if the forwarding service, message originator, and forwarding path were
well understood from external sources.   In that case, a message from a
well-identified forwarding service with a sufficiently-identified message
originator might be a candidate for whitelisting.   But those situations
seem likely to be rare.

Conclusion
---------------
The overall conclusion is that if a message's only negative indicator is an
authentication failure, the best practice would be to send it to
administrative quarantine for local policies to be tailored to the
situation.   It seems foolish to block a message if sender authentication
failure is the only problem, unless domain abuse is actually confirmed
during the quarantine review.   It seems even more foolish to filter only
on the  identities provided by the submitting node, without evaluating the
entire header structure.

The unfortunate part of all this is that I suspect very few commercial
products evaluate the entire header.  Does anyone have data on this
question?

Where full scans are not implemented now, switching to a full scan will
significantly increase the processing overhead per message, which will
impact throughput.

Doug Foster



On Sun, Dec 20, 2020 at 12:10 PM Todd Herr <todd.herr=
[e-mail]> wrote:

> On Fri, Dec 18, 2020 at 4:55 PM Michael Thomas <[e-mail]> wrote:
>
>>
>> On 12/15/20 8:01 AM, Todd Herr wrote:
>>
>>
>> I'm not sure there's anything actionable about DMARC's policy values.
>>
>> you mean p=quarantine, or p=* in general?
>>
>
> Depending on the level of sophistication of a receiving email system, a
> given email message can have dozens of signals (or data points) associated
> with it, of which the relevant DMARC policy record, if it exists, is one.
> Those signals can all be actionable, individually or collectively, or
> ignored entirely, in that receiving system's decision to accept or reject
> the message, and if it accepts it, the decision of where to place that
> message.
>
> If an applicable DMARC policy record exists for the RFC5322.From domain in
> a message, the receiver can:
>
>    - Perform the necessary authentication checks for DMARC validation,
>    and use those validation results and the p= value as the sole or primary
>    determining factor in how to handle the message, strictly applying the p=
>    value in cases where it's not 'none' and validation checks fail
>    - Perform the necessary authentication checks for DMARC validation,
>    and use those validation results and the p= value as the sole or primary
>    determining factor in how to handle the message, loosely applying the p=
>    value in cases where it's not 'none' and validation checks fail (e.g.,
>    placing the message in the spam folder even if p=reject is specified in the
>    policy record)
>    - Perform the necessary authentication checks for DMARC validation,
>    and use those validation results and the p= value as one data point in a
>    larger data set in how to handle the message
>    - Ignore the DMARC policy record entirely
>
>
>
>> Obviously indirect mail flows, such as mailing lists and forwarding,
>> complicate matters greatly here, as the handling by the intermediary
>> host(s) can and will lead to a higher percentage of authentication
>> failures. The community has attempted to mitigate this problem;
>> RFC5322.From header rewriting, RFC5321.From header rewriting (e.g., SRS),
>> and ARC are among the attempts put forth so far, but none have been deemed
>> The Solution(tm) yet. In my opinion, ARC has promise, because if a message
>> reaches me as a receiver or even intermediary and fails the authentication
>> checks I perform, ARC header sets in the message can tell me whether or not
>> such checks passed at previous hops *if I trust the entities that inserted
>> those ARC header sets*. In an earlier thread, I floated an idea about ARC
>> sealer reputation, but it didn't draw much response, so I'll float it here
>> again in the hopes that it does.
>>
>> We've always been able to check the reputation of lists that resign the
>> message. The reputation is the previously (un)solved problem.
>>
>>
>> Lists are a specific instance of the more general case of indirect mail
> flows.
>
> Any intermediate host can ARC seal and ARC-sign (a term I use as distinct
> from re-sign) a message, an act that is meant to capture the results of
> authentication checks done when it reached the intermediary. Whether or not
> the receiver trusts those results (especially those results that claim that
> authentication checks passed at the intermediary) when the message reaches
> the receiver depends on the intermediary's reputation for being a truthful
> sealer and signer, meaning whether or not it has established a history of
> capturing authentication results that are true and accurate.
>
> Since the receiver typically can't perform the same checks under the same
> conditions that existed when the intermediary performed them (if it could,
> we wouldn't need something like ARC) then the receiver has to decide if the
> message is consistent with messages it's previously seen through direct
> mail flows using that same authenticated identity that was captured by the
> intermediary in the ARC header set.
>
> If the indirect mail flow looks like the direct mail flow, then the sealer
> is showing evidence that it can be trusted, and so the indirect mail flow
> can receive the same treatment by the receiver that it would give to the
> direct mail flow associated with the authenticated identity.
>
> If the indirect mail flow looks different than the direct mail flow, then
> the receiver can either decide to not trust the sealer and so treat the
> indirect mail flow as unauthenticated, or perhaps it can dig into matters
> further to see why the difference exists. A mailing list, for example, that
> has subscribers whose domain is a consumer mailbox provider might just show
> an indirect mail flow for that domain to the receiver that has a
> substantially better overall reputation than the direct mail flow for that
> domain to the receiver; rather than letting this difference impact the
> level of trust in its sealing reputation that a receiver would assign to
> the list, it may very well have to put in an override to its system for
> tracking sealer trust.
>
> --
>
> *Todd Herr* | Sr. Technical Program Manager
> *e:* [e-mail]
> *p:* 703.220.4153
>
>
> This email and all data transmitted with it contains confidential and/or
> proprietary information intended solely for the use of individual(s)
> authorized to receive it. If you are not an intended and authorized
> recipient you are hereby notified of any use, disclosure, copying or
> distribution of the information included in this transmission is prohibited
> and may be unlawful. Please immediately notify the sender by replying to
> this email and then delete it from your system.
> _______________________________________________
> dmarc mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/dmarc
>