Re: [dmarc-ietf] DMARC2 & SPF Dependency Removal

Tero Kivinen <kivinen@iki.fi> Thu, 15 June 2023 21:25 UTC

Return-Path: <kivinen@iki.fi>
X-Original-To: dmarc@ietfa.amsl.com
Delivered-To: dmarc@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3CAE8C151525 for <dmarc@ietfa.amsl.com>; Thu, 15 Jun 2023 14:25:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.1
X-Spam-Level:
X-Spam-Status: No, score=-7.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=iki.fi
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eNUnj6qoAQQS for <dmarc@ietfa.amsl.com>; Thu, 15 Jun 2023 14:25:50 -0700 (PDT)
Received: from lahtoruutu.iki.fi (lahtoruutu.iki.fi [IPv6:2a0b:5c81:1c1::37]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C42AFC151532 for <dmarc@ietf.org>; Thu, 15 Jun 2023 14:25:49 -0700 (PDT)
Received: from fireball.acr.fi (fireball.kivinen.iki.fi [IPv6:2001:1bc8:100d::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kivinen@iki.fi) by lahtoruutu.iki.fi (Postfix) with ESMTPSA id 4QhwLx37V4z49PsD; Fri, 16 Jun 2023 00:25:45 +0300 (EEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iki.fi; s=lahtoruutu; t=1686864345; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aFn0ZoffkahIWjQuurIc45TG4x1kPFdn93918xyuRx8=; b=bPdYKnkZLcOMvhsxCvCdjkJfBcDDUXd8eI29c6ANUg0GwFU1HxVMFRR83th0OBFUoctvgG 9Fve513HcEo2BkHHBk6EEoxHQlhYnMz18yuLMEfn9/Takg649rBM2P2ZlG1kX3bB6kQvTx MBoC1w7Oac/spyWzNR0LVS5Nw4ebQvVyo8+98xnTmqTeNNW6ikfctNM7Z/QmF3x2Lz2MIR XDeYtAzU/ycKs+gC3g5TKsnJlVjpucxt4wXiAhc0MoVAFozIw/zyQ4XqUuGtTicwhAb/jE j8RCibalql14hFqOyxRGXtrInxsaFdXKrKLo4ghkzwuPUtC15/ybZ5/SPFOTjw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=iki.fi; s=lahtoruutu; t=1686864345; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aFn0ZoffkahIWjQuurIc45TG4x1kPFdn93918xyuRx8=; b=foXUE73/PN68DSPA0UwWAEHJ/KnuvzWIn1S1AARHXC19FxqKenZ6HJC7bT8wFvuScx6/s6 od1Jk+7KJaiEe2vUqo4Z2XsS6D1dWqeZA81jClZizDqE5+gzbamNYFe2I1ZJvlqS2iGcsO J9c6K9VC2721J2cGZBRV9boAnGhRTYr9PAbc9vIC0jYyYJcPw2ZkDkKJFmb22e+k8O3I08 nI+uouiBQXPpDa9EWLGV1G+ctVVXjjnk1rsUsm8D51jUaqSYikGdm+E9absQcUrdSXxNwa vuRhRWTJNILWnEDHMcpSEt1Ki0dK2MvMyVjDhgy2/kvXctsDJ67G9FrqyAaCVw==
ARC-Authentication-Results: i=1; ORIGINATING; auth=pass smtp.auth=kivinen@iki.fi smtp.mailfrom=kivinen@iki.fi
ARC-Seal: i=1; s=lahtoruutu; d=iki.fi; t=1686864345; a=rsa-sha256; cv=none; b=r9rKHrlPq6d4U0bk/BmRm1Ghh25Or77T9ojW4Am1AVuoOIxATA/b2/hf9CMBep8W1UpUIC tC9oOauyzBvrg9eHhaE6vhZc8LQ3dBdehkcfHPTdbDCEEcL5W9Cr5yy6h/uxL/48d5Zw61 S1dlunfUH2gIe1ZEoKYhzQBG+nJOM2feoyRu7w9rYLSM89vUHP1shioj2k+Igt4+/5pvmu Nb9W8Tr/HE/d/xY8H8aKJap6GdmIqVEYuaqTvchNkpU1pjGM0d+zS+UhgGOWZXVlnaqdKb iYzGAMajHbpeRbHYDsRCngzUEsqT24VjGsIKkent2DKH4lek8nH+14LxM/bQ6g==
Received: by fireball.acr.fi (Postfix, from userid 15204) id 30B5425C1240; Fri, 16 Jun 2023 00:25:44 +0300 (EEST)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Message-ID: <25739.33240.127804.524371@fireball.acr.fi>
Date: Fri, 16 Jun 2023 00:25:44 +0300
From: Tero Kivinen <kivinen@iki.fi>
To: Alessandro Vesely <vesely@tana.it>, dmarc@ietf.org
In-Reply-To: <25739.5435.550786.601699@fireball.acr.fi>
References: <30BB83B2-B454-41B8-992B-8E2569802D9C@1und1.de> <D225D7FC-C570-4B63-A694-9F16DB1F33E1@kitterman.com> <CALaySJKwuOK-81dW2H9dtURxa5mLQDUNo+MWcs+Hho8N+yP9qg@mail.gmail.com> <2817813.dRqVH37e0G@localhost> <CALaySJJbPFBAV_7mZaARYWuMzuX+74r2Cm0jD+z92_iuFRn_MQ@mail.gmail.com> <25736.57534.195344.782189@fireball.acr.fi> <1ec42959-977a-9ce0-907a-83a5eb2b6ef2@tana.it> <25739.5435.550786.601699@fireball.acr.fi>
X-Mailer: VM 8.2.0b under 26.3 (x86_64--netbsd)
X-Edit-Time: 12 min
X-Total-Time: 12 min
Archived-At: <https://mailarchive.ietf.org/arch/msg/dmarc/to6K8nQA2VE9obPv4d47arXElHk>
Subject: Re: [dmarc-ietf] DMARC2 & SPF Dependency Removal
X-BeenThere: dmarc@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "Domain-based Message Authentication, Reporting, and Compliance \(DMARC\)" <dmarc.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/dmarc>, <mailto:dmarc-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/dmarc/>
List-Post: <mailto:dmarc@ietf.org>
List-Help: <mailto:dmarc-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/dmarc>, <mailto:dmarc-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 15 Jun 2023 21:25:52 -0000

Tero Kivinen writes:
> > What are those 0.75%, some 30k SPF - DKIM messages? Are there
> > cases of DKIM random failure salvaged by SPF?
> 
> My current analysis script does not try to calculate that, I would
> need to need to add that step there and rerun the script. If I
> understand correctly you would like to see cases where if there is
> both SPF and DKIM, the cases where the both, only one, or neither
> passed, and how many of those cases would be where dkim=fail, but
> spf=pass?

I rerun the statistics and yes, there is 0.84% cases where dkim
failed, but spf returned eithe pass, softfail or neutral.

There was also 1.12% cases where spf returned permerror but dkim
returned pass, and 1.26% cases where dkim returned pass and spf
returned fail, or softfail.

There is of course much bigger part of emails where there was no dkim,
but there was spf that passed (7.03%) or softfailed (1.08%).

Here are the actual numbers for DKIM and SPF result combinations:

	DKIM & SPF combinations
	===========================================
	78.62%  3133749 dkim=pass,spf=pass
	7.03%   280239  dkim=none,spf=pass
	4.68%   186634  dkim=pass,spf=none
	3.85%   153543  dkim=none,spf=none
	1.12%   44543   dkim=pass,spf=permerror
	1.08%   43212   dkim=none,spf=softfail
	0.82%   32821   dkim=fail,spf=pass
	0.78%   30953   dkim=pass,spf=softfail
	0.61%   24221   dkim=none,spf=fail
	0.48%   19329   dkim=pass,spf=fail
	0.43%   17120   dkim=none,spf=neutral
	0.24%   9612    dkim=fail,spf=none
	0.06%   2320    dkim=none,spf=permerror
	0.06%   2214    dkim=pass,spf=neutral
	0.04%   1712    dkim=none,spf=temperror
	0.02%   995     dkim=fail,spf=fail
	0.02%   924     dkim=fail,spf=softfail
	0.02%   669     dkim=temperror,spf=pass
	0.01%   360     dkim=missing,spf=missing
	0.00%   199     dkim=temperror,spf=temperror
	0.00%   196     dkim=fail,spf=neutral
	0.00%   144     dkim=missing,spf=none
	0.00%   119     dkim=pass,spf=temperror
	0.00%   99      dkim=missing,spf=pass
	0.00%   50      dkim=fail,spf=permerror
	0.00%   38      dkim=missing,spf=softfail
	0.00%   14      dkim=temperror,spf=none
	0.00%   10      dkim=temperror,spf=softfail
	0.00%   7       dkim=missing,spf=fail
	0.00%   6       dkim=fail,spf=temperror
	0.00%   6       dkim=missing,spf=neutral
	0.00%   1       dkim=temperror,spf=fail
	0.00%   1       dkim=missing,spf=temperror

I.e. 78.64% of time both DKIM and SPF passed.

I also calculated statistics for all DKIM, SPF, DMARC, and ARC
combinations, but there were so many of them that I do not include the
full list here but here is top 30 from that list:

	Protocol combinations
	============================================================
	37.74%  1504477 dkim=pass,spf=pass,dmarc=missing,arc=missing
	25.37%  1011277 dkim=pass,spf=pass,dmarc=pass,arc=missing
	10.96%  436838  dkim=pass,spf=pass,dmarc=none,arc=missing
	3.46%   138083  dkim=none,spf=pass,dmarc=missing,arc=missing
	2.15%   85799   dkim=pass,spf=none,dmarc=missing,arc=missing
	2.00%   79739   dkim=pass,spf=none,dmarc=pass,arc=missing
	1.96%   78279   dkim=none,spf=none,dmarc=missing,arc=missing
	1.64%   65205   dkim=none,spf=pass,dmarc=none,arc=missing
	1.60%   63758   dkim=pass,spf=pass,dmarc=missing,arc=pass
	1.54%   61579   dkim=none,spf=pass,dmarc=pass,arc=missing
	1.16%   46309   dkim=pass,spf=pass,dmarc=pass,arc=pass
	1.09%   43529   dkim=none,spf=none,dmarc=fail,arc=missing
	0.92%   36478   dkim=pass,spf=pass,dmarc=fail,arc=missing
	0.79%   31298   dkim=none,spf=none,dmarc=none,arc=missing
	0.56%   22504   dkim=none,spf=softfail,dmarc=missing,arc=missing
	0.56%   22123   dkim=pass,spf=permerror,dmarc=missing,arc=missing
	0.55%   21973   dkim=pass,spf=pass,dmarc=none,arc=pass
	0.40%   15760   dkim=fail,spf=pass,dmarc=missing,arc=missing
	0.37%   14855   dkim=none,spf=softfail,dmarc=fail,arc=missing
	0.37%   14716   dkim=pass,spf=softfail,dmarc=missing,arc=missing
	0.34%   13576   dkim=none,spf=fail,dmarc=missing,arc=missing
	0.32%   12745   dkim=pass,spf=permerror,dmarc=none,arc=missing
	0.31%   12348   dkim=pass,spf=softfail,dmarc=pass,arc=missing
	0.26%   10290   dkim=none,spf=neutral,dmarc=missing,arc=missing
	0.24%   9657    dkim=pass,spf=permerror,dmarc=pass,arc=missing
	0.23%   9367    dkim=pass,spf=fail,dmarc=missing,arc=missing
	0.20%   8121    dkim=pass,spf=fail,dmarc=pass,arc=missing
	0.20%   7785    dkim=fail,spf=pass,dmarc=none,arc=missing
	0.17%   6719    dkim=pass,spf=none,dmarc=missing,arc=pass
	0.16%   6248    dkim=none,spf=pass,dmarc=fail,arc=missing

So 37% emails had dkim and spf pass, but no dmarc. 25.37% had also
dmarc.
-- 
kivinen@iki.fi