Re: [dmarc-ietf] Nonexistent Domain Policy was: Re: Working Group Last Call: draft-ietf-dmarc-psd

Ian Levy <ian.levy@ncsc.gov.uk> Mon, 15 July 2019 07:08 UTC

Return-Path: <ian.levy@ncsc.gov.uk>
X-Original-To: dmarc@ietfa.amsl.com
Delivered-To: dmarc@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C1CC21200B8 for <dmarc@ietfa.amsl.com>; Mon, 15 Jul 2019 00:08:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.002
X-Spam-Level:
X-Spam-Status: No, score=-2.002 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ncsc.gov.uk
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V-bBx9r0ZRAA for <dmarc@ietfa.amsl.com>; Mon, 15 Jul 2019 00:08:06 -0700 (PDT)
Received: from GBR01-CWL-obe.outbound.protection.outlook.com (mail-eopbgr110122.outbound.protection.outlook.com [40.107.11.122]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8E77712001E for <dmarc@ietf.org>; Mon, 15 Jul 2019 00:08:06 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JCEDKXF8HQfoz03FOgDQqGr9fKrls8qyMZKJAfTlh+bM6RT2d/iE6Evd2knCmAoZuzlp9MNP136OrgqlnchA0G+/A/tm5u4c4Mfqt1Vn9tXj2mkm38pp0peLqxVtDADwzFmaVAAq5BSQPo7EDAFsb5ga7yjQCoA0paxojc3f/rxdt4KQ0pCIULpaefvNfi1jHOxstp5FrnhxLZup7WSyZMKYzjIG828ZcNB4DLSeQTMEzvSgF0EsVg6Si1xBCs3c8+cv33gHqyNhL4ETiEdPrC+fOwxxxLfoHJ/Vi0C2upisWPnRA97iXoLaSquQRBQ4IZHN3OuSP2SiuQ88eUtgkg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FNcaRaUdxOIctmu/SDpsci6lTWunneJ5rcL7Ire4Xzk=; b=EFWozfirH6m7OouyC1Y4B6Z64Z+XIZXjSOTMoDAXXlvnbxD5kF2hSfqjI0iaGnbhbHxzgl+deQO3vekui1jIoh9TTdg4aFvKTIO80QPv8g+6wsqY7pKU4EL20NALucSnPmZVrPE/Nu2zH/8g4H8RwX+VxV+MZdbaFZvWaIGfEx1VfQJMED1GmUtJJtjh378osI2WvaYmShYlIaZKFhQx92r7YwaAdcFQKWwTXJlpV+Hz0ifWl8xU0mHG8pom+jUQ/RoK717KEihJS+IIvlMkB1NUCVyeynsCCn3yot2cdQPxjnyHQazX4u7tHCjBd44ODGZjP0U18g4qtWZUnh/JOg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=ncsc.gov.uk;dmarc=pass action=none header.from=ncsc.gov.uk;dkim=pass header.d=ncsc.gov.uk;arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ncsc.gov.uk; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FNcaRaUdxOIctmu/SDpsci6lTWunneJ5rcL7Ire4Xzk=; b=Y2k0jd5YtdaSjKn+bNvDDROJGWxVUlCdjYnDN45kUmW57MmafOnL4ZWWsmD89UNpp6CDNnJZNQI20K5kbxk7aBeIXJ2peY60y7BIbaDX6P/UTAeDxbceQ0E0w4jDFwztYg89npNz0URYR84G8tU430OMTNmCLWz4F60JPWaGjaw=
Received: from LO2P123MB2285.GBRP123.PROD.OUTLOOK.COM (20.176.157.151) by LO2P123MB2030.GBRP123.PROD.OUTLOOK.COM (20.176.157.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2073.14; Mon, 15 Jul 2019 07:08:04 +0000
Received: from LO2P123MB2285.GBRP123.PROD.OUTLOOK.COM ([fe80::815e:ea4e:7133:7de0]) by LO2P123MB2285.GBRP123.PROD.OUTLOOK.COM ([fe80::815e:ea4e:7133:7de0%5]) with mapi id 15.20.2073.012; Mon, 15 Jul 2019 07:08:04 +0000
From: Ian Levy <ian.levy@ncsc.gov.uk>
To: Scott Kitterman <sklist@kitterman.com>, "dmarc@ietf.org" <dmarc@ietf.org>
Thread-Topic: [dmarc-ietf] Nonexistent Domain Policy was: Re: Working Group Last Call: draft-ietf-dmarc-psd
Thread-Index: AQHVONo/47N98C6WmEulUndQgJsxhKbHRCGAgAAF1oCAABAxAIAAFoEAgACGD4CAABkfAIACPhxA
Date: Mon, 15 Jul 2019 07:08:04 +0000
Message-ID: <LO2P123MB22851389EC8F8098BD80211CC9CF0@LO2P123MB2285.GBRP123.PROD.OUTLOOK.COM>
References: <20190713043409.A5EB64A61C0@ary.local> <3017917.gKNyNSpcLf@l5580>
In-Reply-To: <3017917.gKNyNSpcLf@l5580>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=ian.levy@ncsc.gov.uk;
x-originating-ip: [51.141.26.231]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 0a852776-87d7-4a90-f4c5-08d708f32e4d
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:LO2P123MB2030;
x-ms-traffictypediagnostic: LO2P123MB2030:
x-ms-exchange-purlcount: 1
x-microsoft-antispam-prvs: <LO2P123MB203016D2C576C69DCF9FC19DC9CF0@LO2P123MB2030.GBRP123.PROD.OUTLOOK.COM>
x-ms-oob-tlc-oobclassifiers: OLM:6790;
x-forefront-prvs: 00997889E7
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(396003)(346002)(39840400004)(366004)(136003)(376002)(13464003)(189003)(199004)(229853002)(25786009)(14454004)(68736007)(2906002)(446003)(186003)(3846002)(6116002)(11346002)(6436002)(9686003)(6306002)(55016002)(26005)(45080400002)(478600001)(33656002)(2501003)(8936002)(14444005)(44832011)(256004)(966005)(8676002)(486006)(81156014)(81166006)(76116006)(66946007)(5660300002)(66476007)(66556008)(64756008)(66446008)(305945005)(66066001)(74316002)(7736002)(476003)(71200400001)(52536014)(71190400001)(76176011)(110136005)(6246003)(316002)(53546011)(7696005)(86362001)(53936002)(55236004)(6506007)(99286004)(102836004); DIR:OUT; SFP:1102; SCL:1; SRVR:LO2P123MB2030; H:LO2P123MB2285.GBRP123.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: ncsc.gov.uk does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam-message-info: fuw1FgvioVWsXGSV1v1C5iwFDC+M/sCs1dHJVrt/BCiLitgSqoDWl+xoqH1Mi3+kZHfM/QiWnfZIEMa6NoOsarteovX9SdJERBuCE8UO3YAuyxjaHNY6EqOnoNU2nYGHa1NSd7q2ZHI2P+CTkHFBkKx+HbuNStt3KtfwHWMZJeMKJaXSTwmWukg5U11r1GR6cQz/aguw6ohVJN2qQx8+ozyapvEW8I1U57SZKnB97WR7mTDiamB8EBN6YN6fqyQocFuIPUQ4l3IqTaIBWHN6rv5DD6DxmvRsUGnCbxMmHbIGwA7zXOl9M0HyV6YRjUxtsMkjWZCQMd1uNwNp9Bnk+gnI8AMjyIlWbVIxeqrsu4O4L6BpHSHKj15jcC19WElJcSeBnzPBa1obr6NdDZol4s5OLypuZkjP6l98TTLTi7k=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: ncsc.gov.uk
X-MS-Exchange-CrossTenant-Network-Message-Id: 0a852776-87d7-4a90-f4c5-08d708f32e4d
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jul 2019 07:08:04.2480 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 14aa5744-ece1-474e-a2d7-34f46dda64a1
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ian40919@ncsc.gov.uk
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO2P123MB2030
Archived-At: <https://mailarchive.ietf.org/arch/msg/dmarc/CNHjkOHS4FB5jFBEAIDc-XzkwPQ>
Subject: Re: [dmarc-ietf] Nonexistent Domain Policy was: Re: Working Group Last Call: draft-ietf-dmarc-psd
X-BeenThere: dmarc@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Domain-based Message Authentication, Reporting, and Compliance \(DMARC\)" <dmarc.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/dmarc>, <mailto:dmarc-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/dmarc/>
List-Post: <mailto:dmarc@ietf.org>
List-Help: <mailto:dmarc-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/dmarc>, <mailto:dmarc-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Jul 2019 07:08:10 -0000

Sorry for not contributing more to this thread - please don't take it as any indication of lack of interest. For UK NCSC specifically, I think we'd prefer NXDOMAIN rather than NODATA, given it's more constrained and this is an experiment. My view would be that if we've published a name under gov.uk, even with no valid (in the eyes of the receiver) associated records, then *someone* is responsible for it and we can go find and educate them. They may even believe they have a valid reason for doing so that may outweigh any email authentication concerns. But there's a conversation to be had. If there's no published name, then there's no-one responsible, so it should default to the top-level policy. 
 
We've learned a lot running our DMARC processing for gov.uk over the last couple of years and a lot about the consistency of how non-existent domains are treated, or lack thereof more to the point. If I'm honest, we see a lot of inconsistency in how DMARC is processed more generally that makes analysis harder than it needs to be. A lot of receivers have obviously made optimizations for their own specific circumstances. I'm just a little nervous of starting an experiment on a live and (some may argue) quite important PSD in anything other than a constrained manner. 

Incidentally, we *should* be publishing on ncsc.gov.uk our 2nd annual report on our Active Cyber Defence programme tomorrow (Tuesday). This includes a chapter on our DMARC experiences, including a bit of data relevant to this discussion, as well as some novel data science work on our DMARC report archive. As a preview, from July when we set the 'synthetic DMARC' record to p=reject, we've had this many reports : 
Month (2018)	Total reports
July		5,764
August 		274,532 
September 	127,901 
October 	17,553 
November 	17,191 
December 	105,078 

We'll also publish a couple of examples of where synthesizing DMARC/SPF records for non-existent domains has helped stop abuse. However, it's clear that this method isn't universally accepted. 
Here's the volume of reports received on our normal DMARC processing chain in January 2019 (noting Microsoft are one of the bigger providers in the UK and *still* don't generate any reports): 

Reporter 	Total Reports 
google.com 	61,363,605 
Yahoo! Inc. 	18,876,201 
Mail.Ru 	699,554 
sercoglobal.com 227,587 
AMAZON-SES 	178,262

And here's the volume for the same month for the synthetic DMARC reports : 
Reporter 		Total Reports 
google.com 		23,745 
Yahoo! Inc. 		1,060 
emailsrvr.com 		64 
dev.johnlewis.co.uk 	37 
bridgend.gov.uk 	30

Just from that, it's pretty clear that the synthesized DMARC records are not universally processed, which gives weight to completing this work and starting to try things out. Given the level of inconsistency we see in receiver behaviour, I think it'd be easier to start with NXDOMAIN and see what that actually achieves. 

I may well be missing something subtle, so please correct me if I've got this wrong. 

Ta.

I.
 
--
Dr Ian Levy
Technical Director
National Cyber Security Centre
ian@ncsc.gov.uk

Staff Officer : Kate Atkins, kate.a@ncsc.gov.uk

(I work stupid hours and weird times - that doesn't mean you have to. If this arrives outside your normal working hours, don't feel compelled to respond immediately!)

-----Original Message-----
From: dmarc <dmarc-bounces@ietf.org>; On Behalf Of Scott Kitterman
Sent: 13 July 2019 07:04
To: dmarc@ietf.org
Subject: Re: [dmarc-ietf] Nonexistent Domain Policy was: Re: Working Group Last Call: draft-ietf-dmarc-psd

On Saturday, July 13, 2019 12:34:09 AM EDT John Levine wrote:
> In article <2902055.CzhLQO0xIX@l5580> you write:
> >Here's the definition we have in the draft now:
> >> 2.6.  Non-existent Domains
> >>
> >>    For DMARC [RFC7489] purposes, a non-existent domain is a domain name
> >>    that publishes none of A, AAAA, or MX records that the receiver is
> >>    willing to accept.  This is a broader definition than that in
> >>    NXDOMAIN [RFC8020].
> >
> >That's what I was expecting this new tag to apply to (and I think 
> >matches their expectation, but they can speak for themselves).
>
> That's OK.
>
> >Another way to say what's in 2.6 now might be:
> >
> >... a domain for which there is a NODATA response for A, AAAA, and MX 
> >records.
> Not so OK -- if there's no records at all at or below a name you 
> really will get NXDOMAIN.

Good point.  Thanks.  I'll leave it as is.

Scott K


_______________________________________________
dmarc mailing list
dmarc@ietf.org
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fdmarc&amp;data=02%7C01%7Cian.levy%40ncsc.gov.uk%7C347777d406d8456ee39d08d70758084b%7C14aa5744ece1474ea2d734f46dda64a1%7C0%7C0%7C636985946991033447&amp;sdata=S1HHVXL4ftxSYbFhTgxx1pXVcOT2o0S1PM%2B7sUCL9eo%3D&amp;reserved=0
This information is exempt under the Freedom of Information Act 2000 (FOIA) and may be exempt under other UK information legislation. Refer any FOIA queries to ncscinfoleg@ncsc.gov.uk