IETF Logo Mail Archive

Re: [dmarc-ietf] Doing a tree walk rather than PSL lookup

John Levine <johnl@taugh.com> Tue, 24 November 2020 17:03 UTC

Return-Path: <johnl@iecc.com>
X-Original-To: dmarc@ietfa.amsl.com
Delivered-To: dmarc@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C88E03A1216 for <dmarc@ietfa.amsl.com>; Tue, 24 Nov 2020 09:03:57 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.049
X-Spam-Level:
X-Spam-Status: No, score=0.049 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=iecc.com header.b=F/5vcwQy; dkim=pass (2048-bit key) header.d=taugh.com header.b=UE4tdDcB
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fIGQX7wiSfaQ for <dmarc@ietfa.amsl.com>; Tue, 24 Nov 2020 09:03:55 -0800 (PST)
Received: from gal.iecc.com (gal.iecc.com [IPv6:2001:470:1f07:1126:0:43:6f73:7461]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 18CC33A120E for <dmarc@ietf.org>; Tue, 24 Nov 2020 09:03:54 -0800 (PST)
Received: (qmail 62593 invoked from network); 24 Nov 2020 17:03:52 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:cc:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:cleverness; s=f47a.5fbd3cf8.k2011; bh=Rwb9F7Ch3DF9LiCvHSpZ3mT1Yu9Sp+USyxrBrzIT5to=; b=F/5vcwQynWWHZSPbAwjNxvp+FldHPf84FVl/ae1/kp7R3V193xAj1LhUfX/E3R4VZCXtk56MJYmcq9F91AnZ7M8hHfxmrqg1bXBxMD3yZViFynI/PqlC72x60MIWabHLGW7iFMLBozkwLrXea/6bt3iI9K4gVFY0AUduR2r2RrHKr7+UvXpamvnE4fbSkylkqt5OfJzLa9Dhum4wgTgDrHhyUjyLx1N1hQC6pZoc2+5B2NycWvs/Q2xI9ivBA2eDGOlENUfMK0v6XurAVdW7yiA4k2h/OMknHay2b3U9SAdgXy8/WeTxjEpC6PCFPG8pAnt6VS195e/5hA/MY5TYCA==
DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:cc:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:cleverness; s=f47a.5fbd3cf8.k2011; bh=Rwb9F7Ch3DF9LiCvHSpZ3mT1Yu9Sp+USyxrBrzIT5to=; b=UE4tdDcBzjqfOZ/gC4dZnzdmg3g14pYDKnVPmKrmBZvI3lZqma/F0Qh5agXPH4sS23Dx8QBBSr5P/NY3SrJ6JV3E4d4di2wh9F1p01oiiWzu/z61atCd7Uwe6+v/0QTR4J98u51ZFXiUin3mMDYv5PZtDChxbn3E3vnbGCQ++ObMGX9832pHRnD5VEcF5SVQPbR3swkEMSGKv+DAWwRPA6g17qIh1D1ohqtKMzdsXa8GoaSaxAuZu2656MsymR2bkx2s6eACLL9FY0/YT5kv6iSlGJHasajhykhKknKQmcc2bb4XGB0GoLtSNP8tWQZ5l3Cp/M5IChnG4I4amBV+sA==
Received: from ary.qy ([IPv6:2001:470:1f07:1126::78:696d:6170]) by imap.iecc.com ([IPv6:2001:470:1f07:1126::78:696d:6170]) with ESMTPS (TLS1.2 ECDHE-RSA AES-256-GCM AEAD) via TCP6; 24 Nov 2020 17:03:52 -0000
Received: by ary.qy (Postfix, from userid 501) id C430227DFEBF; Tue, 24 Nov 2020 12:03:51 -0500 (EST)
Date: Tue, 24 Nov 2020 12:03:51 -0500
Message-Id: <20201124170351.C430227DFEBF@ary.qy>
From: John Levine <johnl@taugh.com>
To: dmarc@ietf.org
Cc: vesely@tana.it
In-Reply-To: <efa0117e-5b17-800d-820d-b5d2413c6075@tana.it>
Organization: Taughannock Networks
X-Headerized: yes
Cleverness: minimal
Mime-Version: 1.0
Content-type: text/plain; charset="utf-8"
Content-transfer-encoding: 8bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/dmarc/1cEOzFdruQyeMSuC4HjGIw55P58>
Subject: Re: [dmarc-ietf] Doing a tree walk rather than PSL lookup
X-BeenThere: dmarc@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Domain-based Message Authentication, Reporting, and Compliance \(DMARC\)" <dmarc.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/dmarc>, <mailto:dmarc-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/dmarc/>
List-Post: <mailto:dmarc@ietf.org>
List-Help: <mailto:dmarc-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/dmarc>, <mailto:dmarc-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Nov 2020 17:03:58 -0000

In article <efa0117e-5b17-800d-820d-b5d2413c6075@tana.it> you write:
>> One of the points of the tree walk is to get rid of the PSL processing.
>
>The PSL processing is a local lookup on an in-memory suffix tree.  How is it a 
>progress to replace it with a tree walk?  A PSL search is lightning faster than 
>even a single DNS lookup, isn't it?

You have to download a copy of the PSL, read it into your program, and
parse it into some internal form. The PSL is over 200K of text and
13,000 lines, so while it's not a large file, it's not zero either.

If you're lucky you can amortize your PSL parsing across multiple
DMARC checks, but your DNS cache amortizes DNS lookups across mutiple
checke, too.

The DNS approach has the advantage that you don't have to depend on a
third party's text file updated at unknown intervals, and also makes
it easier to deal with what I've called the Holy Roman Empire problem.

R's,
John

v2.23.0 | Report a Bug | By Email