[Asrg] SPF and other spam approaches
gep2@terabites.com Tue, 17 June 2003 21:20 UTC
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged)) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA12135 for <asrg-archive@odin.ietf.org>; Tue, 17 Jun 2003 17:20:47 -0400 (EDT)
Received: (from exim@localhost) by www1.ietf.org (8.11.6/8.11.6) id h5HLKJj11840 for asrg-archive@odin.ietf.org; Tue, 17 Jun 2003 17:20:19 -0400
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 19SNsJ-00034t-Kx for asrg-web-archive@optimus.ietf.org; Tue, 17 Jun 2003 17:20:19 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA12125; Tue, 17 Jun 2003 17:20:16 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 19SNq4-0005Ho-00; Tue, 17 Jun 2003 17:18:00 -0400
Received: from ietf.org ([132.151.1.19] helo=optimus.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 19SNq3-0005Hl-00; Tue, 17 Jun 2003 17:17:59 -0400
Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 19SNs1-00031O-PW; Tue, 17 Jun 2003 17:20:01 -0400
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 19SNra-00030B-AZ for asrg@optimus.ietf.org; Tue, 17 Jun 2003 17:19:34 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA12120 for <asrg@ietf.org>; Tue, 17 Jun 2003 17:19:31 -0400 (EDT)
From: gep2@terabites.com
Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 19SNpK-0005Hg-00 for asrg@ietf.org; Tue, 17 Jun 2003 17:17:14 -0400
Received: from h001.c000.snv.cp.net ([209.228.32.65] helo=c000.snv.cp.net) by ietf-mx with smtp (Exim 4.12) id 19SNpK-0005Hd-00 for asrg@ietf.org; Tue, 17 Jun 2003 17:17:14 -0400
Received: (cpmta 12161 invoked from network); 17 Jun 2003 14:19:29 -0700
Received: from 12.239.18.238 (HELO WinProxy.anywhere) by smtp.terabites.com (209.228.32.65) with SMTP; 17 Jun 2003 14:19:29 -0700
X-Sent: 17 Jun 2003 21:19:29 GMT
Received: from 192.168.0.30 by 192.168.0.1 (WinProxy); Tue, 17 Jun 2003 16:18:22 -0600
Received: from 192.168.0.240 (unverified [192.168.0.240]) by nts1.terabites.com (EMWAC SMTPRS 0.83) with SMTP id <B0000024052@nts1.terabites.com>; Tue, 17 Jun 2003 16:45:18 -0500
Message-ID: <B0000024052@nts1.terabites.com>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
To: asrg@ietf.org
X-Mailer: SPRY Mail Version: 04.00.06.17
Content-Transfer-Encoding: 7bit
Subject: [Asrg] SPF and other spam approaches
Sender: asrg-admin@ietf.org
Errors-To: asrg-admin@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>, <mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>, <mailto:asrg-request@ietf.org?subject=subscribe>
List-Archive: <https://www1.ietf.org/pipermail/asrg/>
Date: Tue, 17 Jun 2003 16:45:18 -0500
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
I'm going to combine responses to several related posts from the last digest. > ...SPF avoids the quicksand of definition entirely. SPF is merely an *extension* of SMTP which *closes a security hole*; Well, I think that "closes" is rather misleading. It shrinks it, but doesn't really close it in any positive, meaningful way. If it really were the single, unobstructive 'silver bullet' that everyone seems to want, its numerous problems might be more tolerable. > it improves the veracity of sender addresses among consenting hosts. "Improves", yes, but not by ENOUGH that globally one can act in a positive way on the conclusion. Yes, it can indicate that a given message, IF it comes from the indicated sender, was sent from an 'atypical' originating domain for that sender. The problem is that that could happen QUITE LEGITIMATELY. Also, the fact that a forging spammer could with TOTAL impunity still impersonate the victimized From: address, as long as they're able to send from within the same range of IP addresses (which is fairly easy if it's a big ISP domain, like "swbell.net", which might have literally millions of subscribers and IP addresses). Similarly, spam from legitimate (but throwaway) E-mail accounts is likewise unaffected... yeah, the sender is exactly who they say they are, "bogus@freeisp.com". And they really do reside at 123 Nosuch Street, I'm sure... > It also fights worms and viruses; No, it really doesn't. As long as they are sent from "legitimate" senders and through their configured SMTP servers, they'll go out just fine. > ...it also fights forgery. It DOES detect (but not eliminate) many (but not all) kinds of forgeries. Unfortunately, that comes at the cost of hugely inconveniencing (or making it impossible) for many legitimate, honest E-mail users to continue using their Internet services the way they always have been able to do. Because of that simple fact, it's not clear that 'detecting' a possible/likely From: address forgery (according to spf) allows an ISP to really do much about that. > Right now lots of our customers misconfigure their home machines as pobox.com, and send bounce messages using From: postmaster@pobox.com. The double-bounces end up in my mailbox. SPF helps solve this problem also. If SPF works against spam, what a happy coincidence! >| ...some objections to SPF have come from people who | | 3) conclude that because technical solutions have not solved spam in | the past, they cannot possibly solve spam in the future; this is | the "spam is ultimately a social / economic / legal problem" camp; | | 4) observe that no single solution is perfect, and fall into despair; > (3) is an objection unique to pessimists. We technical types, being professional practitioners of pessimism, must be especially wary. We have nothing to lose by trying a new approach. If that were true, then testing the approach would make more sense. Unfortunately, there are a number of problems I foresee that SPF does not address or resolve. Meanwhile, it ignores the legitimate if inconvenient needs and requirements of a fair number of honest/legitimate users. The (potentially large) costs to those ignored users don't seem to concern the spf folks very much. > Optimism in this case is more productive than pessimism. Optimism is fine when there are no visible, obvious problems with a given approach. When there ARE problems that are plainly obvious in advance, then forging ahead regardless smacks less of "optimism" and more of "denial". > (4) is a problem for those who are both pessimists and perfectionists. An 80% solution that works is better than a 100% solution that doesn't exist. I certainly agree. The "permissions" system I propose doesn't purport to eliminate spam; what it DOES do is to block a LARGE percentage of it, to make the stuff that DOES get through smaller and less costly, and easier for content filters like SpamAssassin to deal with. What's more, my approach is incrementally and independently implementable at the individual ISP level (or even, with less benefit, at the individual end user level!) and doesn't require reaching any kind of global technology consensus or re-engineering of basic Internet systems architecture in order to achieve HUGE and immediate, direct benefits. > Spam costs ISPs money; they are looking for answers today. Absolutely. Another reason why spf, IMHO, is NOT the answer. > Forwarding systems and kiosks can easily achieve SPF compliance by using a sender rewriting scheme (SRS); I am developing a simple I-D to describe one possible SRS. Yes, but the mere fact that you're expecting EVERY sender WORLDWIDE to change their systems to comply with your suggested standard, to me, is pretty outrageous. It's just NOT likely to happen. > ...The pain of a world where SMTP permits forgery, anonymity, and spam is greater to me than the pain of a world where SASL+SMTP+SPF require authenticated logins to home servers. Yes, but the notable part of your contention is "to ME" [emphasis mine]. Unfortunately, you don't seem to take the legitimate needs of users into much account where other people's answer to that question might be different. > I expect leading ISPs to act for the majority of their users, Depends entirely on what the agreements are that they have with their users. For them to make a voluntary change which blows some (even small) percentage of those (legitimate, honest, reasonable) users out of the water could represent "breach of contract" and subject the ISP to legal liability. > ...and I expect the majority of the users to agree with me. Individual rights, to be meaningful, doesn't just mean the right to take popular, majority positions. The right to free speech doesn't mean much if it only means you have the right to say stuff that nobody else finds objectionable. Now, I am *not* saying that spammers have the unassailable right to shovel their fraudulent, misrepresented, and sometimes lurid garbage indiscriminately into the E-mail inboxes of children, or even other adults. But one CAN'T just ignore the needs and legitimate rights of minorities. > SPF tries to cushion the blow to those with fragile eggs by providing the mechanism of Sender Rewriting and by advocating SASL. For spf to be "mostly useful", it must be adhered to by a large percentage of ISPs worldwide. Frankly, I don't honestly expect to see that happen. > Damian Conway: "if nothing you've tried so far has worked, try something new." SPF counts as new. So do any of a number of other ideas, some of which seem to have a much lower cost and a much higher probability of immediate payback... and which don't have a visible brick wall a little further down the path. > I have seen several people criticize SPF as unworkable. I don't accuse SPF as being "unworkable". I do however feel that it is logistically improbable, doesn't really solve the problems it's purported to solve, and may create more problems (and problems even more intractable, perhaps) than those it DOES solve. > I invite them to propose a specific alternative. I have done exactly that. My problem attacks the spam/virus/worm problems from a different perspective, but is more readily implementable and comes at essentially ZERO cost or inconvenience for the GREAT majority of legitimate E-mail users... (those who send plain ASCII text E-mails, without HTML-burdening, attachments, or encoded text). It allows receipt of unexpected E-mails from previously unknown correspondents, but forces those to be in a more compact and efficient form that is readily processed by better-conceived content filters. Note that my proposed solution would block the majority of today's spam, shrink the average size of tomorrow's spam, improve the effectiveness and efficiency of content filters, and would almost totally eliminate virus and worm and trojan threats for the great majority of users. And it would do that with almost no impact on most E-mail users... and without requiring ANY global technological consensus. > If they will not, then these people won't lead, and they won't follow; they need to get out of the way. Hopefully you won't try to put me into that class. :-) > I've already thought of one response; the "throwaway domain" scenario is addressed in the FAQ at > http://spf.pobox.com/faq.html#noprevent Addresses those points, yes.... unfortunately though there are a LOT of holes and glib presumptions in those "answers" that just don't hold much water in the end analysis. >| ...some objections to SPF have come from people who | | 9) prefer a different approach; | > This is a perfectly respectable objection. In this war on spam we can use as many weapons as we can get. SPF actually does relatively little to prevent or dissuade spam. Unfortunately, it is FAR more effective at preventing (even legitimately) anonymized E-mail... such as E-mails dealing with politically, professionally, or personally sensitive subjects (e.g. "whistle blowing" or commentary from within a repressive regime). > We don't have to pick just one solution; they can work together. On that point we very much agree. I see my sender/recipient permissions list as working hand-in-hand with content filters such as SpamAsssassin to produce a HUGE reduction in spam/virus/worm/trojan message volume, bulk, bandwidth, and fraud/mischief potential. > It's time to review the proposals and unite behind the best ones. That suggests an either/or approach, and the desire at some point to stop listening to new approaches and ideas. I don't think that's a good plan. > If you evaluate the proposals along the dimensions of scalability, backward-compatibility, transparency, level of inconvenience to users, universal applicability, implementation timeframe, and cost of new code and configuration, SPF comes out near the head of the pack. I disagree. I find that SPF requires a global concurrence that we're unlikely to achieve. SPF expects that eventually nearly all MTAs will become spf-compliant; those that do not comply end up being a sore, sticking point essentially forever. SPF blows some users depending on personal domains straight out of the water. SPF creates major problems for public-access E-mail terminals (kiosks, internet cafes, cruise ships, airport waiting lounges, etc). While SPF offers some evidence of possible/likely forged domain, basically NONE of the results it gives are 100% sure. The existence of ONE noncompliant MTA (or open mail relay) within a large domain (say swbell.net) means that anyone using that domain for outgoing E-mail can impersonate any other sender with an address within that domain, and SPF will indicate that (as far as it's concerned) the From: address is perhaps[/likely] not forged. Even if ISPs concur on SPF essentials, it would take YEARS to achieve widespread compliance (and even then, the spam problem still isn't really solved). > One problem, as I see it, are the ever increasing number of zomboid thrall servers; PC's with viruses which turn them into spam delivery slaves. To begin with, there are all manner of potential "denial of service attacks" and not all of them involve E-mail or spam. That said, blocking E-mails containing scripting, ActiveX, HTML, and attachments from anyone not AUTHORIZED SPECIFICALLY to send such stuff to a given recipient will go a LONG way towards reducing the ability so sneak such stuff into an unwary user's machine. > "Spammers" is not a single entity. It's possible that even single spammers will try many different approaches, but certainly as group, they will try almost everything they can think of. Thus, they won't switch to empty envelopes, they'll do empty envelopes /as well/. Or rather, some will do empty envelopes, while some will continue to forge envelopes, and some will buy dozens of new domains to spam "from". If those new domains are unable to get HTML-burdened E-mail, or attachments, or image/hypertext links, or scripting, or encoded text, through to users... then their ability to deceive and defraud while cloaking their actions are SEVERELY damaged. Cleaning up the residual using content filters ought to mop up a sizeable percentage of whatever is left. > I've seen seemingly endless talk recently about the characteristics of people who don't like SPF and various bits of what I assume are deep philosophies, but nothing about what SPF is. Essentially, SPF is a way using DNS to allow a MTA to query for a given E-mail (sender) address to find out which IP addresses that E-mail address is "authorized" to send from. The idea is to detect obvious (the most egregious, anyhow) cases of From: address forgeries. What one can actually DO with that information, once determined, is a lot less obvious than might originally appear. > ...SPF may be based on the false notion that many (or any) major ISPs object or could be forced to object to people sending mail ossensibly from their domain names but via unrelated SMTP clients. And that, indeed, is one of the big problems (IMHO) with SPF. >> Spam with an empty envelope isn't ever going to be effective, > since most will filter it out. My "permissions" approach would (by default) t-can any incoming E-mail with a blank sender, IF it contained HTML, scripting, attachments, or encoding... although the individual recipient could easily set a "blank sender" to allow those IF they wanted to allow it. > How about all the "robots" or CR systems that send their mails with empty envelope senders, because they don't want an answer if the challenge was faked. How about all the autoresponders? There are lots of valid emails with empty envelopes. Sure, but vanishingly small numbers of those advisories need to send HTML or attachments. > Sorry, I don't think it's reliable if bounces get lost. Just like discarding spam qualified emails automatically is .. hmmm .. unwise, because if it's a false classification the sender has no chance to get informed what happend. At times where email is considered a trustworthy medium this is unacceptable. Note that the rules for discarding HTML, attachments, and encoding as proposed in my permission-list system are at least intended to be unequivocal and don't require statistical interpretation. Unknown mail senders sending attachments or HTML or encoded text bodies are refused/bounced/trashed, period. I agree fully that content filters like SpamAssassin can occasionally generate false positives, so those SHOULD use a different, and more auditable, mechanism (possibly redirecting such suspect messages to screenname-spam@userdomain.com or some such.) > So you want to force about 90% of all mailserver administrators (most of the clueless) to change the configuration of their mailserver? Have fun. Exactly... A lot of these folks are not going to make these changes, or will make them wrong. I like approaches much better that don't require global change, or global consensus. Gordon Peterson http://personal.terabites.com/ 1977-2002 Twenty-fifth anniversary year of Local Area Networking! Support the Anti-SPAM Amendment! Join at http://www.cauce.org/ 12/19/98: Partisan Republicans scornfully ignore the voters they "represent". 12/09/00: the date the Republican Party took down democracy in America. _______________________________________________ Asrg mailing list Asrg@ietf.org https://www1.ietf.org/mailman/listinfo/asrg