Re: [Asrg] An Anti-Spam Heuristic

Chris Lewis <clewis+ietf@mustelids.ca> Sun, 16 December 2012 18:16 UTC

Return-Path: <clewis+ietf@mustelids.ca>
X-Original-To: asrg@ietfa.amsl.com
Delivered-To: asrg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AC1DA21F881C for <asrg@ietfa.amsl.com>; Sun, 16 Dec 2012 10:16:05 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.598
X-Spam-Level:
X-Spam-Status: No, score=-0.598 tagged_above=-999 required=5 tests=[AWL=0.450, BAYES_00=-2.599, FH_RELAY_NODNS=1.451, RDNS_NONE=0.1]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e7OUBSAgPUun for <asrg@ietfa.amsl.com>; Sun, 16 Dec 2012 10:16:05 -0800 (PST)
Received: from mail.mustelids.ca (unknown [174.35.130.2]) by ietfa.amsl.com (Postfix) with ESMTP id F07B721F87EC for <asrg@irtf.org>; Sun, 16 Dec 2012 10:16:04 -0800 (PST)
Received: from [192.168.0.8] (otter.mustelids.ca [192.168.0.8]) (authenticated bits=0) by mail.mustelids.ca (8.14.4/8.14.4/Debian-2ubuntu2) with ESMTP id qBGIG0SB029909 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NOT) for <asrg@irtf.org>; Sun, 16 Dec 2012 13:16:01 -0500
Message-ID: <50CE0FE0.90700@mustelids.ca>
Date: Sun, 16 Dec 2012 13:16:00 -0500
From: Chris Lewis <clewis+ietf@mustelids.ca>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666
MIME-Version: 1.0
To: asrg@irtf.org
References: <SNT002-W143FB9A867C92FA80D90E04C54E0@phx.gbl> <DA14FA4D-13CB-4C61-90C4-4E690F0EC745@blighty.com> <SNT002-W1393526B62C0940EF697B2C54E0@phx.gbl> <20682.3413.665708.640636@world.std.com> <50CA0E91.2080304@mtcc.com> <20682.23612.451287.246798@world.std.com> <E26A6D4F-FC05-45B9-80F0-9E6F8A6A9713@blighty.com> <20682.31889.485606.165715@world.std.com> <50CAAD79.8040008@mustelids.ca> <50CDB59D.1040403@tana.it>
In-Reply-To: <50CDB59D.1040403@tana.it>
X-Enigmail-Version: 1.4.6
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit
Subject: Re: [Asrg] An Anti-Spam Heuristic
X-BeenThere: asrg@irtf.org
X-Mailman-Version: 2.1.12
Precedence: list
Reply-To: Anti-Spam Research Group - IRTF <asrg@irtf.org>
List-Id: Anti-Spam Research Group - IRTF <asrg.irtf.org>
List-Unsubscribe: <http://www.irtf.org/mailman/options/asrg>, <mailto:asrg-request@irtf.org?subject=unsubscribe>
List-Archive: <http://www.irtf.org/mail-archive/web/asrg>
List-Post: <mailto:asrg@irtf.org>
List-Help: <mailto:asrg-request@irtf.org?subject=help>
List-Subscribe: <http://www.irtf.org/mailman/listinfo/asrg>, <mailto:asrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Sun, 16 Dec 2012 18:16:05 -0000

On 12-12-16 06:50 AM, Alessandro Vesely wrote:
> On Fri 14/Dec/2012 05:39:21 +0100 Chris Lewis wrote:
>> Ooh, quantitative ;-)
>>
>> For grins, I took one of my smaller spamtraps and applied a 30 second
>> banner delay.  I wanted to quantify
>>
>> "And a lot of spamware doesn't flunk."
>>
>> In the timestamps below, the change happened at 04:52.
>>
>> Flow per minute:
>> [snip]
>>     156 2012/12/14-04:51
>>      30 2012/12/14-04:52
>>
>> A 3:1 spam reduction is nothing to sneeze at.

Small update: Kelihos and Lethic are stopped dead with a 30 second
delay.  Unlike previous reports, Festi actually appears unaffected.

These tests were very short duration and given the small size of the
trap, it's sometimes difficult to tell the difference between natural
flow variation and clear effect.  With Lethic and Kelihos it's
absolutely unmistakeable.  Festi looked at first, but it wasn't doing
much anyway and later showed it wasn't dead.

> You need at least 15 daemons accepting 2 msgs/minute each to get 30
> messages, while at, say, 60 msgs/minute 3 daemons can take 180.

The trap is 20 simultaneous daemons, _each_ multi-threaded (event driven).

When pushed, the traps can have several thousand SMTP sessions
simultaneously in progress.

We weren't anywhere near hitting a limit.

Obviously, this can be a consideration - I wouldn't want to set a
several minute delay on one of the traps currently doing 700 emails/second.

I've tried this on two of our smallest traps for this very reason - and
this is why I was only able to relate the effects on the very highest
volume bots.

[One of our traps is capable of and has _seen_ sustained thruput of 6-7
_thousand_ per second.]

> Can you confirm the max-daemons limit wasn't hit?

Yup, see above.

> On a real MX, rather than being fixed at 30 seconds, the banner delay
> should be made proportional to the spammitude reckoned for the sending
> IP.  Sort of tarpitting, perhaps not the FUSSP itself, but...

Right.  I did this as simply as I could.  In production, you'd want
something much smarter.

The idea isn't to "punish" (like tarpit), the idea is to distinguish as
simply and cheaply as possible what is highly non-RFC-compliant, and
ignore ignore them from then on.

Something like;

> if (this is the first time in t0 seconds you've seen a given IP)
>    then
> 	banner delay it for t1 seconds.
> elsif (this is the second time in t0 seconds you've seen a given IP)
>     then
>         if (it previously passed)
> 	      then
> 	     	  don't banner delay
> 	      else
> 		  banner delay for t1 seconds
> else
>     if (it previously passed)
>         then
>             don't banner delay
> 	  else
>             drop at connect

then twiddle the tn's.   The second "kick at the delay" is to allow for
a legitimate mail server or network hiccup - the banner delay will be
treated as a retry and you want the retry to work.