RE: [Asrg] Some data on the validity of MAIL FROM addresses

"Clayton, Nik [IT]" <nik.clayton@citigroup.com> Wed, 21 May 2003 11:27 UTC

Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged)) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA15057 for <asrg-archive@odin.ietf.org>; Wed, 21 May 2003 07:27:06 -0400 (EDT)
Received: (from mailnull@localhost) by www1.ietf.org (8.11.6/8.11.6) id h4LArpu06586 for asrg-archive@odin.ietf.org; Wed, 21 May 2003 06:53:51 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176]) by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h4LArpB06583 for <asrg-web-archive@optimus.ietf.org>; Wed, 21 May 2003 06:53:51 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA15052; Wed, 21 May 2003 07:26:35 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 19IRic-00052y-00; Wed, 21 May 2003 07:25:14 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org) by ietf-mx with esmtp (Exim 4.12) id 19IRib-00052v-00; Wed, 21 May 2003 07:25:13 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1]) by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h4LAjsB06340; Wed, 21 May 2003 06:45:54 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176]) by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h4LAijB06305 for <asrg@optimus.ietf.org>; Wed, 21 May 2003 06:44:45 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA14915 for <asrg@ietf.org>; Wed, 21 May 2003 07:17:28 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 19IRZo-000519-00 for asrg@ietf.org; Wed, 21 May 2003 07:16:08 -0400
Received: from mail2.ssmb.com ([199.67.141.25] helo=imbaspam-nj03.iplex.ssmb.com) by ietf-mx with esmtp (Exim 4.12) id 19IRZn-000514-00 for asrg@ietf.org; Wed, 21 May 2003 07:16:07 -0400
Received: from imbarc-nj01.nj.ssmb.com (imbarc-nj01-2 [150.110.115.169]) by imbaspam-nj03.iplex.ssmb.com (8.12.9/8.12.9/SSMB_EXT/evision: 1.19 $) with ESMTP id h4LBGUoB009605; Wed, 21 May 2003 07:16:31 -0400 (EDT)
Received: from mailhub-dc2.eur.nsroot.net (mailhub-dc2.eur.nsroot.net [169.186.205.199]) by imbarc-nj01.nj.ssmb.com (8.12.9/8.12.9/SSMB_QQQ_IN/1.1) with ESMTP id h4LBGRPm025729; Wed, 21 May 2003 07:16:28 -0400 (EDT)
Received: from exchuk13.eu.ssmb.com (imceu.eu.ssmb.com [169.186.152.113]) by mailhub-dc2.eur.nsroot.net (8.9.3-GEMSp2/8.9.3/SSMB-HUB) with ESMTP id MAA20154; Wed, 21 May 2003 12:16:25 +0100 (BST)
Received: by imceu.eu.ssmb.com with Internet Mail Service (5.5.2655.55) id <K6HV5608>; Wed, 21 May 2003 12:16:25 +0100
Message-ID: <80AD1C20BCECD6118FD800508BCF7EDA72D9CD@exchuk19.eu.ssmb.com>
From: "Clayton, Nik [IT]" <nik.clayton@citigroup.com>
To: 'Vernon Schryver' <vjs@calcite.rhyolite.com>, asrg@ietf.org
Subject: RE: [Asrg] Some data on the validity of MAIL FROM addresses
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2655.55)
Content-Type: text/plain; charset="ISO-8859-1"
X-Scanned-By: MIMEDefang 2.33 (www . roaringpenguin . com / mimedefang)
Sender: asrg-admin@ietf.org
Errors-To: asrg-admin@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>, <mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>, <mailto:asrg-request@ietf.org?subject=subscribe>
List-Archive: <https://www1.ietf.org/pipermail/asrg/>
Date: Wed, 21 May 2003 12:16:21 +0100

> I don't know but assume that the SpamAssassin milter interfaces (I
> think there are two) use the same information for per-user rules and
> scoring; it certainly could.  I don't know but suspect the milter
> mechanism does not handle .forward files, but I don't think that's a
> significant problem.  The milter mechanism handles alias files and at
> least some other things that cause sendmail to forward the message
> elsewhere in the sense of telling the milter filter program as much.

The Milter interface allows third party applications to hook in to the 
SMTP conversation between Sendmail and the remote MTA, and:

  a) Override Sendmail's choice of SMTP status codes to return at each
     stage of the process.  So the milter can selectively refuse 
     messages with finer-grained control than is possible with the 
     sendmail.cf rules and external maps.

  b) Change everything about the message (envelope information, header,
     body (including attachments) before Sendmail queues the message for
     onward delivery (or hands it off to a local delivery agent).

So all this happens *before* Sendmail consults .forward files, and other
paraphenalia.

SpamAssassin doesn't have a "native" milter interface (i.e., one supplied
by the SpamAssassin project).  At least two projects run as milters, and
(may) call out to the SpamAssassin libraries as part of their filtering,
spamass-milter, and MIMEDefang.

Out of the box, it's tricky, but not impossible, to support per-user
filtering using this mechanism.  At first glance this isn't possible at
all, because SMTP doesn't allow the response to the DATA portion of the
transaction to say "We accept this message for recipients 1 and 3, but
it's been rejected for recipient 2".

At least two workarounds exist for this.  One is to configure Sendmail to
only allow one recipient per SMTP transaction.  I.e., return 250 to the
first RCPT, and 550 to the others.

This increases the load on your mail servers, and we've seen a 
significant number of connections from MTAs that fail to handle this 
properly (i.e., they temp. or perm. fail the message for *all* 
recipients, not just the ones that were 550'd).

The other (which is employed by MIMEDefang if you turn on the necessary
option, it may also be used by others) is to accept the message with a 
rewritten envelope.  The remote server receives a 2xx response to the 
DATA command.  MIMEDefang then resends the message, once per recipient, 
so the filter can make per-recipient filtering decisions.

This is not pretty:

  a) Rejections at this point will generate full bounce messages at
     the local MTA, which, if it's spam, may have invalid return
     addresses, and/or be to servers with poor connectivity, so your
     outbound mail queue gets quite large.  Rejecting during the initial
     SMTP conversation is preferred, because then the remote system has
     to generate the bounce message.

  b) The amount of horse power needed to process mail increases.  Our 
     stats show that roughly 40% of our inbound SMTP connections refer
     to multiple recipients.

N
-- 
1        1         2         3         4         5         6         7    7
         0         0         0         0         0         0         0    5
                                                    -- The 75 column-ometer
Global Messaging,                 A: Top posting
120 Cheapside, x83331             Q: What's the most annoying e-mail habit?
_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg