Re: [ietf-smtp] [Emailcore] Status of Greylisting (i'd wish MessageID were part of SMTP prologue)

[Steffen Nurpmeso <[e-mail]>]

John Levine wrote in
 <[e-mail]>:
 |It appears that Steffen Nurpmeso  <[e-mail]> said:
 |>This is interesting, for example the Firefox browser i use only
 |>can manage one password for all the IETF mailing-lists i am
 |>subscribed too (iirc, many many months, but it tried to auto-fill
 |>a false one now, and i definetely recall having problems with
 |>password auto-fill for mailing-list subscriptions), so
 |>auto-filling the password just does not work.
 |
 |I believe you, but that has nothing whatsoever to do with SMTP or
 |the way that mail works.

It was only a ready to hand example of the importance of
identifiability (of course).

 |>These are interesting numbers far beyond mine, thank you!  (It
 |>surely will increase now that i post on @ietf.org, as always.)
 |>Yes /24, not /8.  Really very interesting that /24 is of so much
 |>use even today.  Many sites use multiple "deferred" until a retry
 |>is accepted, and your "one month" white listing is also a number
 |>quite large i think.
 |
 |It might as well be forever.  The only point of greylisting is
 |to see if an MTA follows the spec well enough to retry.  Once
 |you've seen a retry, greylisting is of no further use for that
 |source.  The only reason I time out after a month is to keep
 |the local whitelist database from getting too big.

Ok i mean with all the checks that the SMTP server performs before
the greylist policy even comes into play (most importantly DNS domain
name and address/mapping verification) this may indeed be
doable, but i do see very much different behaviour, for example by
NetBSD.org, with multiple deferrals and short-time whitelisting.
(Without knowing the actual configuration details, and happily
going down a colour name route btw.)

 |>I find it interesting that such simple greylisting that cannot
 |>even correctly identify a specific message seems so useful still.
 |
 |But identifying the specific message is not important. How often does
 |a spambot send two different messages with the same envelope in a
 |time gap that makes it look like a retry? Basically, never.

You know and that is what is so hard to believe.  Given that the
concept is twenty years old and the standard becomes ten this
year, wouldn't it make sense for a bot to simply try an address
a second time after X minutes, if it has the time and space?

Never done that myself, but in my logs i have seen SSH attacks
which really figured out firewall delays and then tried logins in
an interval that was not causing blacklisting, over months!  (I
actually cheered the attacker via a public list he/she might have
read, and doing this now and here, again!  (The login as such was
not successful, thus.))

What i also have seen is, i tried out OpenSMTPD once for a day, it
might have been in 2017, it was to my shame a misconfiguration
(falsely read the manual) that turned it into an open relay, and,
i really should have kept the logs because it was so fascinating,
one IP connected, and did nothing for several minutes, then
another IP connected, and then they started sending mails
simultaneously (how did they know??), to my shame.  (All against
Yahoo Taiwan if i recall correctly, then there was a friendly
email from Yahoo Taiwan iirc that my account produces suspicious
activity, massively understated.)  It was a nice interval (for
nothing), IPs came, send massively for some time, then went again.
(Luckily i looked once again before i went, reinstalled postfix
and with its *error_limit= directive that storm subsided within
minutes.)

What i mean is, there are smart attacks doable and out there, and
whereas the small attack surface of greylisting may not seem worth
the effort, and maybe only today, improving this situation by
strengthening the concept of greylisting standard-wise may bring
benefits in the future.

 |I have tried a version of greylisting that did the retry after DATA
 |and kept a hash of the message. I found that a certain number of
 |systems regenerate the retried message in a form that is not quite the
 |same, e.g., different timestamps. The hashes don't match and the
 |retries always failed.

I personally did not think about greylisting at such a late time,
quite a lot of processing has happened already by that time.
I thought with say a regular SMTP extension that brings the unique
Message-ID into the SMTP prologue greylisting could be
strengthened by allowing exact identification of messages, and so
greylisting and its delays could be improved by allowing the
greylist daemon to let exact matches pass through after one retry,
whereas software which does not use the extension could continue
to be penaltized by even (many) hours, as can be seen in practice.

I mean, of course grey listing affects only a small subset of
traffic at all, and many gigantic providers like gmail etc., as
well as a lot of well-known domains, they all are or have to be
whitelisted anyway, best at pre-greylisting stages, of course
(though greylist server whitelisting may even be faster than
explicit whitelisting dependent on the used database format and
the file system's locking speed).

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)