Re: [ietf-smtp] [Emailcore] Status of Greylisting (i'd wish MessageID were part of SMTP prologue)

[Steffen Nurpmeso <[e-mail]>]

John Levine wrote in
 <[e-mail]>:
 |It appears that Steffen Nurpmeso  <[e-mail]> said:
 |>I am only implementing a greylisting daemon, a standardized
 |>anti-spam aka anti-abuse policy, then further hopefully next week, ...
 |
 |Everyone I know who does greylisting with Postfix uses Postgrey,
 |available here:
 |
 |https://postgrey.schweikert.ch/
 |
 |Are you sure you need to reinvent this particular wheel?

Well, the answer is a bit lengthy, i know this software, and
i have perl on my server at the moment.  (Ever since lighttpd
suddenly stopped creating cgit output, and on the cgit list then
was no support at all even though maintainer of lighttpd correctly
pointed out that failing a sendfile(2) should be covered as per
"Applications may wish to fall back to read(2)/write(2) in the
case where sendfile() fails with EINVAL or ENOSYS", but the
provided patch did not even get a response ... the maintainer
as a kernel guru in some sense of guru seems to have known it is
a kernel bug, but no report was given to the kernel people until
someone finally stepped up .. anyhow it took months before the
very combination Linux / lighttpd / cgit resumed normal operation
without local effort, and one day i was no longer willing to and
fell back to gitweb that ships with git anyhow, .. which requires
perl.)

So that postgrey uses perl would be ok even though it uses quite
a bit of modules.  It uses BerkeleyDB, which is no longer free
software, version 5.3.28 is, aka
  Berkeley DB 11g Release 2, library version 11.2.5.3.28:
  (September 9, 2013)
as via [1], which is quite old.  But ok.  It however seems to use
what i call "[Chris ]Torek Hash", which faded a bit to grey when
facing possible real world attacks me thinks.  It is also very big
and slow, and imposes expensive locking.

  [1] http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/

As an effort to restrict all my (easy non-SQL, and only those here
right now) DB use cases to only LMDB of the OpenLDAP project for
example i wrote an accepted patch for bogofilter(1) in 2018, and
by then these were the numbers

  | |> it is very small (the raw AlpineLinux code package is 90KB,
  | |> whereas DB is 1.6MB; the cloned repo is 1.2MB, whereas the 5.3.28
  | |> DB tar ball unpacked in git is 31MB), and the code is also open
  | |> and openly maintained.  And Postfix supports LMDB as a replacement
  | |> for DB out of the box, too.  All this is very desirable to me.
  | |
  | |Repo size of a support library isn't normally a relevant metric, but
  | |this is a valid point, as is its license:
  | |
  | |   text          data     bss     dec     hex filename
  | |  80510          1504       8   82022   14066 /usr/lib64/liblmdb.so
  |
  |Runtime is much smaller here, too:
  |
  |  #?0[steffen@essex nail.git]$ size /usr/lib/liblmdb.so
  |     text    data     bss     dec     hex filename
  |    69680    1344      80   71104   115c0 /usr/lib/liblmdb.so
  |  #?0[steffen@essex nail.git]$ size /usr/lib/libdb.so
  |     text    data     bss     dec     hex filename
  |  1549515   38744      64 1588323  183c63 /usr/lib/libdb.so

(It does not support VACUUM or any such, therefore i dump-out and
recreate these DBs in such heavy use cases once a month
.. automatically.)  Very, very fast and minimal overhead.

But in fact all this is part of an effort to replace the
end-of-life Python2 Mailman2 with something homegrown.
I initially planned to use the policy server to whitelist
subscribers fast, for example.  That it also should do regular
greylisting was not the first use case.  However, in the meantime
i added regular use cases for a postfix configuration

   check_sender_access lmdb:/etc/postfix-lmdb/sender_access,

before the envisioned check_policy_service, which made this less
desirable, because LMDB uses fast read/write locks (or what i do
not like, "robust" mutexes) if available, and then even very fast
unix(7) communication will likely be outperformed.
However, maybe that will be replaced by whitelist on the policy
server side (again).
It will use SipHash for its all-in-memory dictionaries.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)