[TOOLS-DEVELOPMENT] Mailman subscribe attacks - Cloudflare

Glen <glen@amsl.com> Thu, 17 September 2015 14:42 UTC

Return-Path: <glen@amsl.com>
X-Original-To: tools-development@ietfa.amsl.com
Delivered-To: tools-development@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 869C01A6F47 for <tools-development@ietfa.amsl.com>; Thu, 17 Sep 2015 07:42:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -99.389
X-Spam-Level:
X-Spam-Status: No, score=-99.389 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NRqo6joWEFzh for <tools-development@ietfa.amsl.com>; Thu, 17 Sep 2015 07:42:01 -0700 (PDT)
Received: from mail.amsl.com (mail.amsl.com [IPv6:2001:1900:3001:11::28]) by ietfa.amsl.com (Postfix) with ESMTP id 304081A6F3A for <tools-development@ietf.org>; Thu, 17 Sep 2015 07:41:55 -0700 (PDT)
Received: from mail.amsl.com (localhost [127.0.0.1]) by c8a.amsl.com (Postfix) with ESMTP id 7FC6D1E5A36 for <tools-development@ietf.org>; Thu, 17 Sep 2015 07:41:11 -0700 (PDT)
Received: from mail-oi0-f46.google.com (mail-oi0-f46.google.com [209.85.218.46]) by c8a.amsl.com (Postfix) with ESMTPSA id 5CDA41E5A2A for <tools-development@ietf.org>; Thu, 17 Sep 2015 07:41:11 -0700 (PDT)
Received: by oixx17 with SMTP id x17so10607527oix.0 for <tools-development@ietf.org>; Thu, 17 Sep 2015 07:41:54 -0700 (PDT)
X-Received: by 10.202.68.215 with SMTP id r206mr26303951oia.87.1442500914430; Thu, 17 Sep 2015 07:41:54 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.202.102.71 with HTTP; Thu, 17 Sep 2015 07:41:35 -0700 (PDT)
From: Glen <glen@amsl.com>
Date: Thu, 17 Sep 2015 07:41:35 -0700
Message-ID: <CABL0ig4HaNMyu=FYGsw1e46-1FK0Mnhwjmw6V8SLCXWbYnZHYQ@mail.gmail.com>
To: Glen Barney <glen@amsl.com>
Content-Type: multipart/alternative; boundary="001a113dce6097cd1e051ff26ad5"
Archived-At: <http://mailarchive.ietf.org/arch/msg/tools-development/CvUyY0jPWEUryliquxSJOnf2hJk>
X-Mailman-Approved-At: Thu, 17 Sep 2015 07:42:49 -0700
Subject: [TOOLS-DEVELOPMENT] Mailman subscribe attacks - Cloudflare
X-BeenThere: tools-development@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: glen@amsl.com
List-Id: Tools Development list server <tools-development.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tools-development>, <mailto:tools-development-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tools-development/>
List-Post: <mailto:tools-development@ietf.org>
List-Help: <mailto:tools-development-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tools-development>, <mailto:tools-development-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 17 Sep 2015 14:42:02 -0000

All -

As previously noted, one of the reasons we've been unable to just directly
block the hosts which are participating in the Mailman attack is because of
our use of Cloudflare.  Because we are behind Cloudflare, all incoming
requests appear (to our IP stack) to be coming from Cloudflare (which,
technically, they are.)  Cloudflare does pass the remote IP address in a
transaction header, but firewalls can't do anything with that.

When we began using Cloudflare, we expected that they would detect and
block attacks of this type.  Their apparent inability to do so, combined
with their lack of responsiveness, increases the difficulty we face in
dealing with these attacks.  As I have mentioned previously, we have
reached out to Cloudflare for help with this on several occasions - we have
explained the situation to a number of Cloudflare representatives - and we
have never received any response or guidance from them at all.  Just
yesterday, Russ connected me to a new Cloudflare person - I responded to
that immediately, but have not heard back from them at all as of yet.  I
have high hopes that this new contact will work with us, but I mention this
again so that you're all aware that we're trying to work within this system.

Meanwhile, working independently, one of my engineers has discovered that
there may be an automated way to submit IP bans to Cloudflare.  He is
currently running this down to determine whether such a thing actually
exists, and whether its implementation is feasible.  Since we have had no
communication or support response from Cloudflare, we are, in essence
guessing, but we are pursuing this possibility in the hope that we can
better fight off the attack in this way.

Of course, this is not an ideal solution either:  as additional hosts
become compromised, the "ban list" will grow.  And, once banned, we will no
longer know if a particular host is still compromised and/or attacking us,
so we will have no way to know if we should "unban" that host.  I do not
see this as a permanent solution either, but I wanted to make the
leadership and tools teams aware of this new possibility, so you all are in
the loop as we continue to fight this attack.

As pointed out on some threads - we still hope for a more comprehensive,
high-level solution to problems of this type, at some point in the future.

As always, any questions, let me know.

Best regards,
Glen
Glen Barney
IT Director
AMS (IETF Secretariat)