IETF Website Outage

Alexa Morris <> Thu, 08 May 2008 19:41 UTC

Return-Path: <>
Received: from [] (localhost []) by (Postfix) with ESMTP id E64EB28C1E9; Thu, 8 May 2008 12:41:26 -0700 (PDT)
Received: from localhost (localhost []) by (Postfix) with ESMTP id E9D7E3A6C2F for <>; Thu, 8 May 2008 12:41:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -100.495
X-Spam-Status: No, score=-100.495 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FH_RELAY_NODNS=1.451, HELO_MISMATCH_COM=0.553, RDNS_NONE=0.1, USER_IN_WHITELIST=-100]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id YQxVYOluFYqp for <>; Thu, 8 May 2008 12:41:21 -0700 (PDT)
Received: from ( [IPv6:2001:1890:1112:1::14]) by (Postfix) with ESMTP id 03D3128C1E9 for <>; Thu, 8 May 2008 12:41:21 -0700 (PDT)
Received: from localhost (localhost []) by (Postfix) with ESMTP id 9E72D480C5 for <>; Thu, 8 May 2008 12:41:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id nKgMeTlJ6TLz for <>; Thu, 8 May 2008 12:41:12 -0700 (PDT)
Received: from [] (unknown []) by (Postfix) with ESMTP id 6A2C947EEB for <>; Thu, 8 May 2008 12:41:12 -0700 (PDT)
User-Agent: Microsoft-Entourage/
Date: Thu, 08 May 2008 12:41:18 -0700
Subject: IETF Website Outage
From: Alexa Morris <>
To: "" <>
Message-ID: <>
Thread-Topic: IETF Website Outage
Thread-Index: AcixQ3vruntlEx02Ed2B6gAUURi4Ig==
Mime-version: 1.0
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IETF Announcements <>
List-Unsubscribe: <>, <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit

As many of you know, the IETF website was recently unavailable for
approximately 6 1/2 hours -- the time between 11:40pm, May 7th, and 6:15am,
May 8th, Pacific Time.

Many of you in Europe attempted to notify us of the outage and were unsure
of how to reach us, or what number to call. Thank you to everyone who
attempted to notify us, and I apologize for the fact that the process was
more confusing and time-consuming than it should have been.

We are taking a number of immediate steps to clarify the process, and to
make sure that any future outages are brief. Namely:

1) We are implementing an email notification system, which will then notify
the IT staff immediately if an outage occurs. This has actually been in the
works for some time but now, with the recent outage, it has been moved to
top priority. 

2) In case the email notification doesn't work properly for some reason, we
are also setting up an emergency IT hotline be to used to notify the AMS IT
department, whatever the hour. This number will be activated within the next
24 hours and I will distribute the phone number to the entire community as
soon as I have it available.

3) This morning we boosted our remote management capabilities, thereby
broadening our ability to respond to any issue from outside the office.

For those of you who are concerned about why the outage occurred in the
first place, it appears at this stage that the server crashed because of
memory management issues (it ran out of memory) and it appears the memory
issues were caused by a flood of mail (overnight spam) causing a huge
increase in TMDA activity.

So, our preliminary understanding is that the TDMA recently put in place
overloaded the server and brought it down. We are now looking into ways to
limit TMDA activity so that this is not an issue moving forward.

As always, please feel free to contact me with any questions or concerns.


Alexa Morris / Executive Director / IETF
48377 Fremont Blvd., Suite 117, Fremont, CA  94538
Phone: +1.510.492.4089 / Fax: +1.510.492.4001

Managed by Association Management Solutions (AMS)
Forum Management, Meeting and Event Planning <>

IETF-Announce mailing list