Re: IETF Service Outage

Eliot Lear <lear@cisco.com> Wed, 23 March 2016 12:59 UTC

Return-Path: <lear@cisco.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5288112D659 for <ietf@ietfa.amsl.com>; Wed, 23 Mar 2016 05:59:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.531
X-Spam-Level:
X-Spam-Status: No, score=-14.531 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lIeokDgCIQCO for <ietf@ietfa.amsl.com>; Wed, 23 Mar 2016 05:59:42 -0700 (PDT)
Received: from aer-iport-4.cisco.com (aer-iport-4.cisco.com [173.38.203.54]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DE0EB12DB8D for <ietf@ietf.org>; Wed, 23 Mar 2016 05:51:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2620; q=dns/txt; s=iport; t=1458737461; x=1459947061; h=subject:to:references:from:message-id:date:mime-version: in-reply-to; bh=TEUbXccDTPKefT92gQF6bCSXCpKqAwTMXbxBEbIp9W0=; b=d4cWKxxWtU48ZkEURdH0nxuCIrWoS9QaEDTfOjB93wO4jRx95t1B3gtH p1+6ccjtjGWAlunNG7GqDKUxRgGDLSOiDCRbGabfc10VpUGuBF5yM+Kw2 YQaFvqmksfq1lHSaBAI3J2fBPh4bLyM9+8MsJiZY2R4t0YAPiiLZJ8y3O 0=;
X-Files: signature.asc : 481
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: A0CpBAAbkPJW/xbLJq1evVOEDYYNAoIHAQEBAQEBZSeEQgEBBCNLChELGAkWCwICCQMCAQIBRQcMCAEBiCOvZ5EDAQEBAQYBAQEBFAiKYoQMEQEGgxiCVgWXWoMegWaJAIk2hVSPB2KCAxmBSjuJB4EyAQEB
X-IronPort-AV: E=Sophos;i="5.24,382,1454976000"; d="asc'?scan'208";a="636486719"
Received: from aer-iport-nat.cisco.com (HELO aer-core-4.cisco.com) ([173.38.203.22]) by aer-iport-4.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Mar 2016 12:50:59 +0000
Received: from [10.61.64.203] (ams3-vpn-dhcp203.cisco.com [10.61.64.203]) by aer-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id u2NCoxXh023429; Wed, 23 Mar 2016 12:50:59 GMT
Subject: Re: IETF Service Outage
To: ietf@ietf.org, glen@amsl.com
References: <CABL0ig4s9PBVS5icz4y=M2nJXtcPH9iVFEptooLsRUCnUjprMg@mail.gmail.com>
From: Eliot Lear <lear@cisco.com>
Message-ID: <56F29132.4010109@cisco.com>
Date: Wed, 23 Mar 2016 13:50:58 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.6.0
MIME-Version: 1.0
In-Reply-To: <CABL0ig4s9PBVS5icz4y=M2nJXtcPH9iVFEptooLsRUCnUjprMg@mail.gmail.com>
Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="j4fLjp68QB6LqGUVXldcU1Ngcm2htet8L"
Archived-At: <http://mailarchive.ietf.org/arch/msg/ietf/zTRmYREviMxqjfgSwMvvcHGXi_Q>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 23 Mar 2016 12:59:44 -0000

Barney,

I'm curious- do you have sufficient data to perform an analysis to
determine the source of the service degradation?

Thanks,

Eliot


On 3/23/16 1:39 PM, Glen wrote:
> Greetings:
>
> Today, we experienced a degradation in network service on the primary
> IETF server.  Network service appears to have been impaired for just
> over an hour.
>
> Our automated systems did detect this issue, and we also received an
> emergency alert message from Stephen Farrell, and began an
> investigation immediately.  Before we could determine a cause, the
> degradation cleared, and network service was back to normal.
>
> In the meantime, this network issue did cause availability
> interruptions for the Datatracker, the mailing lists and related
> services; however, no data was lost, and no system damage occurred.
> Email sent during that interval might be delayed, but should be
> retried by sending systems over the next 1-4 hour period, and all
> other services are up and available as normal.
>
> We apologize for this inconvenience.  We will continue to monitor and
> address any further issues as they occur.
>
> Glen
> Glen Barney
> IT Director
> AMS (IETF Secretariat)
>
>