Re: IETF Mail List Failure

Spencer Dawkins at IETF <spencerdawkins.ietf@gmail.com> Fri, 18 May 2018 14:04 UTC

Return-Path: <spencerdawkins.ietf@gmail.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 355DD12D962; Fri, 18 May 2018 07:04:50 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level:
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HlX5j-G9LScZ; Fri, 18 May 2018 07:04:43 -0700 (PDT)
Received: from mail-yw0-x235.google.com (mail-yw0-x235.google.com [IPv6:2607:f8b0:4002:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6FA9212D947; Fri, 18 May 2018 07:04:43 -0700 (PDT)
Received: by mail-yw0-x235.google.com with SMTP id g16-v6so2438446ywk.0; Fri, 18 May 2018 07:04:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=4ZwjjxyRe1BiK9go/gBXid7xNQs/koTk5JkXGQx+VPE=; b=g1OfhcX7xYM2hbGwq2E9c1O8GRpaCbeZ9+iYFTOZr8vn2XWstsSPkqteNHTkKjN1jx hWKu0lXdvbeOG6a4fXVX/0EHrajZpNtDTkGdu5vyZxVzpQvrdP49vZ42VNmFFPvc31P1 Yq+6EAYwQvCFly0o22KpX4ieLDgVWwEZJ1ijtjxA4FkhIobYb/d1U+pvT6zCCDQWcj27 8qFxgaGm+Z9QgoiO/MwSS8AtzTTU8/KHPHjOcWFCNNjdN85g47fmrLV8dWkXQlzeAJ5s mJXKI/1nMC/8c3XIzI0LKTlF3FDRyhmy0u1WCJSIdDhUbGIW6uq9LAn6Rl1GxW2RyTf0 3IpQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=4ZwjjxyRe1BiK9go/gBXid7xNQs/koTk5JkXGQx+VPE=; b=XXL3EUy5riiruyUePxKCNVSnVO7Rsr6xUWmXUyvGGvpZzfu+Zqg87NL8QiN087o0LV CylnDXH86qLuoHowKLeWD5joukH2COmwxrKmNrBPLIFQs5XKs/RqeKKEFU3NE+bcg9SN NnEQ1YgY2akEjJT9MOuknv+ogiVnVkLBxzfeG15vvQCpEBVbO0bXCkhqvo7aGNhP+xL/ qBU4d/H+Gzxzub9z5YMTGK2brxppw8Vs15w4n0gRd/8IrK9EWr859d4LPUlD9fveFV/9 M8kmp1Yyv95IgWfFji5oEMcEXEe2DO69M21aOMzyiyDWafib98bxU7TFtOJIcD1QWWIS cWmg==
X-Gm-Message-State: ALKqPwcNKZWbzqBv7B9NkTNYKMehaQf3i0VH/skhr9SY9+j4722rXZgD 5+O/XLh9LSAmudT4oM4uWUx0syU5rue3I5YQztQ=
X-Google-Smtp-Source: AB8JxZqx2SY9ij98xG26aCQ3sCD2MkSYDdUMkByQ/Xu/q/OZASnD7lGXnPO548FWC5Ui4du7WXkIGLU6xQdfSgSJ+Ko=
X-Received: by 2002:a0d:ca96:: with SMTP id m144-v6mr4757975ywd.279.1526652282488; Fri, 18 May 2018 07:04:42 -0700 (PDT)
MIME-Version: 1.0
References: <152661185460.1578.3189815605270968957.idtracker@ietfa.amsl.com> <m2603lieru.wl-randy@psg.com>
In-Reply-To: <m2603lieru.wl-randy@psg.com>
From: Spencer Dawkins at IETF <spencerdawkins.ietf@gmail.com>
Date: Fri, 18 May 2018 09:04:29 -0500
Message-ID: <CAKKJt-epe5HTk3eEhT6+y-wEv2Ae+A-5ePF8gAZjfsnFkyQ1Mg@mail.gmail.com>
Subject: Re: IETF Mail List Failure
To: Randy Bush <randy@psg.com>
Cc: exec-director@ietf.org, IETF list <ietf@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000fe4e7d056c7b6ec0"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/TE36HTjNIZ1PQoR5Q7AS3gFzCP8>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 18 May 2018 14:04:50 -0000

Hi, Alexa,

On Fri, May 18, 2018 at 2:50 AM Randy Bush <randy@psg.com> wrote:

> > On 17 May, from 17:48 UTC (13:48 EDT) to 23:48 UTC (18:49 EDT),
> > outbound email from the IETF email lists failed to send. Mail sent to
> > the lists went into the archives, but was not received by list
> > subscribers.
> >
> > A problem in the DMARC-rewriting system caused outbound mail delivery
> > to fail silently, without alarm or error message. The rewriting system
> > has been taken offline, which restored outbound email service.  The
> > lists are back up.
>

I echo Randy's thanks for technical details.


> > Unfortunately, because the mail was discarded rather than queued, if
> > you sent mail to any IETF list today during the outage window, we ask
> > you to please resend your message.
>

Just so I do the right thing, do you folks think all mail to be forwarded
was silently discarded, or only the mail sent from addresses that were
subject to re-writing?

Spencer


> > We apologize for this issue and are working with the Tools Team to
> > correct the system before it is put back online again.
>
> thanks, alexa.  these things happen, and i won't even demand a refund :)
> sympathies for the pain.
>
> < micro-management = off >
>
> i really appreciate technical details.  there are still a few operators
> left in the ietf community, and we try to learn from our oopsies.  a
> deeper post mortem would even be welcome as many of us are also trying
> to deal with dmarc hell.  you run a medium scale mail operation and any
> lessons you share are appreciated.
>
> randy
>
>