Re: [Sidrops] RPKI Outage Post-Mortem

Nathalie Trenaman <nathalie@ripe.net> Mon, 11 January 2021 08:40 UTC

Return-Path: <nathalie@ripe.net>
X-Original-To: sidrops@ietfa.amsl.com
Delivered-To: sidrops@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 999933A16FF for <sidrops@ietfa.amsl.com>; Mon, 11 Jan 2021 00:40:21 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level:
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=ripe.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aVYlV2RPRWxO for <sidrops@ietfa.amsl.com>; Mon, 11 Jan 2021 00:40:20 -0800 (PST)
Received: from molamola.ripe.net (molamola.ripe.net [IPv6:2001:67c:2e8:11::c100:1371]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6DFD53A0EFA for <sidrops@ietf.org>; Mon, 11 Jan 2021 00:40:20 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ripe.net; s=s1-ripe-net; h=To:Cc:Date:Subject:Mime-Version:Content-Type:Message-Id: From; bh=aG440ghX1+yuaxJfHrJkNtQ7lMk5ot1JFOF6t3tv1vI=; b=oYvOo8zYl5wKL8o8M0mW 76D2PsR5UVmPjlLddPR6hYAnVtYBaa/VBX18Vu8spVD5o258RpHKDWvPdpGZ5Y0b48AMLkRwqoxLO Grfa7nRmPTtcngE7YNVtIm55jSQABZzxKh5tmjFHtQE7iNsMLgLh63j2OkQXP4Uali4br5Ds1XghB yZSqH6tNqKR8MYn2ovPl5sFGxKRGB0OFklwx4j3MxWPQpqwV36UcImgwzwfN+gsxjz/sYgI2BtIgj oND+gGVfOy2zc2ZvhwLFRyEL+0vrqowlfNunErDL3LnQwTps9QW2ixWYE3AIs2HaJOTmIZqSPMM6G VXoUP1I5O7dd0w==;
Received: from allealle.ripe.net ([193.0.23.12]:51360) by molamola.ripe.net with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from <nathalie@ripe.net>) id 1kyskL-0008uj-Q2; Mon, 11 Jan 2021 09:40:17 +0100
Received: from sslvpn.ipv6.ripe.net ([2001:67c:2e8:9::c100:14e6] helo=[IPv6:2001:67c:2e8:1200::50e]) by allealle.ripe.net with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from <nathalie@ripe.net>) id 1kyskL-0006G5-NA; Mon, 11 Jan 2021 09:40:17 +0100
From: Nathalie Trenaman <nathalie@ripe.net>
Message-Id: <DF42061A-2414-4725-9031-42CEF1F4E79C@ripe.net>
Content-Type: multipart/alternative; boundary="Apple-Mail=_6F359678-8D78-48F8-9A73-D3BA20678C62"
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\))
Date: Mon, 11 Jan 2021 09:40:17 +0100
In-Reply-To: <CAGQUKcc+t5M1QXaB3wgn=2-BmCi2cgRsd51UW5T9szRfB1Ld4A@mail.gmail.com>
Cc: SIDR Operations WG <sidrops@ietf.org>
To: Tony Tauber <ttauber@1-4-5.net>
References: <11932542-611A-4DDC-AD2D-3356E0CB44ED@ripe.net> <CAGQUKcc+t5M1QXaB3wgn=2-BmCi2cgRsd51UW5T9szRfB1Ld4A@mail.gmail.com>
X-Mailer: Apple Mail (2.3608.120.23.2.4)
X-ACL-Warn: Delaying message
X-RIPE-Signature: b23882c8c47abee4cf35af21618ca92a1e8ce56982d5343daab751b9025d1f78
Archived-At: <https://mailarchive.ietf.org/arch/msg/sidrops/QKJmWTkL5ssiMGalvdr7DrK-nI4>
Subject: Re: [Sidrops] RPKI Outage Post-Mortem
X-BeenThere: sidrops@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: A list for the SIDR Operations WG <sidrops.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sidrops>, <mailto:sidrops-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/sidrops/>
List-Post: <mailto:sidrops@ietf.org>
List-Help: <mailto:sidrops-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sidrops>, <mailto:sidrops-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Jan 2021 08:40:22 -0000

Hi Tony,

> Op 8 jan. 2021, om 16:34 heeft Tony Tauber <ttauber@1-4-5.net> het volgende geschreven:
> 
> On Fri, Jan 8, 2021 at 8:56 AM Nathalie Trenaman <nathalie@ripe.net <mailto:nathalie@ripe.net>> wrote:
> <snip>
> Some older Relying Parties had applied a strict manifest handling interpretation in their validator software. This meant that they were configured to reject all certificates in the manifest if a single entry was invalid. As a consequence, all RPKI certificates covering RIPE resources were rejected by these validators during this period.
> 
> Based on our access logs, we estimate that 327 instances of Relying Party software were impacted.
> 
> Hi Nathalie,
> 
> Thank you for the detailed write-up.
> 
> I'm curious how you arrived at this estimate of "...327 instances impacted"?
> 
> I'm guessing many more instances are out there querying RIPEs repository, even w/in the outage window.
> But maybe I'm mistaken?
> 
> Tony

We can see the user agent that connects to our repository. So we can see the type of Relying Party software and the version number. Based on this, plus the knowledge which versions were impacted we made this estimation. 
You are right that there are many more instances querying the repo, but not all of them were impacted.

Nathalie