Re: [Sidrops] ARIN RPKI Repository issue - Wednesday 24 October 2018

Tim Bruijnzeels <tim@nlnetlabs.nl> Fri, 26 October 2018 09:37 UTC

Return-Path: <tim@nlnetlabs.nl>
X-Original-To: sidrops@ietfa.amsl.com
Delivered-To: sidrops@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BC86912D4F2 for <sidrops@ietfa.amsl.com>; Fri, 26 Oct 2018 02:37:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.121
X-Spam-Level:
X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=nlnetlabs-nl.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v2CraeeNLsEt for <sidrops@ietfa.amsl.com>; Fri, 26 Oct 2018 02:37:10 -0700 (PDT)
Received: from mail-ed1-x541.google.com (mail-ed1-x541.google.com [IPv6:2a00:1450:4864:20::541]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B530D1252B7 for <sidrops@ietf.org>; Fri, 26 Oct 2018 02:37:09 -0700 (PDT)
Received: by mail-ed1-x541.google.com with SMTP id t10-v6so689410eds.12 for <sidrops@ietf.org>; Fri, 26 Oct 2018 02:37:09 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nlnetlabs-nl.20150623.gappssmtp.com; s=20150623; h=from:message-id:mime-version:date:subject:in-reply-to:to:references; bh=oYylAnPE047pdiZa4ggxPIGxiL5AWbl9XDbYAry7jgA=; b=Cm9CtDa3iZynHNUhoVIT6HoAtEb3sZIRmTwYi4iRtnqIi/ZuO5Dv4A3j06jqElleEt gMjzsEUEDOtLRW6OmKVmkvNfce43wDEwU7PBoC73QzJRXqEEPPUi4RogPaxXus//F8QY 2ttSiGJsxeU991fal3B+ej2xae2OyAb7QgsnjyB37PHyUOkgKiLxuUdHNVZtZbGd+vO0 vf/3/D/17k/3rcz4ynq9XkxXFG7dZB7A/7cHcA7QIyvnqTtlIlnlOVV6lAXzPgxAg6Tt /0ULWeAXcEGJtKJWJ4A3xpcGaFKCg9WhxR7c2GelvWVe9O9h3oIYO3BpR1uZeO9kL56m Yd6Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:date:subject :in-reply-to:to:references; bh=oYylAnPE047pdiZa4ggxPIGxiL5AWbl9XDbYAry7jgA=; b=N7Ih9OriXgFhyQMmT0wlKj0terqwHahAUXV2VxodQyzh087WtVwI7UqHGUYxlf5+hZ YVMshwAvfwpDuz+Sj2fDK/SH9D1WwjVV1NFrOy3R1kgDjIwXiY9D8gOe1+9/m/xnTO6n c3Bwf1tHflT1QEeYjD11OVZ8be/XmFg/zyV/bUXzA9Oq0ygKvekE8k+ADOwL0Y6d0IgI +eV0v0/+gwokuTW4cov2zkR+aCpeNKx8sqozVHuVHT+JY3NV8vevmIpjQ3AmIuvAqNAX rBSTcwL9PNhsxvPNZ8sYn9w5yOt70T2gtToBr4bYW+oAjrPR+xOOcV98285pIo8QdRud DHtA==
X-Gm-Message-State: AGRZ1gJrxP08sF4sa+eut3VSPmjr531yNRJtleOtXEY0S4pe3bMbj4pn i6aI7RP9/vfWGt7tFRbL+iGrCk6vFy8=
X-Google-Smtp-Source: AJdET5fGeRMYTyp9z1aH8M81LRh5JHhEYU4Cf7XXo8JZ/s7Qdplm2jDyoM6MLYuMAPTDXf2SLhv4Nw==
X-Received: by 2002:aa7:d410:: with SMTP id z16-v6mr2517376edq.26.1540546628022; Fri, 26 Oct 2018 02:37:08 -0700 (PDT)
Received: from [192.168.192.27] (dhcp-089-098-091-015.chello.nl. [89.98.91.15]) by smtp.gmail.com with ESMTPSA id b36-v6sm4824711edb.5.2018.10.26.02.37.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 26 Oct 2018 02:37:07 -0700 (PDT)
From: Tim Bruijnzeels <tim@nlnetlabs.nl>
Message-Id: <3640355A-FDE4-4A26-89ED-5B8134F36E4A@nlnetlabs.nl>
Content-Type: multipart/alternative; boundary="Apple-Mail=_2CD37006-DA1A-4D6B-B358-502B904BE7A0"
Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\))
Date: Fri, 26 Oct 2018 11:37:06 +0200
In-Reply-To: <15B30322-2CE0-4E0B-A54A-86A2743B73AF@arin.net>
To: "sidrops@ietf.org" <sidrops@ietf.org>
References: <15B30322-2CE0-4E0B-A54A-86A2743B73AF@arin.net>
X-Mailer: Apple Mail (2.3445.100.39)
Archived-At: <https://mailarchive.ietf.org/arch/msg/sidrops/SeF-zysKEssMO5I6KfGiCe2of0g>
Subject: Re: [Sidrops] ARIN RPKI Repository issue - Wednesday 24 October 2018
X-BeenThere: sidrops@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: A list for the SIDR Operations WG <sidrops.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sidrops>, <mailto:sidrops-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/sidrops/>
List-Post: <mailto:sidrops@ietf.org>
List-Help: <mailto:sidrops-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sidrops>, <mailto:sidrops-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 26 Oct 2018 09:37:12 -0000

Hi,

> On 25 Oct 2018, at 09:36, John Curran <jcurran@arin.net> wrote:
> 
> ARIN experienced an RPKI Repository issue yesterday afternoon (Wednesday 24 October 2018) which resulted in the number of objects in the repository dropping from its normal hundreds of objects to only 7 objects.  The root cause for the failure was too many ROAs for a resource cert that led to an untested situation whereby we exceeded the MTU for the HSM and an invalid manifest in repository..  It appeared that some validators stop processing after the invalid manifest, thus resulting in a greatly reduced count of objects published. 

So this caused the existing the MFT to go *stale* after 48 hours, while the embedded EE certificate had not yet expired (it uses 7 days from what I can see).

Validator software treats these cases differently, because the standards are vague on this, I believe too vague:
https://tools.ietf.org/html/rfc6486#section-6.4 <https://tools.ietf.org/html/rfc6486#section-6.4>

There are many words here. And some seem contradictory, but let me quote:
[…]the entity that has published the stale manifest at this publication point, SHOULD be viewed as somewhat suspect, but MAY be used by the RP as per local policy.

There is similar vagueness around discrepancies between repository content and manifests. Also.. local policy.

As a result the RIPE NCC RPKI Validator 2.x series leave this local policy as a choice to the user, but defaults to rejecting all stale MFTs. RIPE NCC RPKI Validator 3.x accepts stale MFTs with warnings, but only rejects when the EE certificate is expired. I do not know what RPSTIR does, but Rcynic does the same I believe. As does NLnet Lab's ‘Routinator 3000’ - coming real soon now :)

I am not 100% sure about RPSTIR but all other validators use the ‘local’ policy that the MFT is the authoritative signed statement of what is current for a CA. In other words anything not listed on a MFT, or objects with a different hash, are rejected.

In my opinion it would be best if the relevant sections in the MFT RFC (6486) were updated to ensure that there are no unexpected differences between RP implementations. Accepting stale, rejecting expired && only processing what is on a valid manifest seems the sane way forward to me, and reflects most of the operational reality.

I would be happy to come up with text for people to shoot at :)

Tim