[Sidrops] 6486bis: referenced object validation

Ben Maddison <benm@workonline.africa> Thu, 03 December 2020 22:42 UTC

Return-Path: <benm@workonline.africa>
X-Original-To: sidrops@ietfa.amsl.com
Delivered-To: sidrops@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1D0B23A0E57 for <sidrops@ietfa.amsl.com>; Thu, 3 Dec 2020 14:42:28 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.099
X-Spam-Level:
X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=workonline.africa
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IuouGwz7G8uk for <sidrops@ietfa.amsl.com>; Thu, 3 Dec 2020 14:42:24 -0800 (PST)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2051.outbound.protection.outlook.com [40.107.22.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3E2843A0DF9 for <sidrops@ietf.org>; Thu, 3 Dec 2020 14:42:22 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JZkTMwex2IbXKWxq0LvyVeA2Wz7JnvaReOtQlw5bwYfq882SDAMJUKDtl0bmoRE9xZlupiyrhT8i+LQHwsU009n85tKJd9A/nql6syveEaYBh4oj5z5A6pYbhqby20pDvhWax1iyOVQOXtfBf4B3jwXAUZDuxiwwfyvlwKv/EgHxdPK029loT88P6tJeRLY5NcVl0ZbBJwJvuOP04Bs/4uMOHuoRP1GzWauvEd1pfUd1TlVeUB/vtLpTaZwm8Lr4nkK/ENSAbp4RkIVRSez8w9kshlMhNKWWc/UkHSDyRiQIsphcqoef7uzh3HIqz+7QV55ZbHfXXGFJ7Udhs+r28A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AC2f0Sfq4R6mvAUHi6xqLeMd/O58voAMT9p7qMjI6nU=; b=ghML4KQMMaG+l2pAiag9W4zttyYmpXKerRYY92JSZuL1F5gHGEsJBsNjRHYB/MdQdeNedSDkcjoWPxD+PTYAkXJI1/eYKI4cwUfAxQyke9uzlgkBlGQx8uglEHeXNdo1Rl0fYuFAquzjcD8nYYVb2QE98wV6CwcFBHUjY5qRGUzD7hlKDR4WmhXg8a6bKfJ3RLRSWo6aooW0yF8D0EphqKE9YKU4hVCDiPOfP7JQ6HcMSpHxGskba5tnuuKZTtRpFEJqWyTDNzIALCaSZJxf4qJDfJ5kI+Jq3WhdLv1zY/Wb++Md+objNqUrs99xhVQZqf24BKYarVGn8rmFRgP8SQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=workonline.africa; dmarc=pass action=none header.from=workonline.africa; dkim=pass header.d=workonline.africa; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=workonline.africa; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AC2f0Sfq4R6mvAUHi6xqLeMd/O58voAMT9p7qMjI6nU=; b=hosKwPKbV6hHlxzqYZh8D95C1uW5uQILuPNrtxbaN7eJ1cv/hWm0PhOErvq0MoH/8XL7yxa3RCOS4QpXgMxZkfEJNPIHDG0AYzQ+8z1HHXh8Nnvoc8293Lxfo/rjn6MOVOItBFSQqbndg0rwhRXo4TLAI1ql5IgeVDU7QN6z9Fg=
Authentication-Results: ietf.org; dkim=none (message not signed) header.d=none; ietf.org; dmarc=none action=none header.from=workonline.africa;
Received: from DB8P190MB0746.EURP190.PROD.OUTLOOK.COM (2603:10a6:10:12a::24) by DB6P190MB0549.EURP190.PROD.OUTLOOK.COM (2603:10a6:6:3b::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3611.31; Thu, 3 Dec 2020 22:42:20 +0000
Received: from DB8P190MB0746.EURP190.PROD.OUTLOOK.COM ([fe80::cdc3:7ba:4bf7:dcda]) by DB8P190MB0746.EURP190.PROD.OUTLOOK.COM ([fe80::cdc3:7ba:4bf7:dcda%3]) with mapi id 15.20.3632.017; Thu, 3 Dec 2020 22:42:20 +0000
Date: Fri, 4 Dec 2020 00:42:13 +0200
From: Ben Maddison <benm@workonline.africa>
To: sidrops@ietf.org
Message-ID: <20201203224213.gnb2nawujxm7a32q@benm-laptop>
Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ngqh4sdjdhb2t6jz"
Content-Disposition: inline
X-Originating-IP: [160.119.236.50]
X-ClientProxiedBy: JN2P275CA0005.ZAFP275.PROD.OUTLOOK.COM (2603:1086:0:3::17) To DB8P190MB0746.EURP190.PROD.OUTLOOK.COM (2603:10a6:10:12a::24)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from localhost (160.119.236.50) by JN2P275CA0005.ZAFP275.PROD.OUTLOOK.COM (2603:1086:0:3::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3632.17 via Frontend Transport; Thu, 3 Dec 2020 22:42:19 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 0772fcc4-3186-4b5c-4fb7-08d897dcb186
X-MS-TrafficTypeDiagnostic: DB6P190MB0549:
X-Microsoft-Antispam-PRVS: <DB6P190MB0549A07FD127F69E0FF53DA9C0F20@DB6P190MB0549.EURP190.PROD.OUTLOOK.COM>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 02dlD2k2JsCHoPu1bWSxcBduLtlltB6qNxULAWNZoyF5+wbfsuRcNWpaEmn3FWqNCTHM1+xoHjCiCRPSv7pr5EMgrncZrHcp4WPdr617CMZOsKXHuPP8Um7oaN07Zty/B6ImfjloVtm475/lH8luHNy612BEYigGtRIINlAPDlw5Z8p4PS2/kHc1uoAJkmqWN/K7aEfogem2dtEa+6XQ34Suy+bxXr4CBskxWPgZVXH5Kj4VwI3wx3/F+qIINicDWpWoB1xo+J+KMdk9eo7Hws3hYtf/8hF7iGxinCXKy2OJMrXARdd9dmaV1Dsy7CuJr6b2Hw99a9K4iFR5/UIDUJt7Lb23tPejOQKHb8VlTgwLINxGP1CEy7ro+JA/Kz+vmtp9mY3IMTnWEopMPwsbU3grSiu+e993AuvwQ0oP6uD9duYg9qxJXgh03bUL2g1/u6Gl8YhWO6aFp4/vjl3Jj5tIUpOP8npHdwpAj/YeFLarZ1h26DpS+tbOkI8m13PghdE8/P0EjdkRXTWwT44NLA==
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB8P190MB0746.EURP190.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(7916004)(346002)(396003)(376002)(136003)(39830400003)(366004)(6486002)(956004)(86362001)(8676002)(8936002)(83080400002)(44144004)(66476007)(186003)(66946007)(16526019)(66556008)(5660300002)(966005)(83380400001)(26005)(6496006)(478600001)(21480400003)(52116002)(316002)(2906002)(33716001)(6916009)(1076003)(6666004)(9686003)(46492008)(2700100001); DIR:OUT; SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: =?us-ascii?Q?bld1yEiObN69o3/yxbEWt8UmuUMk0jmMqfVz16lJ/JAo0ScSrsQopqEPSW4S?= =?us-ascii?Q?TK07tyecR5BdokyG3anVEOuuTFcggfhzZh1OlaqzqKd99ytNERUp7M/CdWUz?= =?us-ascii?Q?pZjlZtQ5t+y0QXGWquVJO3VbO13ftWy+4J4iSdcz8cMH/F/ZORdlh0qSHqik?= =?us-ascii?Q?0H4MmM01Pa1+hZqJ6r6pY6I1Dbqz+q54i4FJCF/GG5A1xActS9QyuVBi2FMe?= =?us-ascii?Q?my6icQk8mz0GFUB2h7wIOGAHqmM55HgAGTfuwlGcpa1GKjRUC2R8wIZz9g79?= =?us-ascii?Q?2Z/scLb/4oRfYh/GcG/x+w3qfN1uZ/GUsp2GPpjOHwDDACVtuhy2UO1XlfPr?= =?us-ascii?Q?LA5dv6Klj9M29dPIujot+q45Rc4LyRjnIPwagGCdvk2l2zekqcLsTfLgxd2N?= =?us-ascii?Q?nzydL+lr7jtehnn2XDCCFK2sG3IlY+dZWTS6D1224sc3oqVwbTCkeZAGituJ?= =?us-ascii?Q?t55xpHeHPIslaP3b/ZsnuK3+9jrAf/uUDujvbONLDLInyQLwqMNyvRSp2PEt?= =?us-ascii?Q?+SbLEw4L4z9IAD28jQ6P3UxlKUe60eu32l2ASZb1QoePpUhAZ/WQtKRCH6nf?= =?us-ascii?Q?p8Nnp9eaZmBVDLnscWMekaP/alJaWNFaDxs+/zJzwnTu2sOKG9/QidT3EBG2?= =?us-ascii?Q?hO4Qo3IKBc0/Sp45sRYRGuQxJjjGkwPl2Ox4MJ2ItXzf21M7xnyqOiLJCGSi?= =?us-ascii?Q?kRUl7z94xIJWvvaWUcByDyAl1G7Ao9Pef3BuyQL7w3u1cRjF5jQ4dhW83Jbz?= =?us-ascii?Q?ikEylafdiO9yBJ3NJOgkI6NWGeEuBG79QMmzJyv3rTNKzorjghQMyjRLUVsq?= =?us-ascii?Q?jNP7NqKwT/54sDV1Eum2ZMa6X6uSML32ASSS835OP7Mwbu9mjMKQ95BMS1U1?= =?us-ascii?Q?0NjHhmoxZrrP5Px1d7SA/mc/Kx/ebVSMCCEep0bi4FcARN2V3bavoc8ZFb96?= =?us-ascii?Q?QgNxX9S5bmmorMXPcEdVSB5ZR0ymjkX4YH0X3K03U8AgX/baRMu9TTlCJmC0?= =?us-ascii?Q?/DAY?=
X-OriginatorOrg: workonline.africa
X-MS-Exchange-CrossTenant-Network-Message-Id: 0772fcc4-3186-4b5c-4fb7-08d897dcb186
X-MS-Exchange-CrossTenant-AuthSource: DB8P190MB0746.EURP190.PROD.OUTLOOK.COM
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Dec 2020 22:42:20.3856 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: b4e811d5-95e8-453a-b640-0fba8d3b9ef7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7ubzgI4ew6tEr+0tgRO7fzKo/bsAJqcS7i1rNkpFlkmsZQpAeBLVkOcNsrDEBPY0N5YzR3/8iBYqSFlgYWjdLA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6P190MB0549
Archived-At: <https://mailarchive.ietf.org/arch/msg/sidrops/Qz5edfz0jhYp6X8m_ywE3t1XQOk>
Subject: [Sidrops] 6486bis: referenced object validation
X-BeenThere: sidrops@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: A list for the SIDR Operations WG <sidrops.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sidrops>, <mailto:sidrops-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/sidrops/>
List-Post: <mailto:sidrops@ietf.org>
List-Help: <mailto:sidrops-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sidrops>, <mailto:sidrops-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 03 Dec 2020 22:42:34 -0000

Hi all,

As a result of the incident last night that caused the number of VRPs
output by some RPs (certainly routinator, possibly others too) to drop
dramatically, a couple of us have been going over the observed behaviour
in comparison with the current text of 6486-bis.

It appears from Job's analysis [1] that the incident was triggered when
several resource certs that were listed on an APNIC issued manifest
became invalid due to their 'notAfter' time expiring, which in turn
caused the affected RPs to consider the manifest invalid and fail the
fetch.

It should be perfectly legitimate for a manifest to list objects which
will become invalid during the lifetime of the manifest, or which are
not yet valid when the manifest is generated (for pre-staging).
The presence of such objects should not invalidate the manifest.

I think there are at least two places that need an update in light of
this:

Section 6.4 states:
    """
    If there are files listed in
    the manifest that cannot be retrieved from the publication point, or
    if they fail the validity tests specified in [RFC6488], the fetch has
    failed and the RP MUST proceed to Section 6.7
    """

The reference should probably be to both RFC6487 and RFC6488, as the
former provides the validation details for .cer and .crl objects, which
are presumably intended to be included in this step.

Additionally, due to the values in the 'notBefore' and 'notAfter' in a
Resource of EE cert, an object that was legitimately included in a
manifest may become invalid when processed later in the manifests
lifetime by an RP.

I would therefore suggest the following revision:
    """
    ... if they fail the validity tests specified in [RFC6487] and [RFC6488],
    with the exception of the checks against certificate Validity From
    and To values described in Section 7.2 of [RFC6487], the fetch has
    failed ...
    """

Although not directly related to the current problem, the list of
objects in section 6.4 for which the ability to process is mandatory 
should perhaps be reduced to those in general circulation today, and not
include BGPsec certs or ghostbuster records, since those add
approximately zero utility to an operator who wishes to run an RP today,
and are therefore unlikely to be priorities for implementation.

Next, section 6.6 states:
    """
    If a current manifest contains entries for objects that are not
    within the scope of the manifest (Section 6.2), the fetch has failed
    and the RP SHOULD proceed to Section 6.7
    """
I believe that the reference to 6.2 is a typo, and should point to
section 2:
    """
    A manifest associated with a CA's repository publication point
    contains a list of:
    o  the set of (non-expired, non-revoked) certificates issued and
       published by this CA,
    o  the most recent CRL issued by this CA, and
    o  all published signed objects that are verifiable using EE
       certificates [RFC6487] issued by this CA.
    """

The term "non-expired" begs the question of whether a certificate should
be non-expired at the time of manifest generation or RP processing to be
in scope.

On the one hand, insisting that a listed certificate be valid throughout the manifest
lifetime is what gave rise to the failure described above, and is
clearly, IMO, the wrong choice.

On the other, insisting that a certificate be valid at the time of
manifest generation precludes certificates which have not yet reached
their notBefore time. Validating as if "at manifest generation" also
appears to contravene the guidance in section 4.6.1 of RFC6487:
    """
    Relying Parties SHOULD NOT attempt to infer from this
    time information that a certificate was valid at a time in the past,
    or that it will be valid at a time in the future, as the scope of an
    RP's test of validity of a certificate refers specifically to
    validity at the current time.
    """

As above, therefore, I would suggest that rather than choosing either
of the above options, the question of temporal expiry should be left 
out of manifest validation altogether.
Thus point one of section 2 becomes:
    """
    A manifest associated with a CA's repository publication point
    contains a list of:
    o  the set of non-revoked certificates issued and published by
       this CA,
    o  the most recent CRL issued by this CA, and
    o  all published signed objects that are verifiable using EE
       certificates [RFC6487] issued by this CA.

    The list MUST include any objects that have a validity period that
    overlaps the period between the thisUpdate and nextUpdate times of
    the manifest.

    ...
    """

There may be other parts of the text that are similarly in need of
update or clarification on this point: I'll update if I find any.

Hopefully the above makes sense. Please let me know if anything is
unclear.

Cheers,

Ben

[1]: https://lists.nlnetlabs.nl/pipermail/rpki/2020-December/000245.html