RE: How do we handle 4 billion CRH-FIB entries ¿?

Ron Bonica <rbonica@juniper.net> Fri, 22 May 2020 15:37 UTC

Return-Path: <rbonica@juniper.net>
X-Original-To: ipv6@ietfa.amsl.com
Delivered-To: ipv6@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1B05A3A0B7B; Fri, 22 May 2020 08:37:35 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.099
X-Spam-Level:
X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=juniper.net header.b=kUJsi7Li; dkim=pass (1024-bit key) header.d=juniper.net header.b=XVGOnXy4
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nmQnpkbjLT7d; Fri, 22 May 2020 08:37:33 -0700 (PDT)
Received: from mx0b-00273201.pphosted.com (mx0b-00273201.pphosted.com [67.231.152.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A13B73A0C5D; Fri, 22 May 2020 08:37:31 -0700 (PDT)
Received: from pps.filterd (m0108163.ppops.net [127.0.0.1]) by mx0b-00273201.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 04MFbSEA005693; Fri, 22 May 2020 08:37:28 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : mime-version; s=PPS1017; bh=oJgqoLQR5ZN1jSXty0gQj/aLdUvhRpjGLqoxYHhoRSA=; b=kUJsi7LiHfhshbMkA6tBLI2gkEpouev1aDEwfnmKdiDR1FceJzr/udf3nCnspltwjUlE jRuknqfUFrL7AayOqch0bkMPdNgiOH47IDbVlmkV70GKb/dUL+o/agae0kQPiqS6fVuy Ar1zCzhcVHd3si5SYpkkpJu9DHGfVnMTTRrglDAVumB5s/ts0ffnamlmmUUvXZslbDW3 Cq0/iG9L6NDculux+H03Zp9rDPciU2w1/qeKg0dHXfUIu4DcEOPzNYpZkiak1Q7oYvyQ GjWwUojfbjK75xpbtPDcjuDqx3qit62h5/OYOg6T5hP599Z3BOrCXrL+kpoeNmIOPirH 1w==
Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2171.outbound.protection.outlook.com [104.47.55.171]) by mx0b-00273201.pphosted.com with ESMTP id 3165hhh5s1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2020 08:37:28 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LeIG+voMnL9LebdYgZYDSg8dIvFAjHrB+kLhab346Ee1UNkG/IHhcH25HL42c675hCI11vMbcsYpIiIZI5lz3uYxlqof46++BSLi/tvRM4lO5NLQzMkS+E+zlqxTQn8IzDW9+PUjGDHTol+0KXeGsWD8dEHPJwGnpj4sZvkjsfxExMuen2JZ+5NqkDDVKyI6Hp1V3gQfoXNBch8qJgOB9rTtFEbbDmSlm39mlQKtQFSQz0FHpRQeIAK+tW0Pi+qqjrhoOV/XJQw2CWc5MxmD9TIx/RVeGofGncUXJReGiiscQZlVcCXiOie/jA0oiEPLXjbMQfe6rESXdT3GVvyC6g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oJgqoLQR5ZN1jSXty0gQj/aLdUvhRpjGLqoxYHhoRSA=; b=hHEEgKb6RSSF5fArmcE/Cim6/E3FRbI3LJFl3T1HEjX0lkxmdnBlTr45h7NEPiYiRDZMk4NHZAVHLyzK4hFuALxV7vB57A6JV3RHz2PyoNh0fslBHIaA8ecsRFhcW5FL86ceUfY7csvX9ZmM7oaPPC3hTQSNBulUagaqWUIzNfEKJX0UDSDYKGtRfobG3Rat5r21hUeasUMGgJTZDclRtpIaTq4BX3Det2wFjBTNPMPz+kEa9vz5OV89VN9TZgcFe2n07Vpwzv71arLKbzFFgvJGSeW4WeojYsiVHJgsp54YCiUDdWsEeU6kY5YK6kofCdpKNoynafEH09vRRMpXoA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=juniper.net; dmarc=pass action=none header.from=juniper.net; dkim=pass header.d=juniper.net; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oJgqoLQR5ZN1jSXty0gQj/aLdUvhRpjGLqoxYHhoRSA=; b=XVGOnXy4J9uZi7vs1xvBH7NIA6iB+CXTXus6QxnZFJgSAqezQ1JD3TAQrHf1vqzw9Qfms1fpZgWMex7W+vhaTi9tGkZdUvVhTMh+XJJH4IpUjyVvxhtg4XXo/AF5WDbShyxIcScJDPh/Q9p2TLnO4yWAZZEPlLQEz69fkVZKcMU=
Received: from DM6PR05MB6348.namprd05.prod.outlook.com (2603:10b6:5:122::15) by DM6PR05MB5516.namprd05.prod.outlook.com (2603:10b6:5:b::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3021.11; Fri, 22 May 2020 15:37:25 +0000
Received: from DM6PR05MB6348.namprd05.prod.outlook.com ([fe80::c020:3bf5:7230:75e3]) by DM6PR05MB6348.namprd05.prod.outlook.com ([fe80::c020:3bf5:7230:75e3%4]) with mapi id 15.20.3021.019; Fri, 22 May 2020 15:37:25 +0000
From: Ron Bonica <rbonica@juniper.net>
To: "Pablo Camarillo (pcamaril)" <pcamaril@cisco.com>, "draft-bonica-6man-comp-rtg-hdr@ietf.org" <draft-bonica-6man-comp-rtg-hdr@ietf.org>
CC: "6man@ietf.org" <6man@ietf.org>, Ole Troan <otroan@employees.org>, Robert Raszuk <robert@raszuk.net>
Subject: RE: How do we handle 4 billion CRH-FIB entries ¿?
Thread-Topic: How do we handle 4 billion CRH-FIB entries ¿?
Thread-Index: AdYwJjh/R/k43WV0RIKSb2XOux8mcwAIpz9Q
Date: Fri, 22 May 2020 15:37:24 +0000
Message-ID: <DM6PR05MB63486A07EEBE3F043D600D2BAEB40@DM6PR05MB6348.namprd05.prod.outlook.com>
References: <MWHPR11MB13748B1068560BD2878C89CBC9B40@MWHPR11MB1374.namprd11.prod.outlook.com>
In-Reply-To: <MWHPR11MB13748B1068560BD2878C89CBC9B40@MWHPR11MB1374.namprd11.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels: MSIP_Label_0633b888-ae0d-4341-a75f-06e04137d755_Enabled=true; MSIP_Label_0633b888-ae0d-4341-a75f-06e04137d755_SetDate=2020-05-22T15:37:22Z; MSIP_Label_0633b888-ae0d-4341-a75f-06e04137d755_Method=Standard; MSIP_Label_0633b888-ae0d-4341-a75f-06e04137d755_Name=0633b888-ae0d-4341-a75f-06e04137d755; MSIP_Label_0633b888-ae0d-4341-a75f-06e04137d755_SiteId=bea78b3c-4cdb-4130-854a-1d193232e5f4; MSIP_Label_0633b888-ae0d-4341-a75f-06e04137d755_ActionId=911ae312-2407-43c2-9d56-bd5c35c2a3d7; MSIP_Label_0633b888-ae0d-4341-a75f-06e04137d755_ContentBits=2
dlp-product: dlpe-windows
dlp-version: 11.4.0.45
dlp-reaction: no-action
authentication-results: cisco.com; dkim=none (message not signed) header.d=none;cisco.com; dmarc=none action=none header.from=juniper.net;
x-originating-ip: [108.28.233.91]
x-ms-publictraffictype: Email
x-ms-office365-filtering-ht: Tenant
x-ms-office365-filtering-correlation-id: 355410f2-f840-42c3-0c82-08d7fe6606c8
x-ms-traffictypediagnostic: DM6PR05MB5516:
x-microsoft-antispam-prvs: <DM6PR05MB551643ACF1FAA824AAE5BE07AEB40@DM6PR05MB5516.namprd05.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-forefront-prvs: 04111BAC64
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: T1D6sxxhAsvcq9yEnI0NFxVubYSCjGEUF4lfr9xyEBOBykvasly9LCt68ockANFRZiFNyx4NaWwP8+pJ+S9OP5RYdv7heud9ethuJVdjdop2f1Ghlg9O/gSSwUCEtw6TbeysryTSJjUhJtDXVnBRv15M4jDx+56167Q6uE9ZUPDCkvfuZ0JfNecem/ICOWmPFiI/luPkUpuwCCWSoDQt2atQvkDdbktM30JYgj8uMKovfk3kKpnUqhuohmTmvvv5E2AaJuVtLa/pxFCBBC5c5LTeQdKjfRREJvBky0yI3IGPd0nbxsmwHm5KeuCOqkNA8A94Q8fnUVzrRv+/bNo85jFzkIuf+/XBkABz6K06sDQOI1mOrEm/JDglXkB9g1hg9bGR10YjquKTBgFLR5y5/A==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR05MB6348.namprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(346002)(396003)(366004)(39860400002)(376002)(2906002)(66446008)(6506007)(66476007)(53546011)(66946007)(316002)(76116006)(64756008)(66556008)(71200400001)(5660300002)(7696005)(66574014)(110136005)(4326008)(55016002)(52536014)(166002)(478600001)(86362001)(8936002)(186003)(9686003)(54906003)(33656002)(26005); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata: MXNNiIoqNisWvfrwpZr6Be6p+roVrYMLP/tTaJTMhrqM67TDCUKf8N7zNT+pVvnrMgb+eNk9NHL+eZKBdc8sDwNT8gIjLCP3eSPEVA40YOiTOaNRjBYMmFChhKt7P66wO+Bv3tFfK9oZOKgKdx2HW7sIg2Om93MjXuIjRgcUmn9y/XreC5/qhm5RxrhiS8lX3rnC8AHLZ3/xEcs5jEGhPWTfh/oNMHN0Tn1bvLQgzDrcIcMw4SMBcmcxKHqCv1e/7yEMGRWTDnU5V/XjYpiYR3VQ3ZeXuJAQ1gSLOHDUbZ2X28wc1zG1BRvCkunTeJ7diOEP8hVZn1hqh5Mf3DSuvzBk90YrQBW/7FI1gUHR2CtdHM/jsLGo1haZoQw5QiJd0OSqDXMU9XZEn2WbFpBoAehshMlwgnKuF/G/VNtYkX0UvtRfTUErEtjoGx4F5cTPEN6ULRVAVyuM7z+XUTwYBWo7zzYZfp9aUYh/oyv9ZjKcS4hksbs266ulsl/2nitS
x-ms-exchange-transport-forked: True
Content-Type: multipart/alternative; boundary="_000_DM6PR05MB63486A07EEBE3F043D600D2BAEB40DM6PR05MB6348namp_"
MIME-Version: 1.0
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-Network-Message-Id: 355410f2-f840-42c3-0c82-08d7fe6606c8
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 May 2020 15:37:24.9028 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 4FI2li3twcv+a6gadelLN5QC+S/iUkZm46U1795G1MH4iWMJoYK37S6V+lB6Zd5ajvHieSAHS42kaIJDhTeEzw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR05MB5516
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.676 definitions=2020-05-22_05:2020-05-22, 2020-05-22 signatures=0
X-Proofpoint-Spam-Details: rule=outbound_spam_notspam policy=outbound_spam score=0 lowpriorityscore=0 malwarescore=0 mlxscore=0 bulkscore=0 adultscore=0 impostorscore=0 phishscore=0 suspectscore=0 cotscore=-2147483648 spamscore=0 priorityscore=1501 clxscore=1011 mlxlogscore=904 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2005220126
Archived-At: <https://mailarchive.ietf.org/arch/msg/ipv6/1i-usK1lJ0yQY5B73PTnzgF6cAs>
X-BeenThere: ipv6@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "IPv6 Maintenance Working Group \(6man\)" <ipv6.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ipv6>, <mailto:ipv6-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ipv6/>
List-Post: <mailto:ipv6@ietf.org>
List-Help: <mailto:ipv6-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipv6>, <mailto:ipv6-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 22 May 2020 15:37:35 -0000

Pablo,

The scope of the current draft is limited to CRH forward plane behavior and CRH-FIB scalability. Your question concerns the scalability of a path computation module. While your question is legitimate, it is beyond the scope of this document and the charter of this working group.

Nonetheless, I will try to answer it. Section 7 of this document says that although identifiers have node local significance, they can be assigned as if they had domain-wide significance. In my last email to Robert, you will see that I allocated loosely routed identifiers as if they had node local significance.

In many cases, this strategy is beneficial and facilitates scaling. You will see it in the IS-IS extensions for CRH <https://datatracker.ietf.org/doc/draft-bonica-lsr-crh-isis-extensions/> document. When this strategy is used, control planes that support CRH have roughly the same scaling characteristics as control planes that support SR-MPLS.

                                                             Ron






Juniper Business Use Only
From: Pablo Camarillo (pcamaril) <pcamaril@cisco.com>
Sent: Friday, May 22, 2020 7:00 AM
To: Ron Bonica <rbonica@juniper.net>; draft-bonica-6man-comp-rtg-hdr@ietf.org
Cc: 6man@ietf.org; Ole Troan <otroan@employees.org>; Robert Raszuk <robert@raszuk.net>
Subject: How do we handle 4 billion CRH-FIB entries ¿?

[External Email. Be cautious of content]

Hi Ron, authors,

I've been reading the example you provided to Robert/Ole, and there's something concerning to me:

> Consider the following network:
>
>      -    Contains 65,000 routers
>      -    Each router has 500 directly connected neighbors or fewer
>      -    Uses 16-bit CRH
>
>In this network, each node might have 65,499 CRH-FIB entries:
>
>     -     64,999 CRH-FIB entries cause packets to follow the least-cost path to another node in the domain
>     -     500 CRH-FIB entries cause packets to traverse a specific link to a specific neighbor.

In addition, you've said several times that:
> Now recall that identifiers have node local significance

In summary:
You have 65000 routers in your network, and each router has 65499 "node-local" CRH-FIB entries.
In other words, this network has in total 4257435000 node-local CRH-FIB entries.

1.- How does it scale for an operator to configure on CLI more than 4 billion node-local CRH-FIB entries for the entire network?
2.- Assuming that some day in the future you would like to advertise this in the IGP: how is it going to deal with 4 billion node-local entries?

Also, you said:
> Somewhere in the network, there is an entity constructs the CRH and the list that it contains. That entity needs access to another data structure, that includes a global view of each node's CRH-FIB.

So, this entity needs a data structure with 4 billion node-local entries, and I guess some path computation algorithms with that scale...

Note that as per your explanation, we could configure the same CRH-FIB entry/tag on several nodes. However this changes nothing: as long as entries are defined as node-local you still have to deal with 4 billion.

In my opinion this is a fatal scalability problem.

Thanks,
Pablo.

PS. Those numbers are only assuming 65.000 routers; however as part of the 16b vs 32b the authors have stated several times the need to support more routers...