Re: [Idr] some questions from {RC, LC, EC} analysis presentation in GROW

"Sriram, Kotikalapudi (Fed)" <kotikalapudi.sriram@nist.gov> Thu, 12 August 2021 17:07 UTC

Return-Path: <kotikalapudi.sriram@nist.gov>
X-Original-To: idr@ietfa.amsl.com
Delivered-To: idr@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ECD813A431B; Thu, 12 Aug 2021 10:07:22 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.165
X-Spam-Level:
X-Spam-Status: No, score=-3.165 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.452, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_GOV_DKIM_AU=-0.612, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=nist.gov
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XM4GT8jvLQ1s; Thu, 12 Aug 2021 10:07:17 -0700 (PDT)
Received: from GCC02-DM3-obe.outbound.protection.outlook.com (mail-dm3gcc02on2109.outbound.protection.outlook.com [40.107.91.109]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 042793A4317; Thu, 12 Aug 2021 10:07:16 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WW9kqUhMptq0WHai9M/DQqx85Nszo2mmnCeQ5FzaAnfQMNyW8SgJESHtem+GCZ+qKoDPQpHafeOKjmEl+GkeovAg+zY2ruvz0TwPcY1rO2lPvj5nDxi7BW08ypNkq4TIuecrI6mRCBlkIdqggDAF4Zc0lv4NWH+vD7hvEBIW5F4FcF++d6ueHn8zCCBV0Yi6hoCVlSaADefaa9wx8qJD1OHdjAG6UHsgXMwHLPs7dw3u0i7C4WEj74YfN7lqcXsWHT9d3GFay10PCdzlt/JPMS6SvcGG3t15L+9aa5xYF6p9caW30pTRsJ0y3e8+NC4z8hctiWL/g0FQtU1AsoWjSg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fB/selKe8VfkW966sFrMuXLSaIPdDR21PJ9UrTt37Ns=; b=Ca8A4V+eHSo2SpXoF++71I5pkPyhul9neekc3qgzN6HnbPsEYuyaXPPBsKQ7LrJU2KH+damQrdTAC2fDhRd/BIdgimDZz/l7AccxIDLUIaZd3B0ElXpkgunGTbWBEKMnxQdmr6fQ8W0S8bvCm0dAcS1zMkEgQ0uQv/8KKQ8o4CJR9rZqHDjW2MFI8VdKqVS9m0uVbZwX2m3ntOTNL6uQogvaP2caaKE8snCKMhvKTEdiXhMmDpY0MhbqkBCu7LO/qMIlbyY1yx+JY9zi3kwBZolS0866Js1vSuHAFMhVB8UcdJSpL5HgclACo2JhBwOjPuqagaTJvYCKHdrxBlMGCg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nist.gov; dmarc=pass action=none header.from=nist.gov; dkim=pass header.d=nist.gov; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nist.gov; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fB/selKe8VfkW966sFrMuXLSaIPdDR21PJ9UrTt37Ns=; b=UH6vG33yYppYvEVHS6pgdY4l97rPYnSzhZNx0FjKJnZqjQPVEBX7pNQZvgGp0tYB7ecPGm+LNZ3VBFcoxemNIE25ybCwl/5LVPj1lxUy83Fp8XFfeh1PBlQoiBs+DLsFgxYtTWMpRKZwukrpue77unkrTC08IpgS/kFKdYYY0BE=
Received: from SA1PR09MB8142.namprd09.prod.outlook.com (2603:10b6:806:171::8) by SA9PR09MB5389.namprd09.prod.outlook.com (2603:10b6:806:45::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.17; Thu, 12 Aug 2021 17:07:14 +0000
Received: from SA1PR09MB8142.namprd09.prod.outlook.com ([fe80::153f:4e8c:eadd:935d]) by SA1PR09MB8142.namprd09.prod.outlook.com ([fe80::153f:4e8c:eadd:935d%7]) with mapi id 15.20.4415.017; Thu, 12 Aug 2021 17:07:14 +0000
From: "Sriram, Kotikalapudi (Fed)" <kotikalapudi.sriram@nist.gov>
To: Zhuangshunwan <zhuangshunwan@huawei.com>
CC: GROW WG <grow@ietf.org>, IDR <idr@ietf.org>
Thread-Topic: some questions from {RC, LC, EC} analysis presentation in GROW
Thread-Index: AQHXiHXG0T0iPqYYLk6NCjnOwJ+5gatinG8AgADwG9OAB9cbw4AAmMdAgAQk2wk=
Date: Thu, 12 Aug 2021 17:07:14 +0000
Message-ID: <SA1PR09MB81420314611474AD2471C0B284F99@SA1PR09MB8142.namprd09.prod.outlook.com>
References: <SA1PR09MB8142ADE02512DB13887086AC84F09@SA1PR09MB8142.namprd09.prod.outlook.com>, <76c169816a174f4c8907af0e8b64b932@huawei.com>, <SA1PR09MB8142D8366448EDD90909FDEC84F19@SA1PR09MB8142.namprd09.prod.outlook.com> <SA1PR09MB8142699ECB6700439DC4D32A84F69@SA1PR09MB8142.namprd09.prod.outlook.com> <a618abaf2b1f41419aabd03c8b16aa20@huawei.com>
In-Reply-To: <a618abaf2b1f41419aabd03c8b16aa20@huawei.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: huawei.com; dkim=none (message not signed) header.d=none;huawei.com; dmarc=none action=none header.from=nist.gov;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 73132e95-e7dc-461e-139b-08d95db3a1a6
x-ms-traffictypediagnostic: SA9PR09MB5389:
x-microsoft-antispam-prvs: <SA9PR09MB538906077A16FBFDC0CCEC5384F99@SA9PR09MB5389.namprd09.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: YWiRP23EKbGGMOMFmwtYjWTZK9lTSPXTb3VZdIMNMJFAXaEGM9hR1FcO8zb6l58u6SO2kiVNnEXHxK7GKd8qNIWDObrDriER3efnjIzzkSjvYfFCNRbqZHdFDYmYSE+acwO2uuxN/yzqdZP6uI0RS7p/pdGZZu3j5bJsXShxRSHGn82D1hV7pC8Pqn78rln7BA2nVsGgk9PXfDkqUEo8XsGVlj1ucrOELCSoKcdEgZ1jzeTCMjEJYVtCebJFoXKBODxZRemv5N4keqUWZBlx+tqbDnlvwcQ22GZl6yFpRaL5YFifKC8aS51iEbWqvRiVL1uhuAZcTvWWvPVt777VvTZkdv1utztREgjR57Qd7mRo++faUH7U2tNOhNRJ0EPusyQN41EdO4NPq9pTkvCQHw3W73wrya6swUJZBhuBK0ot+Gfx6ohK+W+5hgfa2qoS3DFMEZWZZ+yWbWZZntJOlDSBjvnAfKkqBfJramEWYnNJQkefgP0B5Yr+wnEYWRi8ogUX3LLr65tih4ntWJoxNmHabL4Rj+orEq/+9w0Ejs6DHykHxaJP01PvzshdA0BY+DZnBcOuFuZK+MS7jazca9Zg9417LkJvThgXdJpflk45Cu6+qivc/mOZNfPfG5NSeKUkhUyu2hj2jG3d7+HFDWU/TPAt6dwECPp29pwljkx9GcpXxKmBCj3mYolNob67kFoM7w8dB69AYJsavcgkPw==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SA1PR09MB8142.namprd09.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(39850400004)(376002)(346002)(136003)(366004)(54906003)(9686003)(76116006)(122000001)(91956017)(8936002)(316002)(186003)(55016002)(4326008)(66446008)(64756008)(66556008)(66476007)(66946007)(86362001)(8676002)(5660300002)(2906002)(26005)(7696005)(52536014)(6506007)(478600001)(6916009)(38070700005)(71200400001)(33656002)(66574015)(83380400001)(38100700002); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 0yxz3WDHJoUppLbIrhwHVYAR750w99jetHFevXa/YK+LJ8yxCzYgE5V4bdtljK7ESDWB+s0rd1KRFZanFK4CCqhsHJ7BES/kDKUKX/SMGp4ZTWbJtZ7CpINpdcRZOy44C9NEY+j28uMrA/WtL2aTRApuq4ugDsjxg/PqvzdkAVHw2bfqOaSt9/3EByQLA0SEbCWxJZjJr44xFVvgUQ13uA0OpKlzw80Gku/I9kG4pyHxwsZkPVbh/Beybw1b45O3fbbrrwP0L0FAtLfXNEpMFE24s7b9/clUX0/+7CazVPb5wtFKO36VBN9zkhCtTk3jXeRRuJLRIUNDoEwd2+HBBsk4Xj19nTZHCYxqb7pFnHdzQoXftZW7mNKYOiCEq5Cj2CiO45JTqVxeQf5KkJcw8sScXdJT0/eSRjLkfq7PhqATMiCj6NtOQ/SifawReM7lgW8cxPwpixs3FrjDRaQvuwMz4nm66FzEWm8aBeG2yUa5fwLgerzvlo1/y1mrAe0nRyyfydZCzJaCh2/XLlm/8lKE+NM6r0f93sv71Izf5bQEviCrQ3p8V8T6Vq7BKTZ9iNA+CD+SA/ZcDRS7QBG6y/8LWtPhhl1rnsdlnhBxggxySbFjCwk7HMWgw/44xdAIqTLtzC98u5o6skQPhx7X0+A/7n902t4dSHtMSbmVZ02p7ugg48pZXi5hBPvqpaO4bLM0pZFwsCpWnt9y7yBC/0+cPJkTHbYp0afL3xFgQZ0fvoRzTxkDjOXfRfzVXaBaGOaw4KdKb45Koxor0LKE+CLSk3zEa16h1u6K1RUgccmUATNL+byfLvxjQLzCUJmoRHr601qJGLYol4UOfEUKBY2WxZbRLwV9KDQjkGY1xeMHVzEebEPy4O8TSYkSD4WdWYC9bN4EdQP8VTduDqD8KZmsI1u/CGEOigpw3sIx4VuXafTehzsdg9OSuG28D7w91ME70vO46PBmgleNXnJ9lm/hZXWEQP+qcmYGittNICXEG+RaXICJwTkQiulwE0AVDSQhCOjY1cDE5xXvotr7uQBYYIqNdk88yWE8XHyE2LtAA3M+m6UQqDW63SY9J3mc8ngWCoauoQHnXk8+6eLmp7TIIstcEA+r1MaEksgs+U9cTJUIQSQP81AFDl0Xqbst99/GDKJUQofY5OlGH0xEzLQfQKdRTrk2SXjkdlI4dxk3zZGtlHHp+YHFGbAs7DUt1rLT3036kD1KpAFTjt5QDfG6PV0bIxKQD1TEdJxLjmKzddyAibief8DsFy7ENhuH2+4aax1X1qfk1RHeJXjLlz/VohmGqvC0Ikgp+1j5kbY=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nist.gov
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SA1PR09MB8142.namprd09.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 73132e95-e7dc-461e-139b-08d95db3a1a6
X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Aug 2021 17:07:14.0876 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 2ab5d82f-d8fa-4797-a93e-054655c61dec
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA9PR09MB5389
Archived-At: <https://mailarchive.ietf.org/arch/msg/idr/S-3qcQClLN5ekJ2N69cPt09wcOU>
Subject: Re: [Idr] some questions from {RC, LC, EC} analysis presentation in GROW
X-BeenThere: idr@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Inter-Domain Routing <idr.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/idr>, <mailto:idr-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/idr/>
List-Post: <mailto:idr@ietf.org>
List-Help: <mailto:idr-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/idr>, <mailto:idr-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Aug 2021 17:07:23 -0000

Hi Shunwan,

>Thanks for your great job! Your work has given me a very in-depth understanding of the propagation behavior of BGP community attributes on the Internet.

Glad to hear that. I share the compliments with my colleague Lilia Hannachi.

>Regarding " Total # Unique {Prefix, RC = 3356:9999} ; 28", why is the number only 28? It may be that the mask of black hole routes is usually greater than 24 (for IPv4 prefixes), preventing such routes from spreading widely on the Internet?

The routes with Blackhole community 3356:9999 or (more generally) ASN:666 (where ASN is not 3356, 5511, or 2603) should be short-lived. The AS providing the corresponding RTBH service should clean up those Blackhole routes from the RIBs after the DDoS mitigation is done. See additional explanations below.
  
>If the answer to the above question is "yes", then if other communities "ASN:666" are widespread in the wild, then such "ASN:666" may not be a black hole community attribute too? As far as I know, the other two examples are 263:666 and 5511:666.

Since you mentioned that 5511 and 2603 also do not use ASN:666 for Blackhole, we were able to confirm the same and measured the following:

RIB data (RouteViews3, 2021-07-15.0000):
# Unique {Prefix, RC = 65535:666} = 221
# Unique {Prefix, RC = 3356:666} = 509900
# Unique {Prefix, RC = 5511:666} = 15157
# Unique {Prefix, RC = 2603:666} = 0  (this zero is based on Routeviews3 RIB, 
      but we do see a substantial # 2603:666 in RIPE-RIS BGP Updates 
      since AS 2603 is located in Europe!)  
# Unique {Prefix, RC = ASN:666} where ASN is NOT equal to 3356, 2603, or 5511 = 4638

So, when we eliminate prefixes with 3356:666, 5511:666, or 2603:666, the remaining prefixes with ASN:666 (presumed Blackhole) are much fewer ( = 4638). This is a good thing. Not too many Blackhole ASN:666 should be seen propagating on the Internet because of three reasons: (1) They should propagate typically only one or two hops and then they should be prevented from propagating further by the corresponding AS providing RTBH service; (2) (as you said) they also do not propagate because often their route mask (prefix length) is greater than 24 (IPv4) or 48 (IPv6); and (3) the AS providing the RTBH service should clean up the Blackhole communities from its RIBs after the DDoS attack is mitigated. So, at any given time there should not be too many routes with Blackhole communities in the RIB.         

As the above data shows that after eliminating just the three ASNs that you pointed out the remaining presumed Blackhole ASN:666 are already much fewer. 

I think you’ll find the following measurements about observed prefix lengths interesting as well:

Frequency distribution of IPv4 prefix lengths in the set of Unique {Prefix, RC = ASN:666} where ASN is NOT equal to 3356, 2603, or 5511: 

12 ; 2
14 ; 8
15 ; 5
16 ; 40
17 ; 12
18 ; 9
19 ; 34
20 ; 58
21 ; 80
22 ; 262
23 ; 275
24 ; 2185
30 ; 4
32 ; 1641

Most of the mass is at /24 and /32 (in the above), possibly indicative of genuine use as ASN:666 Blackhole communities.

Frequency distribution of IPv6 prefix lengths in the set of Unique {Prefix, RC = ASN:666} where ASN is NOT equal to 3356, 2603, or 5511 : 

25 ; 1
32 ; 7
36 ; 1
44 ; 1
48 ; 12
128 ; 1

In the above IPv4/IPv6 distribution data, some prefixes with large prefix lengths made it to the collector, but most such prefixes were likely not propagated (correctly so). 

Please let me know if you find other ASNs for which ASN:666 is not Blackhole. Thanks.

Sriram