Re: [Idr] some questions from {RC, LC, EC} analysis presentation in GROW

Zhuangshunwan <zhuangshunwan@huawei.com> Fri, 20 August 2021 09:44 UTC

Return-Path: <zhuangshunwan@huawei.com>
X-Original-To: idr@ietfa.amsl.com
Delivered-To: idr@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6FE083A1C07; Fri, 20 Aug 2021 02:44:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Level:
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A2ll7EDce7mr; Fri, 20 Aug 2021 02:44:53 -0700 (PDT)
Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A42D73A1CB1; Fri, 20 Aug 2021 02:44:53 -0700 (PDT)
Received: from fraeml740-chm.china.huawei.com (unknown [172.18.147.207]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4GrcCP16QZz6D9lx; Fri, 20 Aug 2021 17:43:45 +0800 (CST)
Received: from kwepeml500003.china.huawei.com (7.221.188.182) by fraeml740-chm.china.huawei.com (10.206.15.221) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.8; Fri, 20 Aug 2021 11:44:48 +0200
Received: from kwepeml500004.china.huawei.com (7.221.188.141) by kwepeml500003.china.huawei.com (7.221.188.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 20 Aug 2021 17:44:47 +0800
Received: from kwepeml500004.china.huawei.com ([7.221.188.141]) by kwepeml500004.china.huawei.com ([7.221.188.141]) with mapi id 15.01.2176.012; Fri, 20 Aug 2021 17:44:47 +0800
From: Zhuangshunwan <zhuangshunwan@huawei.com>
To: "Sriram, Kotikalapudi (Fed)" <kotikalapudi.sriram@nist.gov>
CC: GROW WG <grow@ietf.org>, IDR <idr@ietf.org>
Thread-Topic: some questions from {RC, LC, EC} analysis presentation in GROW
Thread-Index: AQHXiHXG0T0iPqYYLk6NCjnOwJ+5gatinG8AgADwG9OAB9cbw4AAmMdAgAQk2wmADA3CcA==
Date: Fri, 20 Aug 2021 09:44:46 +0000
Message-ID: <6fd70c5bf464418e945a1c17aecd63a2@huawei.com>
References: <SA1PR09MB8142ADE02512DB13887086AC84F09@SA1PR09MB8142.namprd09.prod.outlook.com>, <76c169816a174f4c8907af0e8b64b932@huawei.com>, <SA1PR09MB8142D8366448EDD90909FDEC84F19@SA1PR09MB8142.namprd09.prod.outlook.com> <SA1PR09MB8142699ECB6700439DC4D32A84F69@SA1PR09MB8142.namprd09.prod.outlook.com> <a618abaf2b1f41419aabd03c8b16aa20@huawei.com> <SA1PR09MB81420314611474AD2471C0B284F99@SA1PR09MB8142.namprd09.prod.outlook.com>
In-Reply-To: <SA1PR09MB81420314611474AD2471C0B284F99@SA1PR09MB8142.namprd09.prod.outlook.com>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.108.152.178]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-CFilter-Loop: Reflected
Archived-At: <https://mailarchive.ietf.org/arch/msg/idr/DHWbQULavQTlhtorA_hMCSTYBHg>
Subject: Re: [Idr] some questions from {RC, LC, EC} analysis presentation in GROW
X-BeenThere: idr@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Inter-Domain Routing <idr.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/idr>, <mailto:idr-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/idr/>
List-Post: <mailto:idr@ietf.org>
List-Help: <mailto:idr-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/idr>, <mailto:idr-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 20 Aug 2021 09:45:00 -0000

Hi Sriram,

Thank you so much for the information, it was very helpful and interesting!
As a fan of the BGP protocol, I particularly like seeing Internet routes carry various community attribute information as they propagate, which gives us the opportunity to see some of the details of the actual operation of the Internet rather than just a big black box. In particular, AS3356 carries various rich community attributes. (Some identify business relationships, some identify geographic information, and so on), It gives us the opportunity to learn about the mysterious Internet..
For security reasons, many ISPs delete received community attributes at their ingress border and then tag their own community attributes, which are deleted at the egress border of that ISP, and these ISPs seem to be a tight black box.
By default, some BGP software does not send community attributes to its neighbors. Instead, it needs to be explicitly enabled a knob before sending community attributes. In addition, many software inherits the community attribute behavior when implementing Large Community.  As a result, contrary to our expectations, large communities may not be widely spread on the Internet.

Looking forward to more interesting output from your research work!

Regards,
Shunwan

-----Original Message-----
From: Sriram, Kotikalapudi (Fed) [mailto:kotikalapudi.sriram@nist.gov] 
Sent: Friday, August 13, 2021 1:07 AM
To: Zhuangshunwan <zhuangshunwan@huawei.com>
Cc: GROW WG <grow@ietf.org>; IDR <idr@ietf.org>
Subject: Re: some questions from {RC, LC, EC} analysis presentation in GROW

Hi Shunwan,

>Thanks for your great job! Your work has given me a very in-depth understanding of the propagation behavior of BGP community attributes on the Internet.

Glad to hear that. I share the compliments with my colleague Lilia Hannachi.

>Regarding " Total # Unique {Prefix, RC = 3356:9999} ; 28", why is the number only 28? It may be that the mask of black hole routes is usually greater than 24 (for IPv4 prefixes), preventing such routes from spreading widely on the Internet?

The routes with Blackhole community 3356:9999 or (more generally) ASN:666 (where ASN is not 3356, 5511, or 2603) should be short-lived. The AS providing the corresponding RTBH service should clean up those Blackhole routes from the RIBs after the DDoS mitigation is done. See additional explanations below.
  
>If the answer to the above question is "yes", then if other communities "ASN:666" are widespread in the wild, then such "ASN:666" may not be a black hole community attribute too? As far as I know, the other two examples are 263:666 and 5511:666.

Since you mentioned that 5511 and 2603 also do not use ASN:666 for Blackhole, we were able to confirm the same and measured the following:

RIB data (RouteViews3, 2021-07-15.0000):
# Unique {Prefix, RC = 65535:666} = 221
# Unique {Prefix, RC = 3356:666} = 509900 # Unique {Prefix, RC = 5511:666} = 15157 # Unique {Prefix, RC = 2603:666} = 0  (this zero is based on Routeviews3 RIB, 
      but we do see a substantial # 2603:666 in RIPE-RIS BGP Updates 
      since AS 2603 is located in Europe!) # Unique {Prefix, RC = ASN:666} where ASN is NOT equal to 3356, 2603, or 5511 = 4638

So, when we eliminate prefixes with 3356:666, 5511:666, or 2603:666, the remaining prefixes with ASN:666 (presumed Blackhole) are much fewer ( = 4638). This is a good thing. Not too many Blackhole ASN:666 should be seen propagating on the Internet because of three reasons: (1) They should propagate typically only one or two hops and then they should be prevented from propagating further by the corresponding AS providing RTBH service; (2) (as you said) they also do not propagate because often their route mask (prefix length) is greater than 24 (IPv4) or 48 (IPv6); and (3) the AS providing the RTBH service should clean up the Blackhole communities from its RIBs after the DDoS attack is mitigated. So, at any given time there should not be too many routes with Blackhole communities in the RIB.         

As the above data shows that after eliminating just the three ASNs that you pointed out the remaining presumed Blackhole ASN:666 are already much fewer. 

I think you'll find the following measurements about observed prefix lengths interesting as well:

Frequency distribution of IPv4 prefix lengths in the set of Unique {Prefix, RC = ASN:666} where ASN is NOT equal to 3356, 2603, or 5511: 

12 ; 2
14 ; 8
15 ; 5
16 ; 40
17 ; 12
18 ; 9
19 ; 34
20 ; 58
21 ; 80
22 ; 262
23 ; 275
24 ; 2185
30 ; 4
32 ; 1641

Most of the mass is at /24 and /32 (in the above), possibly indicative of genuine use as ASN:666 Blackhole communities.

Frequency distribution of IPv6 prefix lengths in the set of Unique {Prefix, RC = ASN:666} where ASN is NOT equal to 3356, 2603, or 5511 : 

25 ; 1
32 ; 7
36 ; 1
44 ; 1
48 ; 12
128 ; 1

In the above IPv4/IPv6 distribution data, some prefixes with large prefix lengths made it to the collector, but most such prefixes were likely not propagated (correctly so). 

Please let me know if you find other ASNs for which ASN:666 is not Blackhole. Thanks.

Sriram