Re: [RTG-DIR] [Idr] Rtgdir early review of draft-templin-atn-bgp-07

"Templin (US), Fred L" <> Wed, 25 July 2018 22:22 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 0E9CF130E4B; Wed, 25 Jul 2018 15:22:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -4.201
X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id aXJfhbxfZSRd; Wed, 25 Jul 2018 15:22:22 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 9827B130E0D; Wed, 25 Jul 2018 15:22:22 -0700 (PDT)
Received: from localhost (localhost []) by (8.14.4/8.14.4/DOWNSTREAM_MBSOUT) with SMTP id w6PMMLxG038333; Wed, 25 Jul 2018 15:22:21 -0700
Received: from ( []) by (8.14.4/8.14.4/UPSTREAM_MBSOUT) with ESMTP id w6PMMKV1038317 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=OK); Wed, 25 Jul 2018 15:22:20 -0700
Received: from (2002:8988:eede::8988:eede) by (2002:8988:efdd::8988:efdd) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 25 Jul 2018 15:22:19 -0700
Received: from ([]) by ([]) with mapi id 15.00.1367.000; Wed, 25 Jul 2018 15:22:19 -0700
From: "Templin (US), Fred L" <>
To: Stewart Bryant <>, "" <>
CC: "" <>, "" <>, "" <>, "" <>, "" <>
Thread-Topic: [Idr] Rtgdir early review of draft-templin-atn-bgp-07
Thread-Index: AQHUIz7jU5XGWh1/CkCJaib5rfbbEaSev/HwgAFebICAAFzP0A==
Date: Wed, 25 Jul 2018 22:22:19 +0000
Message-ID: <>
References: <> <> <>
In-Reply-To: <>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: []
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
Archived-At: <>
Subject: Re: [RTG-DIR] [Idr] Rtgdir early review of draft-templin-atn-bgp-07
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Routing Area Directorate <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 25 Jul 2018 22:22:26 -0000

Hello Stewart,

See below for further follow-up:

Thanks - Fred

> -----Original Message-----
> From: Stewart Bryant []
> Sent: Wednesday, July 25, 2018 2:14 AM
> To: Templin (US), Fred L <>;
> Cc:;;;;
> Subject: Re: [Idr] Rtgdir early review of draft-templin-atn-bgp-07
> On 24/07/2018 20:53, Templin (US), Fred L wrote:
> > Hello Stewart,
> >
> > Thank you for your effort in reviewing the document. Please see below for
> > follow-up discussion:
> >
> > Fred
> >
> >> -----Original Message-----
> >> From: Idr [] On Behalf Of Stewart Bryant
> >> Sent: Tuesday, July 24, 2018 4:10 AM
> >> To:
> >> Cc:;;;;
> >> Subject: [Idr] Rtgdir early review of draft-templin-atn-bgp-07
> >>
> >> Reviewer: Stewart Bryant
> >> Review result: Has Issues
> >>
> >> This is basically a well written draft, although it has has a lots of spelling
> >> mistakes and needs to passed through a spell checker before a further version
> >> is uploaded.
> > Certainly we will fix that; apologies if it impacted the readability.
> >
> >> However, I have a number of issues that the chairs and authors should consider
> >> before deciding how to proceed.
> >>
> >> The first question to ask is whether the work belongs in RTGWG or in IDR since
> >> with the exception of the reference to  the optimization described in
> >> draft-templin-aerolink-82  this is a pure BGP solution. This is something that
> >> the RTGWG chairs should discuss with the IDR chairs. They should also discuss
> >> whether the draft needs to be looked at by a BGP specialist before RTGWG adopts
> >> it.
> > The document has been presented at RTGWG at three different IETF meetings
> > now and seems to have been well received there. So, we would like to see it
> > become an RTGWG working group item so we would not have to start the
> > socialization process over again, but your point is well taken. I do not have
> > enough experience with routing area working groups to understand all of
> > the implications, so any guidance would be appreciated.
> Actually it might also be a candidate for BESS (BGP enabled services).
> This is a chair/AD decision as much as anything, and they are normally
> guided by where
> the required expertise is.

Jeff Tantsura posted a message on this subject that I will defer to.

> >> The approach of building an overlay to separate the network from the main BGP
> >> system is a sound one, indeed from both the numbers and a security point of
> >> view, it would appear to be  essential in a safety critical application such as
> >> this. What bothers me is whether the author is underestimating the risk of
> >> running an application such as this over the public Internet. Whilst as they
> >> explain in the security section, high profile civil users do understand how to
> >> mitigate risk and minimize the effect of attack, none of these organizations
> >> are as large a target as civil aviation flight safety, and thus I would expect
> >> considerable resources, even to nation state level, to be directed at the
> >> attachment points. Hopefully ICAO fully understands the risks in running this
> >> on the public Internet as proposed. If ever there were an application for
> >> inter-domain network slicing, this is it.
> > For civil aviation, this is indeed a critical consideration. On the open public
> > Internet, for example, although IPsec could be used to establish secure
> > tunnels other threat vectors such as DDoS need to be considered. In my
> > understanding, ICAO is thinking about an underlay network comprising
> > secured physical links (e.g., fiber) and/or constructs like SD-WAN. But,
> > the construction of the underlay I think is something that has been
> > taken for granted and as you observe is pivotal to the design security.
> Indeed, I would have expected this to be on a secure network of some
> sort either purely
> private or some form of VPN. However, I am sure I read in your text that
> you were
> considering using the Public Internet much in the way of SD-WAN.

This is an area that could certainly use more discussion, as we have been
seeing on the list. It occurs to me that, if critical services like online banking
can be made secure and robust while using the public Internet as an
underlay, then it may be that carrying air traffic management services via
secured tunnels over the Internet can also be possible. Failing that, it
seems like the only other option is to run dedicated private fibers with
no connections to the Internet - or are there other methods I may be
unaware of?

> >> There is a suggestion in section 3 that for reasons of cost, globally unique
> >> ASNs would not be used. It is difficult to believe that cost is an issue in an
> >> SoF system. What is surely needed is the most robust approach, which in the
> >> long term is usually the cheapest anyway. Using global identifiers minimizes
> >> the possibility of error and issues if ever there is a routing leak, and thus
> >> the decision on whether to use private or global identifiers needs some careful
> >> thought beyond that expressed in this version of the draft.
> > I don't mind dropping the assertion that private AS numbers could be used.
> > As you say, there is probably no downside to procuring real global AS
> > numbers up front even if the overlay is never connected to the global
> > public Internet.
> >
> >> I am worried about the text towards the end of section three which proposes
> >> splitting the network into two or more disjoint systems, since that will surely
> >> lead to operation and integration issues.
> > Looking at that text with fresh eyes, I can see that a rewrite would help
> > bring across the point we are trying to make. What we want to have is
> > a means for supporting multiple independent RIBs and bounding the
> > numbers of prefixes each RIB will be sized to carry. So, if each RIB
> > supported up to 1M prefixes and we had 1K RIBs, we would be able
> > to service 1B prefixes or even a bit more. Does that help clarify? If so,
> > I will consult with co-authors to improve that text.
> Wow, are you sure that such a system is buildable?

I have proven that a single set of core ASBRs can carry 1M BGP routes
even on lightweight linux virtual machines running Quagga. And, only
a few cASBRs (perhaps as many as 10) need to carry the full routing
table. The stub ASBRs only keep track of their associated Clients, and
configure a default route with a cASBR as the next hop for other prefixes.
So, this is a very leaned-down BGP service in comparison to the way BGP
works in the global public Internet.

> What you seem to be building is a huge unaggregated routing system.
> People have considered that before and found that it was
> unimplimentatble. Remember that for every unaggregated prefix in the RIB
> you need an entry in the FIB. The FIB in a core router is high speed
> (expensive) memory and will be replicated on every line card.

I have implemented it, and tested with up to 1M unaggregaed prefixes.
And, again, only a few nodes would need to maintain a full RIB/FIB.

Remember also that the global public Internet has some 600K or more
unaggregated prefixes and seems to still be doing fine, so there is
precedence for a single RIB supporting such large numbers. So, if we
deploy 1K independent RIBs then our 1B target seems feasible.

> I think that you really need to discuss your scaling requirements with
> some hardware
> vendors to verify that the design you propose can support the required
> aircraft
> population over the expected lifetime of the system.
> What I suspect that you need is some sort of formal hierarchy or
> layering in the system to support
> long term scaling, and that is going to be a lot easier if it is
> explicitly designed in on day one.

I understand your concern, but I think we are OK. But, I will bring this
discussion point up with co-authors to make sure we are in agreement.

> >> In section 6 consideration is given to scaling the core, which looks basically
> >> under control for the existing flight profile, but with relatively little
> >> headroom for expansion.  Given the rise in UAVs, that will surely need to be
> >> integrated into a common flight safety system, I am concerned as to whether the
> >> authors have allowed sufficient room for expansion. Again hopefully the flight
> >> specialists have taken this into account and this is not a problem.
> > The upper bound in terms of scaling we were targeting was up to 1B
> > prefixes carried as BGP routes in order to allow for expansion. You are
> > right to observe that we are at the advent of wide-scale deployment of
> > UAVs, but this network would be only for those UAVs that fly to, from
> > or through airspace controlled by Air Traffic Controllers (ATCs) - meaning
> > it would only be for UAVs that take off and land at airports.
> Well, you are the application expert here, but that seems to be a
> critical assumption
> that you need to test in multiple geographies. Thinking about the London
> airports, that
> is quite a restriction.

It is true that there are lots of airplanes out there, but probably not as many
as most people think. I think the numbers of airplanes deployed worldwide
is on the order of 10^4 - 10^5 - not millions upon millions. So, I think we
could live with a system that scales up to 1B unaggregated prefixes for
a long time. 

> > For the future,
> > the types of UAVs that will eventually do that include unmanned cargo
> > delivery freighters and the like. But, we would only be expecting a few
> > thousands, ten-thousands, or hundred-thousansds of those; certainly
> > not millions and millions.
> I am always reminded that when IPv4 was designed, and (in much the same
> era Ethernet)
> their address sizes were an approximation to infinity. They are now a
> long way from infinity.
> This will be an expensive system to replace so it needs migration path
> to its infinity.

It is a good point, but up to 1B unaggregaed prefixes already seems pretty
robust for state of the art dynamic routing protocols. So, to go to something
that supports even greater scaling would require either extending our BGP
based architecture (e.g., by adding another tier in the hierarchy) or introducing
a new architecture that scales in a similar fashion as the global DNS. 

> > For small UAVs that stay away from the airports, however, there is
> > potential for many millions of those or more. For them, the FAA and
> > NASA have a concept known as "Unmanned (air) Traffic Management"
> > or "UTM". The UTM would need to be a separate network unto itself,
> > but we believe these same BGP principles could apply there.
> >
> Best regards
> Stewart