[RRG] APT security and other matters
Robin Whittle <rw@firstpr.com.au> Wed, 12 March 2008 12:44 UTC
Envelope-to: rrg-data@psg.com
Delivery-date: Wed, 12 Mar 2008 12:43:51 +0000
Message-ID: <47D7D010.7040102@firstpr.com.au>
Date: Wed, 12 Mar 2008 23:44:00 +1100
From: Robin Whittle <rw@firstpr.com.au>
Organization: First Principles
User-Agent: Thunderbird 2.0.0.12 (Windows/20080213)
MIME-Version: 1.0
To: Routing Research Group <rrg@psg.com>
Cc: Michael R Meisel <meisel@cs.ucla.edu>, Dan Jen <jenster@CS.UCLA.EDU>
Subject: [RRG] APT security and other matters
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit
Here are some comments on: http://www.cs.ucla.edu/~meisel/apt-rrg.pdf & http://cs.ucla.edu/~meisel/draft-apt-incremental-00.txt Path MTU and Fragmentation? ITR and ETR placement Load Sharing Failure Detection and Recovery Anycast and multicast DMs collating mapping information Recursive Application Detecting bogus MapSets - after the fact Changing providers Decentralised authentication of end-users credentials Charging for mapping changes I am mainly pointing out things I am critical of. The basic architecture of DMs as local query servers with other ITRs being caching ITRs is a really good one, I think. - Robin Path MTU and Fragmentation? --------------------------- APT makes no mention of coping with the PMTU and fragmentation problems which I think any map-encap scheme needs to cope with. TRRP doesn't have an approach to this either. Recent discussions on the list have been aimed at improving LISP's approach to PMTUD and fragmentation. Ivip's approach will be based on: http://www.firstpr.com.au/ip/ivip/pmtud-frag/ and Fred Templin's work: http://tools.ietf.org/html/draft-templin-seal hopefully without any extra headers. ITR and ETR placement --------------------- apt-rrg.pdf: I note that APT places the ITR and ETR functions (typically in the same device) at a very specific location: the Provider Edge routers of ISPs. This contrasts with what Dino wrote a few days ago - that with LISP the ITR and ETR functions would be in Customer Edge routers: http://psg.com/lists/rrg/2008/msg00687.html For Loc/ID purposes (there are several other reasons to use LISP than what is intended by this venue), we want to strongly suggest that xTRs be placed on CE (CPE) routers. We think that is the best balance of tradeoffs. The Provider Edge location makes more sense to me. With Ivip, they can be placed anywhere, but the Provider Edge would be a good place. I figure your Default Mapper (DM) function is placed somewhere else, such as in the ISP network somewhere, or at border routers. apt-incremental: Here you state that the ITR and ETR functions are performed at Border Routers: APT islands configure their border routers as "tunnel routers" (TRs) so that their customers' data packets will be encapsulated and decapsulated as they enter and exit the AS. which seems to contradict the text in apt-rrg.pdf: APT places TRs at the provider-side of the link between edge networks and their providers ... However, you do need ITRs at the border routers if APT networks are to receive and correctly tunnel packets sent by hosts in non-APT networks which are addressed to APT-mapped addresses. Load Sharing ------------ The DM makes a decision about which ETR to use every time it gets a mapping request (in the form of a traffic packet) from an ITR. I assume that where there are multiple ETRs which it considers reachable at the same priority level, it uses a pseudo-random number to choose between them according to their weights. The DM tunnels the packet to that ETR and tells the ITR to use that ETR. I note that the DM uses that ITR's address in the outer header, so the ETR has the impression it was tunnelled by the ITR. This seems good to me, but breaks a convention of using the genuine tunnel start point address in source address of the outer header. I completely break this convention with Ivip - by using the sending host's address - so I am interested to see the convention slightly broken in APT. I understand that the ITR keeps this one ETR address and tunnels all packets there. So if the ITR has one or more sending hosts sending packets to a particular edge prefix (prefix with an APT-mapped address) then it will keep sending all those packets to one ETR. This is more likely to keep them in order at the receiving host than if the ITR sent some packets to one ETR and others to another ETR. This also simplifies the ITR, compared to LISP or TRRP, which I think is a very good thing. Failure Detection and Recovery ------------------------------ APT detects 3 kinds of failure - in all cases without any explicit probing or ETR -> ITR communication. I am not sure this is a complete list of failure modes in a multihoming situation. With Ivip, failure detection is left to the end-user's choice of system - so they can use whatever techniques they like. LISP, APT and TRRP hard-wire the failure detection system into the protocols and ITRs, ETRs (and DMs for APT), so it had better be complete and robust. Even then, it is fixed in its functionality and I think a major advantage of Ivip is its modular nature - that it lets the user do whatever they like to detect failure and decide which ETR traffic should be tunnelled to. 1 - The transit prefix that contains the ETR becomes unreachable. I think you mean by this that the BGP system does not provide a route to this prefix, in which case the ITR (or DM) finds this out either directly from its BGP activities, or via the internal routing system. However, perhaps the network in which the ETR is located is unreachable, but this has not yet propagated through BGP to the APT system with the ITR and DM. So I think this does not rapidly or reliably detect all actual failures of this nature. 2 - The ETR itself becomes unreachable. You are relying on the ETR in the destination network's ISP going missing from the internal routing system, and so having the tunnelled packets arrive at a DM there. This would capture some faults, but what if the ETR was simply dead or not working properly? That wouldn't necessarily show up in the internal routing system. This system assumes that there is a DM there which is working, and that it can send messages to the DM in the sending ISP. But perhaps some kinds of failure would disrupt this, and so not be visible to the DM in the sending network's ISP's network. 3 - ETR can't reach the destination network. This seems more robust, since you state somewhere that the ETR has a direct connection to the destination network. Again, it depends on a DM in the destination ISP network, but that is less likely to be affected by whatever clobbered the link to the edge network's CE router. How does the DM in the ETR-end network send a message to the DM in the sending network's ISP's network? This is supposedly specified in section 4.5.2. A DM in ISP-B needs to know the address of a DM in ISP-A's network, based only on the ITR address in ISP-A - which is all it knows about the source of the tunnelled packet. 4.5.2 doesn't mention this. I guess the DM in ISP-A uses the web of trust and certificates to check the validity of the failure report from the DM in ISP-A. Since ISP-A could have hundreds or thousands of ITRs, and dozens or hundreds of DMs, is it OK for the ISP-B DM to choose any one of the DMs in ISP-A? What if that DM is dead? How does it choose another? How could ISP-A in some way publish all its DM addresses, without this being a major storage burden for all other APT networks, which would presumably have to remember this stuff in preparation for one of their DMs sending a failure message? This brute force approach would have every APT network knowing the address of every DM in every other APT network. As with LISP and TRRP, you rely on each ITR (or in this case ITR DM combination) to find out in their own way about failures which would mean they need to select another ETR. With Ivip, some external system finds this out, makes a decision, and tells all the ITRDs and QSDs the new ETR address. The QSDs immediately notifies any ITR which needs to know of the new ETR address. I understand how the DM in a sending network's ISP network can be informed of failures which would require a new decision about which ETR to tunnel the packets to. I can't see in apt-rrg.pdf how the DM tells the ITR about this, which it needs to for failure types 2 and 3. I recall there was some kind of notification system, but I can't easily see this in draft-jen-apt-01. Anycast and multicast --------------------- The DMs within an APT network are supposed to share information with each other via an internal anycast address DMany and that there is also an internal DMall multicast group. I am not sure how this could be achieved reliably. How do you use either anycast or multicast to reliably send information to or between a group of devices? An external anycast address DMany_ext is initially (p3) mentioned as an option, but later (p5) assumed to exist for all APT networks. If you have an external anycast address which all DMs respond to, then if you have multiple border routers and multiple DMs, unless there are packet losses, the incoming packet will go to one DM. This means you can't use TCP etc. but must use simple UDP message and response arrangements. You also have to assume that the one DM which gets the message can get it to all the others in the network. I think this inter-DM traffic could become a scaling problem in the really large networks. They all need to respond as if they were one system, since any one of them is supposed to accept a message for all of them, and acknowledge its receipt. DMs collating mapping information --------------------------------- DMs have a lot of work to do. If you have hundreds of millions of edge prefixes (EID prefixes in LISP terminology = micronets for Ivip) and each one has two or three ETRs, then the DM will get two or three sets of mapping for each edge prefix, one from each ETR's ISP's DM. (I think only Ivip will get the really large numbers of micronets, EID prefixes etc. because it is the only one which provides a new form of mobility. Nonetheless, other schemes such as ALT and TRRP supposedly can cope with billions of EIDs etc. They can't do mobility either, but their proponents criticise APT and Ivip for being less scalable in this regard. So I will continue to assume APT could be called upon to do the hundreds of millions of edge prefixes the ALT and TRRP folks talk about.) This is a lot of data to hold and store. When a mapping request (traffic packet) arrives, it has to look at the two or three or whatever sets of mapping information for each prefix. Fortunately, the DM is an intensive device and only does this for fresh flows. However, with P2P traffic, there are going to be lots of fresh flows! Worm activity is even worse. So I think the DM has a lot of work to do - compared to an Ivip full database ITR or query server (ITRD or QSD) which simply looks up the micronet and finds a single ETR address there. Recursive Application --------------------- I couldn't envisage this section in a concrete manner. Detecting bogus MapSets - after the fact ---------------------------------------- This seems to be the most striking security weakness in APT. The DM accepts all MapSets which are correctly signed by any one of the tens or hundreds of thousands of APT ISPs. Assuming an attacker convinces any one of these ISPs that it is the legitimate "owner" of some edge prefix 22.22.22.0/20, then the attacker can have a MapSet for this prefix propagated around the world, directing traffic to its own ETR. You state (p7) that the only protection against this is the legitimate owner monitoring the APT mapping traffic, detecting the bogus MapSet, and then to "take action". By then, it would be too late, since the bogus MapSet would be used by some DMs and their ITRs, probably for quite a while longer, since I don't know how a DM can tell an ITR to change its mapping. This seems an onerous task for every end-user network, though I suppose it could be handled efficiently by a suitably equipped company. That company could set up servers all around the Net, sucking in the APT mapping traffic at various points, and checking it for validity against its customer's specifications. But how to take action? You have to find out which ISP signed the MapSet and contact them by some presumably non-automated means. Then you need to prove your bona-fides better than the attacker proved themselves to be the legitimate owner. Then the ISP has to withdraw the MapSet and presumably block the traffic packets being sent to the attacker's ETR, which is presumably in one of its ordinary prefixes. Meanwhile, packets are being snarfed by the attacker . . . Changing providers ------------------ In order to change providers, an end-user network needs to get the old provider to stop issuing MapSets. I guess this is the same as with ISPs advertising PI space now. However, it is not a problem with Ivip, since the system which controls the mapping information has nothing necessarily to do with whatever ISPs the end-user selects for connectivity. Decentralised authentication of end-users credentials ----------------------------------------------------- Ivip's approach is pretty simple. The Replicator system fans out signed updates for each Mapped Address Block. A separate server system provides snapshots for boot time, also signed. The mapping for each MAB is controlled by an RUAS - Root Update Authorisation System. The ITRDs and QSDs basically trust the updates and snapshots they get, which are signed by one of these few dozen RUASes. The RUASes make sure that only the end-user can give commands to their system to change the mapping. This makes life relatively easy for the ITRDs and QSDs which receive the full stream of mapping updates. This has absolutely nothing to do with ETRs or with whichever ISPs the end-user selects for connectivity. So the ISPs don't do anything, and can't do anything, to hasten or thwart a change of service from one ISP to another. Also, they can't do anything, prompted by an attacker, to change the mapping information. With APT, you rely on each ISP somehow deciding that the entity which purports to be the rightful owner of edge network X really is that entity. Once an ISP accepts this, that entity can have the ISP propagate a MapSet all over the world. LISP-ALT is probably similar in this regard. I think it is much better to have a centralised system, unrelated to ISPs and ETRs, by which the end-user controlling the mapping. Charging for mapping changes ---------------------------- Like today's BGP system, APT has a completely distributed system by which end-users inject their changes into the global system. Every such change propagates and costs resources in some or all the networks in the global system. Because there is no central registry of what is legitimate or not, and no gateway, there is no way of preventing someone changing their mapping ever 20 minutes or whatever. This is true for BGP and APT. With Ivip, the end-user pays the RUAS for propagating each mapping change, and possibly for the rental of their address space - with the RUAS running "anycast ITRs in the core/DFZ" for their address space, and contributing to the cost of the global "Replicator" fast-push network. So Ivip contains a revenue generation system - the RUAS charging per update - by which the cost of the fast-push system can be recovered. This ensures that no matter what the demand and whatever the cost of supplying the push service, there will be some price set which makes the system profitable, or at least sustainable. With BGP and ALT, each injected change is a burden to all others - and there is no way of collecting fees or stopping an "unreasonable number" of changes without also disrupting "legitimate" changes (however judged). -- to unsubscribe send a message to rrg-request@psg.com with the word 'unsubscribe' in a single line as the message text body. archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg
- [RRG] APT security and other matters Robin Whittle
- Re: [RRG] APT security and other matters Scott Brim