Re: [RAM] LISP NERD/CONS, eFIT-APT & Ivip compared

Hi Dino,

Before I respond, I want to mention that I am keen for people with
your expertise to discuss the question of how to prevent ETRs
being used as a conduit for packets with spoofed source addresses
matching the addresses inside the network.

This is my second message in the "ETRs checking src & dest
addresses" thread, about a week ago:

  http://www1.ietf.org/mail-archive/web/ram/current/msg01705.html

As far as I can see, the best way to do this involves the outer
source address being equal to the sending host's address, not the
ITR's address as LISP and eFIT-APT uses.  A second reason for
doing this, regarding MTU auto discovery, is in my recent message
"Tunneling overheads and fragmentation".

  http://www1.ietf.org/mail-archive/web/ram/current/msg01729.html

You wrote, in part:

>>   http://tools.ietf.org/html/draft-meyer-lisp-cons-00 (July 3)
>
> I would suggest
> http://www.dinof.net/~dino/ietf/draft-meyer-cons-01.txt

This has the same date as the one on the IETF site.  I haven't
looked at the 01 version in detail, but I guess you have updated
some details of the headers.

>> While NERD is "push", in comparison to CONS - which is purely a
>> query and cache system - the mapping updates are not actually sent
>> to the ITRs.  Each ITR has to poll a server and retrieve the
> 
> What you state above is not clear if you are talking about NERD or CONS.

I was referring to NERD ITRs polling servers to get the mapping
data for an entire area of the LISP-mapped address space, and then
polling for updates.

> But in both cases the ITR can get mappings. That is if the ITR has
> enough storage capacity it can take the entire NERD database. 

I think that in NERD the ITRs do need to get the full database,
since there is no reasonably rapid query system as in LISP-CONS

> In CONS,
> the ITRs make requests over TCP connections to their peering CARs that
> return mapping replies back to them.

Yes, if the CAR has the information cached.  If not, the CAR
launches a query (I guess just one) into the CDR network, which
somehow causes the query to reach one or more CARS which are
authoritative for whatever EID address is being queried.  It is
not clear to me how either the initial CAR or the CDR network
identifies which two or more CARs are authoritative - maybe the
query just propagates and is split up at some CDRs when those CDRs
have two paths to peers which are advertising a path to an
authoritative server.

Also, I don't know how multiple replies from authoritative CARs
are either all sent to the querying CAR, or how the CDR network
somehow recognises multiple replies and lets only one propagate.

>>     This will lead to much faster and more reliable query
>>     responses than LISP-CONS - but as with Ivip, raises the
>>     challenging problem of distributing the database updates in
>>     real-time to all these Default Mappers.  However, eFIT-APT
>>     has very slow goals regarding the speed of distributing
>>     these updates.
> 
> I was told by a credible source that most DNS servers cannot handle a
> DNS query packet with multiple query types in it. So you will need
> numerous changes to DNS servers to get QSDs to work. In my book, that is
> a non-starter.

The Query Servers - QSD and QSC (full database and caching) -
which I propose for Ivip have nothing to do with nameservers or
the DNS.  Maybe there is some existing protocol to work from, but
I think the requirements are different enough to make it worth
creating something from scratch.

>> Here are some questions about each scheme, with "LISP" implying
>> LISP 3.x.  There is no concrete information about LISP 3.x other
>> than that implied by NERD and CONS that I won't consider how LISP
>> might work with APT.
> 
> LISP 3.X is not a protocol. It refers to doing encapsulation with an
> external mapping database service such as NERD, CONS, or APT.

Because APT is described as applying to eFIT, that is what I used
to compare with other systems.

I don't see how APT could be applied to what I know about LISP -
since APT has a whole bunch of restrictions on where the ETRs and
ITRs are (always combined together in the PE routers, never
anywhere else) and specific forms of communication between ITRs,
ETRs and Default Mappers which are not present in LISP as far as I
know.

> The variants are models where LISP 1 and LISP 1.5 use a data-triggered
> mapping model. LISP 2 uses the directory already deployed on the network
> which is DNS, and LISP 3 are new models which you see there is a lot of
> activity on.

OK.  In one sense, I can imagine "LISP 3.x" applying to any system
   with a database and communication system separate from the DNS
or from data-triggered communications.  In this sense, perhaps,
eFIT-APT and Ivip are part of "LISP 3.x".  But I think it is
better to think of LISP-NERD and LISP-CONS as being the two LISP
3.x proposals so far.

>> Encapsulation of data
>> ---------------------
>>
>> LISP-NERD/CONS: Not documented yet, but presumably using UDP, with
>> the outer source address being that of the ITR which encapsulated
>> the packet. (Outer SA = ITR's address.)  See the recent messages
>> "ETRs checking src & dest addresses" about how I think this makes
>> it harder for the ETR's filtering system to protect against
>> spoofing of local source addresses.
> 
> Encapsulation is defined in draft LISP-02. NERD and CONS are
> control-plane mapping database services only.

OK, your LISP-02 was announced a few hours after I posted my
comparison.  I can't easily see what has changed, but I noticed these:

  New text at end of definitions of ITR and ETR.

  On page 9, new text for paragraph starting with "EIDs":

     They are expected to be used locally for intra-site
     communication.

  On page 20, the vertical "Record" bar goes higher.

  Section 6.1.3 the format has changed for a longer nonce and
  the Path Vector List is not used for Map Replies.

>> LISP-NERD/CONS:  As noted above, ITRs have complex functions,
>> including making decisions on a packet-by-packet basis for MH
>> service restoration, TE and I think load balancing functions.  The
>> ITR may or may not communicate with the ETR, but I think the ITR
>> is always ready to accept ICMP messages as sign it is tunneling
>> the data packets to the wrong address.
> 
> I beg to differ. The ITR simply does a cache lookup on the DA when the
> DA has a routing table miss or a default route match. When the cache
> lookup returns an RLOC it slaps a header on. It's as simple as that.

It was my impression that in LISP-CONS, as in LISP 1/1.5 and
LISP-NERD that the ITRs have to make decisions regarding choosing
another RLOC for service restoration in a multihoming situation.

For instance, from LISP-CONS-00 (01 is the same):

|  Note that LISP-CONS is not designed for the "fast-mobility"
|  case.  That is, it is envisioned that the mappings distributed
|  by LISP-CONS are reasonably static.  LISP-CONS is also not
|  designed to carry Locator Reachability status information; see
|  [LISP-01] for details on how LISP determines locator
|  reachability.

LISP-01 (02 is the same) has a bunch of stuff about reachability,
including, for instance:

|  6.3.  Routing Locator Reachability
|
|  There are 4 methods for determining when a Locator is either
|  reachable or has become unreachable:
|
|  1.  Locator reachability is determined by an ETR by examining
|      the Loc-Reach-Bits from a LISP header of a Data Message
|      which is provided by an ITR when an ITR encapsulates data.
|
|  2.  Locator unreachability is determined by an ITR by receiving
|      ICMP Network or Host Unreachable messages.
|
|  3.  ETR unreachability is determined when a host sends an ICMP
|      Port Unreachable message.
|
|  4.  Locator reachability is determined by receiving a Map-Reply
|      message from a ETR's Locator address in response to a
|      previously sent Map-Request.

Does any of this apply to LISP-CONS?  Each of these involves an
ETR or an ITR in doing something much more complex, involving the
central CPU etc. (and also more open to security problems) then
simply encapsulating or decapsulating packets.

Is it true to say that in LISP-CONS, the ITRs don't accept any
messages from anything, including ICMP messages - other than the
replies to their queries to CARs?

Does it mean that service restoration when an end-user wants the
packets to go to an ETR in ISP-B instead of the ETR in ISP-A are
all controlled by changing the mapping data? (This is the Ivip
approach.)

If so, then I don't see how that fits with what I quoted above
from LISP-CONS: "the mappings distributed by LISP-CONS are
reasonably static".

If in LISP-CONS the ITR doesn't send any messages to the ETR, or
take any notice of ICMP messages, then I will amend my comparison,
because that makes LISP-CONS a conceptually much simpler system,
like Ivip, compared to LISP-NERD and eFIT-APT - which both
delegate responsibility for service restoration and TE to the ITRs
and ETRs in some way.

I had thought that in LISP-CONS the mapping data includes the TE
control parameters Priority and Weight.  I forgot that in the
LISP-CONS I-D these are marked as "unused".

Is it also true to say that if the mapping data is changed in the
authoritative CAR(s) that the only way this propagates to ITRs
which currently have the last version of the mapping data is by
each ITR's cached data expiring, and the ITR requesting the
mapping data again because it is still handling packets for this EID?

That is to say, there is no message emanating out of the
authoritative CAR to CARs which recently requested mapping
information (cache invalidation, or like "Notify" in Ivip)?

If so - which is my current understanding - then I think it would
be very hard for LISP-CONS to respond quickly to mapping changes,
such as with multihoming service restoration, unless the cache
times were short - which would burden the entire CAR-CDR network
with vast numbers of requests and responses.

>> Since LISP-mapped addresses are not in any prefix which is
>> advertised in BGP, ITRs must be inside or at the border of
>> provider or end-user networks.
> 
> This is not true. If you wanted to put an ITR at a PE edge and the
> EID-prefixes for each site connected to the PE advertised them via BGP,
> the ITR could act as an ETR for packets it receives from the provider
> network. For encapsulating, you still need a mapping database.

I don't understand most of this, and I don't understand how it
relates to the text quoted above.

I don't understand the term "EID-prefixes for each site", because
to me the whole idea of an ID/Loc separation system is that there
is one or more block of IP addresses (prefixes of EIDs in LISP
terminology) which are for hosts which could connected at any
ETR-equipped provider network in the Net.

The key point is that for sending hosts in non-upgraded networks,
how do they send a packet to an address which is now an EID as
part of the LISP system?

The destination host could be anywhere - in, or connected to, any
provider network with an ETR which is configured to handle packets
for this host.

The only way the packet is going to get to the destination is to
be encapsulated by an ITR.  But in all versions of LISP, as best I
understand it, any address which is an EID is not part of any
advertised BGP prefix.  So in the non-upgraded network, it is
dropped by the first router, since there is no reason to send it
anywhere.  There is no ITR in the non-upgraded network and that is
the end of the story.

With Ivip, the packet is forwarded out the border router, because
it matches a BGP advertised prefix, and then it is forwarded to
the nearest ITR in the core of the network (or perhaps a border
router of another network) - since there are lots of such ITRs all
advertising the same prefix which this packet matches.

>> LISP-NERD/CONS: ETRs are located in the provider network or at its
>> border router, or in the end-user's network or at its border (CE)
>> with the provider network - as long as the address is BGP
>> reachable.  I think ETRs have complex communication functions.
> 
> NERD and CONS say nothing about where ETRs are placed in the network.
> And they don't need to either.

I guess I am extrapolating from the basic LISP I-D.  It is hard to
figure some of this LISP 3.x stuff out, since the LISP I-D only
formally describes variants 1 and 1.5. - and yet LISP-CONS refers
to LISP-0x for some of its definitions, such as regarding
reachability.

>> Incremental deployability
>> -------------------------
>>
>> LISP-NERD/CONS: According to current definitions and all my
>> off-list attempts to understand LISP, LISP-mapped addresses are
>> not part of BGP advertised prefixes.  So in order for a host to
>> send packets to a host on any LISP-mapped address, the sending
>> host must be in a network with an ITR.  I think this makes LISP
> 
> Or the EIDs are from PA space or the EIDs are from PI space that
> continues to get advertised into the network. There will be these PIs
> that will never be withdrawn. And this is a good thing. We just don't
> want all sites to do this.

This appears to be different from everything I knew about LISP -
that the EIDs never matched BGP advertised prefixes.

If a site Z has a prefix (PI or PA, it doesn't matter) and this is
advertised in BGP, then all the packets from non-upgraded networks
will be forwarded to Z's border router.

Maybe you have an ITR there which tunnels them to the ETR of
whatever other network the destination host might be in.

That would achieve full reachability, but the total path length
from sending host to destination host could be much longer than
optimal, and you would be relying on a single border router to be
the ITR.

Ivip does better than this by having many ITRs in the "core", or
at least near the border routers of non-upgraded networks.  By
using anycast, the packet automatically finds its way to the
closest ITR, and then is tunneled on a path to wherever the
destination host's ETR is, which is likely to be a close to
optimal path length.  With Ivip, the total path length will
generally be much shorter than with what I understand from what
you wrote above: all the packets going to a single site's border
router, which tunnels them to an ETR which may be anywhere in the Net.

>> not at all incrementally deployable.  Why would an early adopter
>> want to use LISP-mapped addresses when it will make their hosts
>> unreachable for any communication initiated by a host on a
>> non-LISP-upgraded network?
> 
> This is not true. Remember what else you need for an incrementally
> deployed design. That is no changes to hosts, no changes to DNS, no
> changes to core routers. 

Ivip doesn't involve any changes to DNS.

There does need to be some "anycast ITRs in the core" - but they
could be multiple border routers of other networks which all
(anycast) advertise the prefixes of Ivip-mapped address space.

If you could confirm what I wrote above - me trying to express my
understand of what you wrote - then it seems you are proposing a
single network's border router(s) be the ITR for packets from
non-upgraded networks.  I think that would remove the major
difficulty I see regarding reachability, but it would lead to
longer paths in general than using anycast as I propose with Ivip.

> LISP requires *only changes* to CE routers. Or
> P-routers in the core if you need LISP for ISP-based TE.
> 
> There are many respects to incrementally deployable not just an non-LISP
> site talking to a LISP-site.

I agree.  There are real questions of who is going to write code,
build servers and new functions in routers to get the thing going
- LISP or Ivip or anything else.

>> As with all proposals, there are major challenges in setting up
>> the infrastructure.  Ivip's Replicator and ITR system is
>> adventurous, but the ITRs are doing a simpler job and the overall
>> design is simpler than the designs of other proposals.  The ETRs
>> are free-wheeling, low-cost devices and require no coordination.
>> With sufficient provider networks with ETRs, one or more
>> independent Ivip or Ivip-like systems could be set up, and work
>> fine alongside each other (each with their own set of core ITRs,
>> or perhaps all driving data to a common set).  These could
>> probably run at a profit, because they provide portability,
>> multihoming (with basic TE) and mobility with full reachability
>> (and generally pretty good path lengths, if the core-ITRs are well
>> distributed) from all hosts.
> 
> I haven't heard how any of the non-LISP proposals deal with ISP-based
> traffic engineering and multicast.

With Ivip, the end-user who has ultimate control of the mapping of
their Ivip-mapped addresses, gives their username and password (or
whatever they use to control the mapping) to whoever it is they
want to control the mapping.

Can you give me an example of "ISP-based traffic engineering"?

I tend to think of TE concerning incoming traffic being spread
over two or more links, and therefore ETRs and their provider
networks, under the control of the end-user whose multihomed site
is the recipient of those packets.  They can easily choose the
path of outgoing packets.  At present they use BGP to control
incoming traffic.  With Ivip they directly change their mapping
data to make packets for some IP addresses or prefixes be tunneled
to one ETR and others to another ETR.

Maybe that is what happens with LISP-CONS too.

I think LISP-NERD involves the ITRs making decisions about where
to tunnel packets, based on various priorities and weights.

As far as I know, packets to multicast addresses are not found
handled by the Internet.  LISP-CONS doesn't mention multicast.  My
copy of LISP-01 has a bunch of question marks down the side of the
"Multicast Considerations" page.

Can you explain how multicast packets are, or will be, used such
that they are forwarded between BGP routers, and then try to give
a better explanation of how LISP handles them?

  - Robin

_______________________________________________
RAM mailing list
RAM@iab.org
https://www1.ietf.org/mailman/listinfo/ram