Re: [rrg] [lisp] [Int-area] Please respond: Questions from the IESG as to whether a WG forming BOF is necessary for LISP

Short version:   Timeline of the LISP-ALT/CONS proposal with
                 respect to APT and Ivip.

                 LISP-ALT development looks rather hasty so far
                 and could proceed quite well without a WG.

                 But maybe a WG is the best way - develop
                 LISP-ALT ASAP and see how attractive it is to
                 the great majority of end-user networks.

                 We need most end-user networks, of all sizes,
                 to want to adopt the solution for their own
                 immediate reasons - otherwise, the solution
                 won't cover enough networks to solve the
                 scaling problem.

                 There is a LISP-ALT snowball.  Forming a
                 LISP WG would push the snowball faster.

                 The more prominence and IETF-given
                 credibility LISP gets, the more this might
                 detract attention from the alternative
                 approaches, including APT and Ivip so far.

                 I think these are more sound from technical
                 and business perspectives than LISP-ALT.
                 These are developed by smaller teams with
                 less resources and experience than the LISP
                 team - but maybe these efforts will result in
                 a better architecture.

Hi Noel,

You wrote:

>> The LISP team should do detailed critiques of APT and Ivip as
>> part of convincing IETF folks of the merits of LISP-ALT.
>
> Perhaps I'm confused, but I don't think the people working on
> LISP are trying to "convince" the IETF as a whole of the
> "merits of LISP".

I don't know the criteria is for creating a WG, but I guess the
IETF doesn't create them except for projects which have
sufficient merit in general, as well as meeting formal criteria.

> If some people out there in the Internet technical community
> think it looks good, and want to work on it, run it, etc,
> that's great; but if other people reach a contrary conclusion,
> and don't want to work on it, run it, etc (as is their
> choice), that's fine too.

I am not completely opposed to there being a LISP-WG.  I am just
saying that I think the LISP development has been hurried in a
number of respects, that it doesn't necessarily need a WG and
that a WG could detract attention from more promising other
approaches.

I think the LISP team could best facilitate progress towards
scalable routing solutions by taking more interest in other
proposals and by discussing in public why they think their
LISP-ALT approach is better than the alternatives.

Their goal is 10 billion EIDs.  Assuming this is not an
unrealistic scaling goal, they need to show why ALT could
work well with this, while APT and Ivip could never do so.

The history, as reflected in Internet Drafts, is something like
this (ignoring LISP-NERD, which doesn't scale due to the need
for every ITR to have a copy of the full mapping database):

2007-01-17 draft-farinacci-lisp-00

           This is a conceptual overview which assumes it is
           impossible to get real-time mapping to the ITRs.
           Therefore, ITRs get multiple ETR addresses and have
           to do reachability testing and ETR choice on their
           own.

           There is detailed description of a useful mapping
           system.

           There is no support for packets sent by hosts in
           networks without ITRs.

2007-07-02 draft-jen-apt-00

           APT is the first core-edge separation scheme to
           document a scalable mapping distribution system in
           in Internet Draft.  The full mapping database is
           distributed (slowly) to local query servers -
           Default Mappers.  As with LISP, ITRs - or rather
           Default Mappers - do their own reachability testing
           and make their own ETR choices.

           It later emerged that APT can support packets sent
           from hosts in non-APT networks, but only, for IPv4,
           if the EID prefix is no longer than the in-practice
           limit of the DFZ: 24 bits.  This limit would
           disappear if APT networks connected not only by
           direct router links, but by tunnels, so there was
           only one global "APT Island".

2007-07-03 draft-meyer-lisp-cons-00

           CONS is LISP's first mapping system which is intended
           to be practical in a scalable routing solution.  It
           is a global query server system which involves delays
           for initial packets.  CONS uses a a novel network
           structure comprised of special router-like devices
           called CARs and CDRs.

2007-07-15 draft-whittle-ivip-arch-00

           Fast (effectively real-time) push of all mapping info
           to local query servers.  I developed this
           independently of APT, but the two proposals involve
           some commonalities: the local query server means that
           there is no significant delay for initial (or any)
           packets.

           Ivip includes Open ITRs in the DFZ (then known,
           incorrectly, as "Anycast ITRs in the Core").  These
           support multihoming, TE etc. for packets sent by
           hosts in networks without ITRs.  A business plan for
           OITRDs appeared in 2008-04-18:

             http://psg.com/lists/rrg/2008/msg01158.html

2007-11-11 draft-fuller-lisp-alt-00

           LISP-ALT replaces CONS as the preferred mapping
           distribution system.  Like CONS, it is a global
           query server system, so it has the problem of
           delaying initial packets.  The ALT routers and
           the overall ALT network are created from pre-
           existing components: BGP, RIB, FIB and GRE.

2007-12-04 RRG message that there will be a LISP BOF regarding
           forming a LISP WG.

2007-12-05 draft-lewis-lisp-interworking-00

           LISP Proxy Tunnel Routers provide support for packets
           sent by hosts in non-upgraded networks.  AFAICT, PTRs
           are functionally identical to Ivip's OITRDs.

           However there was no business plan for PTRs or the
           related questions of how EID space is allocated.
           Dino's reply to my point 6:

             http://www.irtf.org/pipermail/rrg/2009-January/000864.html

           indicates there still is no such business plan.

In the view of many people, the delay of initial packets remains
a serious problem for LISP-ALT.  Part if this delay is due to the
global nature of the query server system.  This is made worse by
the long-path problem where the highly aggregated structure of the
ALT network leads to geographically long tunnels between the ALT
routers, which will often be physically located without much regard
for their place in the ALT topology.

In a recent message:

  http://www.irtf.org/pipermail/rrg/2009-January/000900.html

Dino confirmed the problem of initial packet delay and indicated
there is no alternative to a fully distributed query server system
(as opposed to having the full mapping data in hundreds of
thousands of local query servers, as with APT and Ivip) if the
number of EIDs, micronets etc. is to scale to 10 billion.

Yet in the same message he states (presumably about the
long-term future with 10^10 EIDs) that:

   I envision the number of hops a Map-Request takes is 4
   ALT-router hops.

In LISP-ALT, map queries (or initial traffic packets which are
implicitly map queries) arise in ITRs and need to ascend the
highly aggregated ALT hierarchy until they can cross over to the
part of the hierarchy which matches the destination address.
Then they descend, and reach the authoritative query server -
the ETR which is currently advertising this EID in the ALT
network.  (The map reply from the ETR goes to the ITR directly
via the ordinary Internet.)

There's no way that the ALT network could handle 10 billion
separate EID prefixes and avoid large numbers of ALT routers
- and therefore large numbers of hops up and down the ALT
hierarchy for the typical map request.

A 10 billion EID system could only materialise when most EIDs
are for "end-user networks" are for mobile devices which
everyone and their dog has at least one of.

There will be some number of ITRs and some number of ETRs
- I guess a million or so of each, perhaps often combined
into the one device.

The ETRs collectively advertise 10 billion separate EIDs to
the first level (L1) of ALT router.  How numerous are these?

Each such L1 ALT router needs a GRE tunnel (AFAIK) to every
ETR which is one of those currently active for a EID in the
address range the L1 router is aggregating.

The whole idea of a core-edge separation scheme is that the
EIDs (micronets for Ivip) can be freely used with any ISP
for Internet access - and that each EID can be used with
two or more arbitrarily located ISPs for the purpose of
multihoming.

Ignoring multihoming for a moment, there is still the need
for this first level (L1) of ALT routers to collectively
handle 10 billion EID prefixes.  For any one of these L1
ALT routers, there will sometimes be two or more such EIDs
advertised by the same ETR.  Those two routes share the one
ETR to ALT router GRE tunnel.  (AFAIK, GRE tunnels are used
to link ETRs to ALT routers.)

That is a lot of tunnels - and a lot of L1 routers, such as
a hundred thousand or a million or so.

We don't know how much aggregation will be achieved from
ETR to L1, or from L1 to L2 ALT routers.  Many packets
from ITRs will need to ascend to some high level in the
ALT hierarchy before there is a route to the branch of
the hierarchy they need to descend in order to reach
the ETR.

So I don't think it is reasonable for LISP people to say
their system scales to 10^10 EIDs while any other
system definitely can't.

See also a recent RRG discussion "Economic issues of long-
path / stretched routing" where Scott Brim discusses each ETR
announcing its EID prefix to ALT routers at multiple levels
above the first level.  Each such announcement involves a
GRE tunnel (AFAIK) and perhaps constitutes a BGP neighbour
for the ALT router.  This sort of thing is needed to make the
ALT network robust against failure, but it also makes it much
more complex and less able to be flattened into a few levels
- as would be required to reduce the total path length from
ITRs to ETRs.

I mention these difficulties as an example of the mismatch
between the expectations the LISP developers have - for LISP
and therefore, implicitly for APT, Ivip etc. - and the
confidence some LISP developers express in solving the
fundamental problems which have been identified with ALT.

> Sure, the people working on LISP would like to convince _some_
> people that it's a good idea, so they get i) some help working
> on it (as much remains to be done), and ii) some places to
> deploy it (so some practical experience can be gained, in
> seeing how well it works, what problems crop up in actual
> service, etc), but that's a very different kettle of fish.

I fully support all this.  As far as I know an IETF WG is not
required to do any of this.  Maybe a WG would help.  Maybe
the imprimatur of the IETF in deciding that LISP in its current
state is worthy of a WG will encourage more people to help with
LISP.

However, if a WG results in LISP gaining still more prominence,
then I think there could be a serious downside: making it
harder for other proposals to attract the same sort of interest,
attention and involvement.

I believe the initial packet delay problems and the general
fragility of any global query server system such as LISP-ALT
will ultimately sink any such proposal.  Maybe it won't sink in
the IETF, but it will fail to gain the widespread, ideally
ubiquitous,support from end-user networks which will be required
for any routing scaling solution to succeed.

I am sure that the only practical solution is of the core-edge
separation class (LISP, APT, Ivip, TRRP and maybe RANGER - I
still have to read it).

The only scalable way to avoid initial packet delays is to use
local query servers.  I think a system such as APT of Ivip would
be scalable, practical and attractive - and that something like
LISP-ALT will never be attractive enough to be widely accepted.

By the time we get to 10 billion EIDs, micronets or whatever -
which may be never - RAM and CPU power for local query servers
will be way cheaper and more plentiful than today.

Local query servers based on COTS servers today would be fine
with 100 million EIDs in RAM.  For IPv4, for Ivip, each
micronet's mapping information is only 12 bytes and could
probably be stored in less.  IPv6 mapping is more bloated, but
still, there is even more time before that is widely adopted -
more time for RAM to get more plentiful still.

The rate of change of mapping is well within practical limits
for sending to hundreds of thousands of query servers -
especially using a purpose built, streamlined, crosslinked
fan-out system as I propose with Ivip.  Even with mobility,
using the TTR approach to mobility does not result in high
rates of change of mapping.  For instance a mapping change
is only typically required when the MN moves more than about
1000km.

LISP-ALT is the most frequently discussed solution to the routing
scaling problem.  Yet it is a more recent development than APT or
Ivip - and I think it is the most problematic of the three.

I think the prominence of LISP-ALT is not a function of its
technical merits, but more a result of:

  1 - Four or more people working on it, rather than two or so
      (APT) or basically one (Ivip).

  2 - Some employer support for the LISP developers, including
      for travel to meetings and hardware to run a test
      network.  (The APT developers are studying and I am
      self employed.)

  3 - The greater experience of the LISP developers compared
      to that of the other developers - and likewise being
      better known to RRG participants.

  4 - Greater brand recognition for "Cisco" compared to
      the names of developers of alternatives: APT, Ivip,
      TRRP, Six/One Router and now, I think, RANGER.

A WG may overly focus attention on on LISP-ALT and so detract
from the attention and effort needed to develop the more
promising approaches.

 - Robin

_______________________________________________
rrg mailing list
rrg@irtf.org
http://www.irtf.org/mailman/listinfo/rrg