Re: [RAM] First cut at routing & addressing problem statement

Hi JFC Morphin,

The missing point 3 is just a typo.  The other points you mention
seem to involve long-term new architectural goals involving host
changes.

I have approached the RADIR Problem Statement assuming it concerns
a problem which requires a solution which is incrementally
deployable and automatically enables existing hosts, without
changes, to remain connected, but achieves important benefits such
as multihoming, portability, TE etc. for many more end-users
without adding to the growth in the global routing table.

In fact, there is no timeframe or requirement for unchanged hosts,
or incremental deployability, in the current Problem Statement.
Below, I discuss it as if this was the goal.  Perhaps there needs
to be two problem statements.  One is to get us out of a fix in
the next five to ten years, without requiring host changes.  The
other is to create a new set of protocols or whatever to achieve
some of the more ambitious goals you discuss.  As far as I know,
your goals can't be done without changes to host operating systems
and probably applications too.

The rest of this message responds to Thomas Narten's reply to my
message 1773.

Thanks Thomas for your response.  Some of my message discussed
what I think can and can't be expected from potential solutions.
I wasn't suggesting that that material be included in the Problem
Statement.  I think it is important not to write the Problem
Statement as if something is possible when it is not.

I mentioned the capabilities and limitations of various solutions
to explain why I think it is important to include criteria which
identify solutions which might preclude the realisation of an
important goal which is not part of solving the current Problem.

Thanks for clarifying "site".  I agree with much of what you
wrote. Here are some points where where I disagree, don't
understand etc.

>> I think the current version of the Statement is compatible with
>> these two statements being true - but I think they are not true:
> 
>> 1 - That a new architecture could make renumbering robust
>>     and sufficiently painless and free of costs and risks
>>     that all end-users would, or should, be happy to get
>>     new addresses whenever they choose a new provider.
> 
> If we could do this, then I think we have solved a core
> problem. Whether or not the above is actually achievable is an open
> question (and I recognize that many are skeptical).

> But an important goal of the problem statement is to define the
> problem and not rule in (or out) any particular solution approach up
> front. If a proposed solution actually does addresses the problem (and
> is deployable in practice), we should be happy. :-)

I think that it is distracting and of little or no use to frame a
statement as if some solution could exist for a problem when it
can be reliably proven that no such solution is possible.

Ignoring IPv6 for a moment, consider the large number of IPv4
end-user networks.  There is no way these can be renumbered
reliably, automatically, or manually, because there are many IP
addresses in obscure places, as the RRG "Design Goals" note, such as:

  In addition to being configured into hosts and routers, where
  automated renumbering tools can help, IP addresses are also
  often used for other purposes such as access control lists.
  They are also sometimes hard-coded into applications used in
  environments where failure of the DNS could be catastrophic
  (e.g. certain remote monitoring applications).  Although
  renumbering may be considered a mild inconvenience for some
  sites, and guidelines have been developed for renumbering a
  network without a flag day [RFC4192], for others, the necessary
  changes are sufficiently difficult so as to make renumbering
  effectively impossible.

These obscure locations of IP addresses cannot be known to any
newly introduced architecture, software tools etc. - so there is
no external solution which can enable these networks to reliably
renumber in some clean, automated, manner.  (Even if the IP
addresses were known to some system, how could such a system
reliably know they should be changed - and how could such a system
re-initialise various daemons, in the right order, at the right time?)

I regard the above as complete proof that any new architecture
will be incapable of providing reliable renumbering of most
substantial IPv4 networks in use today, or in the timeframe I
assume is of interest: the next ten years.  Can anyone think of a
counter argument?

In the long term, it might be possible to define some new
architecture to which all new routers, operating systems etc.
would fully comply, with systematic linkages for updating
configurations and reloading those config files etc.  Even then, I
doubt any network administrator would be confident about letting
some intricate automated web of processes suddenly try to steer
their network on another course, all at once.

If some new architecture is developed for reliable fully automated
renumbering, then it would be a decade or more before it could be
assumed that many networks would be fully compliant with that
scheme.  So an architecture which supports reliable automated
renumbering is a solution which might have some value around 2022
or so.

As you note, even with a decade or so of efforts, it is not close
to being achieved with IPv6.

Assuming the Problem Statement is referring to a solution which
will have reliable, practical, benefits in the 2010 to 2012
timeframe, I believe that reliable automated renumbering of
networks can be completely ruled out as a possibility.

In that case, I stand by my argument that it would be best to
frame the Problem Statement without the assumption or implication
that something which is provably impossible may in fact be
possible in the timeframe of interest.

>> 2 - That a new architecture cannot, or should not, enable
>>     a much finer method of allocating IPv4 address space, to
>>     accommodate more end-users with multihoming, portability
>>     etc. - and that this cannot or should not be used to
>>     enable a much greater number of end-users of all types,
>>     (including those who don't need portability/MH/TE) to
>>     use IPv4.
> 
> I'm not sure I'd go as far as the above. I think you may be making
> some assumptions above that are not immediately obvious to me. But I
> do agree that giving out smaller and smaller address blocks to
> sites/users and expecting them to be routed in the DFZ makes the
> problem worse rather than better.

I should have added this to the above paragraph:

... enable a much greater number of end-users of all types,
    (including those who don't need portability/MH/TE) to
    use IPv4 without adding to growth in the global routing table.
             ----------------------------------------------------

I think that something very clear to me is either not understood
or not agreed to by some people:

If you have a largish subnet assigned to the LISP/eFIT-APT/Ivip
system, say 20.0.0.0/14, with 262,144 IP addresses, then this will
require either no BGP advertised prefixes or for Ivip, one
advertised prefix.

The ITR-ETR-database tunneling and mapping system
(LISP/eFIT-APT/Ivip) is capable of dividing these 262,144
addresses between a large number of end-users.  In principle, each
IP address could go to a separate end-user, with portability,
multihoming and TE.

A more likely outcome is some mix of prefix sizes going to
end-users.  It is easy to imagine some mix of prefix sizes between
 /20 and /32 - with thousands of end-users getting what they need,
which may be only one or a few IP addresses, or may in some cases
be a few thousand.

This address space can be divided up much more finely than with
IPv4's current (and likely future) BGP arrangements which, in
practice, though not technically, restrict the advertised prefix
length to 24 or less.  Changing that limit wouldn't help, because
the real problem is the number of advertised prefixes.

Any of these schemes (LISP/eFIT-APT/Ivip) could take this /14 and
with either 0 or 1 extra prefixes in the global routing table,
serve the needs of hundreds or more likely thousands of end-users.

I think that the LISP and eFIT-APT proponents see their schemes as
being beneficial primarily by providing multihoming and TE to
single prefixes - so the overall benefit is mainly that a prefix
can be taken out of the BGP system and then still be used for MH,
TE etc. with LISP or eFIT-APT.

I have always seen the primary benefit of these schemes being that
a single block of address space can be used for many more
end-users than with BGP, because it becomes finely divisible
without concern about route aggregation or the effective 256
address granularity of BGP.

>> I am also keen to see the new architecture either directly support
>> Mobile IP, or at least be compatible with future extensions which
>> do so.  I mean an expansive view of Mobile IPv4 and IPv6, with
>> correspondent hosts not requiring new software, and with close to
>> optimal path lengths.
> 
> I think the document is clear on this point:
> 
>    The problem statement in this document has purposefully been scoped
>    to focus on the growth of the routing update function of the DFZ.
>    Other problems that may seem related, but do not directly impact the
>    route scaling problem are not considered to be "in scope" at this
>    time.  For example, Mobile IP [[RFC2002], RFC3775] and NEMO
>    [[RFC3963]] place no pressures on the routing system.  They are
>    layered on top of existing IP, using tunneling to forward packets via
>    a care-of addresses.  Hence, "improving" these technologies (e.g., by
>    having them leverage a solution to the multihoming problem), while a
>    laudable goal, is not considered a part of this problem statement.

I agree with this, but I am suggesting that since some solutions
to the "multihoming, portability, TE vs. routing table growth"
problem (one so far - Ivip) would enable great benefits for Mobile
IP and that others would not be extendable to provide these
benefits, that the Problem Statement should support a decision
framework which gives preference to a solution which either
provides the Mobile IP benefits, or does not create barriers to
such benefits being added in the future.

In principle, it might be possible to add an Ivip-style
"real-time" database distribution system to LISP-NERD/CONS or to
eFIT-APT, but this would be a major change to those architectures,
and made more difficult by the more complex and lengthier nature
of the mapping data than is used with Ivip.

>> For instance, the new architecture's ITR system could be used to
>> create generally optimal paths to ETRs which are close to the
>> mobile node.  This would enable correspondent hosts without any
>> special mobile IP software to communicate efficiently with mobile
>> nodes, including small devices (cell-phones and laptops) as well
>> as nodes which connect mobile networks - for instance in passenger
>> jets and trains.
> 
> I think you are now going into solution space, something the problem
> statement document is trying very hard to avoid. Again, quoting from
> the document:
> 
>    This document attempts to define the "problem", with the aim of
>    describing the essential aspects so that the community has a way of
>    evaluating whether proposed solutions actually address or impact the
>    underlying problem or "pain points" in a significant manner.
> 
> Our view is that improved mobility is a "nice to have", rather than
> critical requirement.

I am discussing solutions to show that there are important
differences between the current solutions regarding mobility.
I am not suggesting that this discussion appear in section 6 of
the Problem Statement.

>> I am a little wary that the current version of the Problem
>> Statement's goals could be used to justify a new architecture
>> which pressures end-users to adopt IPv6, and/or to make any other
>> such major change their host operating system and application
>> software.
> 
> If IPv6 solved the route scaling problem, I don't think we'd have much
> trouble convincing people to adopt IPv6. But since it doesn't, we
> do not find ourselves in that situation.
> 
>> I think some people see IPv4 as a dead-end and so can only foresee
>> long-term solutions which involve everyone moving to IPv6.
> 
> Not sure what this has to do with the problem statement.
> 
>> While I recognise there are fundamental problems with IPv4 (NAT
>> and limited address space are the most obvious) I think it would
>> be dangerous in the five to ten year timeframe in which this
>> Problem Statement is relevant to think that the way forward may be
>> to discourage people from using IPv4.  I think that any such
>> attempt by the IETF etc. is bound to fail.
> 
> Can you point to specific text that leads you to feel that the problem
> statement is discouraging continued use of IPv4?

I see nothing in the Problem Statement which discourages continued
use of IPv4.  Nor did I want to imply that this was a motivation
of those who wrote it.

The current text does not mention the timeframe the new
architecture is to be developed and deployed in.  Nor does it
mention that in order to be acceptable, the new architecture must
not require changes to host operating systems, software (and
perhaps configuration).

In this whole debate, these requirements are widely assumed - or
perhaps this RADIR statement is for a longer timeframe, so it is
another debate.

If words to the effect of these were added:

   The new approach must be developed and deployed so that its
   benefits start to accrue by 2011, with full benefits being
   achieved by 2015.

   The new approach must not compromise the reliability of
   communications between hosts in networks which have not yet
   adopted the new approach and hosts in networks which have
   adopted it.

   In order that the new approach be reliably and incrementally
   deployable, it must not rely on changes to any hosts, including
   those in networks adopting the new approach, in order that
   those hosts maintain reliable connectivity with all other
   hosts.

     (This does not rule out an architecture which marginally
      degrades performance for all hosts, with some of that
      performance perhaps being restored via host configuration
      changes. For instance MTU settings.)

then I would not be concerned that the Statement could support
someone arguing that end-users should be required to adopt IPv6.

>> It seems that these things can all be achieved with an ITR - ETR
>> scheme as proposed in LISP, eFIT-APT and Ivip.
> 
> The problem statement document takes no position on any particular
> solution space.

I agree that it shouldn't.

>> There are two goals not mentioned in the current Problem Statement
>> which I think are achievable and essential for coping with
>> long-term growth and ubiquitous adoption:  More efficient
>> utilization of IPv4 space and Mobility, including Mobile IPv4.
> 
> Please justify each of these two positions. The document talks to the
> second point. I don't immediately see how a more efficient utilization
> of the IPv4 space helps. Indeed, it probably hurts. From the document:
> 
>    4.8.  IPv4 Address Exhaustion
> 
>    The IANA and RIR free pool of IPv4 addresses will be exhausted within
>    a few years.  As the free pool shrinks, the size of the remaining
>    unused blocks will also shrink and unused blocks previously held in
>    reserve for expansion of existing allocations or otherwise not used
>    due to their smaller size will be allocated for use.  Consequently,
>    as the community looks to use use every piece of available address
>    space (no matter how small) there will be an increasing pressure to
>    advertise additional prefixes in the DFZ.
> 
> It is pretty much an axiom that achieving higher utilization in the
> use of address space leads to increased fragmentation, and hence, more
> routes.

This is what I mentioned above - the LISP/eFIT/Ivip approaches all
enable a drastically more efficient utilisation of IPv4 address
space without adding prefixes to the BGP routing table.  (Ivip
adds one for each block of space which is assigned to it, but
supports many more than one end-users in that space.)

>> The currently low percentage of advertised IPv4 addresses which
>> are actually used is a result primarily of the address management
>> policies of assigning large chunks of space to all applicants.
>> This long-standing practice is based on the intention that
>> end-user won't need to ask for more space for a few years.
> 
> Um, it is RIR policy that an end site can obtain IPv4 space, even if
> it it never will be advertised on the public Internet. Thus, you
> cannot conclude that because only some fraction of the current
> address space is routed on the public internet, the remaining space is
> "unused".

Figure 5 "Current IPv4 Status" at:

  http://www.potaroo.net/tools/ipv4/

shows quite a lot of space, in orange, which has been assigned but
not advertised.

Figure 12 graphs the unadvertised space, which is growing more
slowly than the advertised space.  It is currently about 0.46  of
the advertised space.  So about 31% of assigned space is not
advertised.

I wonder why an end-site would want address space it was not going
to advertise, and why in an era of permanent shortage, RIRs would
allow space to be used in this way.  Couldn't those end-users be
expected to use private address space and let another end-user
have the publicly routable space, to advertise?

> Also, RIR policies use an 80% utilization metric. You cannot obtain
> more space until you have shown that you are actually using 80% of the
> space you already have.
> 
> Many (myself included) would argue that we have already got a rather
> high utilization out of IPv4. While we might be able to squeeze a few
> more drops of blood out of the IPv4 turnip, how much time does that
> actually buy us in the overall scheme of things? I suspect not much.

Perhaps this is where I misunderstand actual utilisation of IPv4
address space.

Does "80% utilisation" mean that 80% of the IP addresses actually
have a host, an ADSL modem/router etc. using them, at least some
of the time?

I wonder what fraction of the space actually has that rate of
utilisation.  Only a fraction of it would have, since much
assigned space would be either old (and often very large)
assignments or an assignment to an end-user or ISP which did prove
the 80% utilisation, but this new assignment generally won't be
that highly utilised yet.

In March this year there were about 1.695 billion IPv4 addresses
advertised.

I know some of this space is /8s with very low utilisation rates.
 I wonder how much of the advertised prefixes really come close to
the 80% utilisation rate.

It is my impression that a lot of this space is sparsely used.

There is no pressure about reclaiming sparsely used address space
this now - such as telling the assignee to keep a quarter of it,
move their hosts to that quarter, and use the rest for other
end-users - but I am sure there will be in two or three years time.

A recent JPNIC statement:

  http://www.apnic.net/news/2007/0626.html

may be foreshadowing this:

  To maintain the existing IPv4 based Internet, and to create
  smooth transition to IPv6, it is also important to recover and
  recycle unused IPv4 addresses.

>> I think it would be highly desirable to have some policy which
>> limits the IPv6 classification tasks of DFZ routers to something
>> like /32 or /35 at the most.  Then, the routers can be optimised
>> for handling the bulk of their traffic within these limits (which
>> are still really demanding).  This is not a goal below, but maybe
>> the new architecture needs to include administrative and other
>> arrangements which put reasonable limits on the FIB functionality
>> of routers, since they are so costly, use so much power, and are
>> going to be handling so many VoIP packets.
> 
> One small observation: There is no place to define such a policy that
> would actually be implemented by (or enforcable on) the DFZ
> routers. Remember, the DFZ routers are operated by ISPs, and what
> routes they carry, etc. is completely unregulated. Nobody can tell
> them what they will and will not route.

Maybe not a policy then, but sufficient ISPs deciding that they
don't want to be paying forever for overly complex routers because
a small amount of address space involves what they consider to be
unreasonably long prefixes.

>> Here is a rewrite:
> 
>>    There is a need for a new architecture for routing and
>>    addressing that:
> 
>>    1.  Reduces the growth rate of the DFZ routing load, where the
>>        routing load is dependent on:
> 
>>        A.  The number of individual prefixes in the DFZ.
> 
>>        B.  The update rate associated with those prefixes.
> 
>>        C.  The length of those prefixes.
> 
> We have not considered  the prefix length to be a significant
> problem. Can you you please expand on this point, and why it is
> significant enough to include?

I am not concerned about IPv4 traffic often requiring up to 24
bits of the address to be analysed.

I accept that IPv6 will generally involve looking at 32 to 36
bits, but I think the recent development of /48 end-user
assignments by ARIN, AfriNIC and yesterday RIPE means that IPv6
routers have to do a huge amount of work in their FIB to classify
each of these packets which match a /48.

If the router uses a TCAM wide enough to handle these bits in a
single cycle, then I agree it doesn't really matter. TCAMs were 36
bits wide and are now available 72 bits wide.

However the latest high-end routers don't use TCAM in their FIBs -
they use ASIC CPUs and RAM lookups.  The more bits which need to
be analysed, the more processing time and RAM lookups are
required.  The shared Dynamic RAM is the bottleneck of Cisco's 188
CPU system.

The initial brochure for Cisco's CRS-1 mentioned that its MSC card
could handle 40Gbps, but only for IPv4.  I discuss this
Tree-bitmap based CRS-1 MSC at:

  http://www.firstpr.com.au/ip/sram-ip-forwarding/router-fib/

You can see the "IPv4" proviso on page 20 of:

http://www.dfn.de/content/fileadmin/3Beratung/Betriebstagungen/bt45/forum-winip-irgens.pdf

  Silicon Packet Processor
    40G of line rate Forwarding capability (IP4)

This is with one or two gigabytes of DRAM, and an ASIC with 188
independent 32 bit CPUs.

IPv6's longer prefixes are hard work, and I think that any new
architecture should only increase the workload of FIBs with very
strong justification.  There was a lot said about FIB burdens in
the RAWS report and presentations.

>>        D.  The complexity and size of the RIB and the
>>            complexity and volume of inter-router communications.
> 
> The difficulty here is that this gets into router architecture,
> something way out of scope for the IETF "on the wire" focus. And the
> discussions that followed the RAWs report (to me anyway) make it clear
> that there is not really concensus that this component of "routing" is
> where the core problem really is. Or that is not a symptom of the core
> problem as outlined in the document.

It is my understanding that the primary reason a new architecture
is required is that the growth in the global BGP routing table
must be constrained in order to ensure that unsustainable burdens
are not placed on individual DFZ routers, particularly in terms of
the amount of state they hold in their RIB, their processing load
and their communications load.

The internal RIB functions of routers are defined, in large part,
by the protocols they must handle, which is under the control of
the IETF.  It would be easy to invent a more detailed form of BGP
message, to enable routers to make better decisions, but either
no-one would implement it, due to the typically 4Gbyte restriction
on RAM, or they would do so and all compliant routers would cost
much more than current routers, with older routers needing to be
completely replaced.

I think "complexity and volume of inter-router communications" is
important, since this is the real problem along with the amount of
RIB state - and since the current text:

     A.  The number of individual prefixes in the DFZ

     B.  The update rate associated with those prefixes.

doesn't cover how long the messages are for each update of each
prefix, or the amount of data they contain which needs to be
stored in the RIB.

>>        E.  The complexity, cost and power consumption of the FIB
>>            functions which handle ordinary routing tasks and any
>>            requirements of a new architecture, such as
>>            encapsulation, decapsulation, authentication and
>>            support for Path MTU Discovery when packets are
>>            tunneled.
> 
>>               (Actually, I think the new architecture needs a
>>                whole new system by which hosts can do a much
>>                better job of PMTUD than is currently possible.
>>                The new tunneling architectures will make PMTUD
>>                much harder, while the problems with MTU and
>>                fragmentation are going to be made worse.  So there
>>                is a lot more which probably should be added to the
>>                goals about this.)
> 
> I would argue that the above is covered by by section 3.1 and 3.2

I think this needs to be part of the formal Problem Statement,
because otherwise an architecture could be chosen which involves
excessively complex processing to achieve important things like
authentication of ICMP messages and filtering encapsulated packets
to drop those with spoofed local source addresses.  I have written
about both these problems on this list recently.  Just the
encapsulation of some proposals involves extra headers, nonces,
etc.  I think there needs to be a good justification for all this
extra work and extra bits in packets.

>>        F.  The costs and difficulties of administering routers
>>            and the routing system as a whole so routers operate
>>            efficiently and do not place unreasonable burdens on
>>            other routers.
> 
> 
>>    2.  Allows the following benefits for end-users, with the
>>        needs of larger organisational end-users being most
>>        important, but in principle extending to individual
>>        end-users's networks and single devices.  These benefits
>>        should support communications initiated by hosts in
>>        networks which have not adopted the new architecture -
>>        perhaps not with the same efficiency as to non-upgraded
>>        networks, or with as high an efficiency as between two
>>        hosts in upgraded networks.  Hosts in non-upgraded
>>        networks should suffer no loss of reliability in their
>>        communications with hosts in networks using the new
>>        architecture.
> 
> I think all of this is really getting more into desirable properties
> of a solution, rather than the problem.

I agree this concerns required or desirable properties of the
solution.  I think this is in keeping with the current form of
Section 6:

  The problem is that we need a solution which will have the
  following properties:

    1. .....
    2. etc.

My understanding is that most people in this field believe that a
new routing and addressing architecture needs to be developed in
the next two or three years and that it must be backwards
compatible by supporting connectivity to and from hosts in
non-upgraded networks.  I also think there is a consensus about
not requiring changes to hosts.

Ideally, that connectivity should be undiminished by the new
architecture, but I think that when any host sends packets to
another host whose address is mapped with LISP/eFIT-APT/Ivip, that
there are likely to be problems with MTU and fragmentation.

That is why I wanted to make it acceptable for a new architecture
to diminish efficiency somewhat - because I think all the
solutions (other than moderate improvements to BGP, which is not a
full solution) involve tunneling and so will have these MTU and
fragmentation problems.

  - Robin              http://www.firstpr.com.au/ip/ivip/

_______________________________________________
RAM mailing list
RAM@iab.org
https://www1.ietf.org/mailman/listinfo/ram