[rrg] CES & CEE: GLI-Split; GSE, Six/One Router; 2008 sep./elim. paper (v2)

Robin Whittle <rw@firstpr.com.au> Sun, 21 February 2010 15:00 UTC

Return-Path: <rw@firstpr.com.au>
X-Original-To: rrg@core3.amsl.com
Delivered-To: rrg@core3.amsl.com
Received: from localhost (localhost []) by core3.amsl.com (Postfix) with ESMTP id 205C828C145 for <rrg@core3.amsl.com>; Sun, 21 Feb 2010 07:00:09 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.443
X-Spam-Status: No, score=-1.443 tagged_above=-999 required=5 tests=[AWL=-0.148, BAYES_00=-2.599, HELO_EQ_AU=0.377, HOST_EQ_AU=0.327, J_CHICKENPOX_63=0.6]
Received: from mail.ietf.org ([]) by localhost (core3.amsl.com []) (amavisd-new, port 10024) with ESMTP id P1-Oq72qHjaW for <rrg@core3.amsl.com>; Sun, 21 Feb 2010 07:00:05 -0800 (PST)
Received: from gair.firstpr.com.au (gair.firstpr.com.au []) by core3.amsl.com (Postfix) with ESMTP id 27E253A7C8D for <rrg@irtf.org>; Sun, 21 Feb 2010 07:00:03 -0800 (PST)
Received: from [] (wira.firstpr.com.au []) by gair.firstpr.com.au (Postfix) with ESMTP id 0C355175C5C; Mon, 22 Feb 2010 02:01:54 +1100 (EST)
Message-ID: <4B814AE2.5010603@firstpr.com.au>
Date: Mon, 22 Feb 2010 02:01:54 +1100
From: Robin Whittle <rw@firstpr.com.au>
Organization: First Principles
User-Agent: Thunderbird (Windows/20090812)
MIME-Version: 1.0
To: RRG <rrg@irtf.org>
References: <4B7545E5.2070909@firstpr.com.au>
In-Reply-To: <4B7545E5.2070909@firstpr.com.au>
Content-Type: text/plain; charset="windows-1252"
Content-Transfer-Encoding: 8bit
Subject: [rrg] CES & CEE: GLI-Split; GSE, Six/One Router; 2008 sep./elim. paper (v2)
X-BeenThere: rrg@irtf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IRTF Routing Research Group <rrg.irtf.org>
List-Unsubscribe: <http://www.irtf.org/mailman/listinfo/rrg>, <mailto:rrg-request@irtf.org?subject=unsubscribe>
List-Archive: <http://www.irtf.org/mail-archive/web/rrg>
List-Post: <mailto:rrg@irtf.org>
List-Help: <mailto:rrg-request@irtf.org?subject=help>
List-Subscribe: <http://www.irtf.org/mailman/listinfo/rrg>, <mailto:rrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Sun, 21 Feb 2010 15:00:09 -0000

This is a revised version of my (msg06060), with new terms:

   SPI   Scalable Provider Independent:  "Edge" space handled by a
         CES architecture's ITRs, ETRs etc. and so which is Provider
         Independent (portable) and suitable for multihoming and
         inbound TE.

   MAB   Mapped Address Block:  A prefix advertised in the DFZ which
         covers a typically large amount of SPI space which is
         typically used for many (tens to hundreds of thousands)
         of individual SPI micronets (Ivip) or EID prefixes (LISP).
         Dino recently used the term "coarse prefix" to refer to
         the same thing in LISP.

   EUN     End User Network:  Any user's single host or network, from
           an IP-connected cellphone, to a single IPv4 DSL service to
           a university, corporate or government network.

   EUN-PI:  End User Network using (unscalable) PI space.

   EUN-PA:  End User Network using PA space - including home & SOHO
            DSL etc. end-users.

   EUN-SPI: End User Network using SPI space = "edge" space in a CES

   CES   Core-Edge (address) Separation: a subset of the global
         unicast address space is separated from the remaining
         "core" global unicast addresses and is supported by the
         CES system (ITRs, mapping system and ETRs) to be used as
         "SPI" space.  Also a class of scalable routing architectures
         in which this is done - including LISP, APT, Ivip, TIDR and

   CEE   Core-Edge (address) Elimination:  A class of scalable
         routing architectures in which the Locator / Identity
         Separation naming model is introduced, replacing the
         current IPv4/v6 model in which the IP address plays both
         the Identifier and Locator roles.  Includes ILNP, GSE,
         GLI-Split, Name-Based Sockets.  (Does not include LISP -
         LISP does not use Locator / Identity Separation.)

         Hosts have portability of Identifiers and are able to do
         multihoming and inbound TE on PA IP addresses, which are
         also known as "core" addresses.  Therefore there is no need
         for any "edge" addresses - the unscalable PI addresses
         which are currently the only way of achieving portability,
         multihoming and inbound TE, or any other type of IP address
         with special characteristics for EUNs.

   EUN-SPI Full Adoption:  (Applicable only to a CES architecture.)
         All EUNs which want or need portability, multihoming and/or
         inbound TE have adopted SPI space.  This includes any
         EUN-PI networks - so there are no more PI prefixes.  The
         CES architecture has achieved 100% of the routing
         scalability improvements it could possibly achieve.  Many
         hosts are still on ISP's prefixes - ISP's hosts and all
         EUN-PA networks - but these in no way constitute a scalable
         routing problem, since they do not burden the DFZ or result
         in any loss of portability, multihoming and/or inbound TE to
         any EUN which wants these benefits.  (Those which do are
         already EUN-SPI networks.)

   Universal SPI Adoption:  Every host - including those of ISPs and
         of those in what were formerly EUN-PI or EUN-PA networks -
         uses SPI "edge" space.  This is well beyond "EUN-SPI Full
         Adoption", and does not involve any further improvement to
         routing scalability.

   CENI  Core-Edge Network Isolation:  Once Universal SPI Adoption
         has been achieved, the routing system is altered so no host
         can send packets to any "core" address: any address of a DFZ
         or some other (ISP) routers, or of any ITR or ETR.  (This
         may not make much sense.)  Only APT had this as a goal, but
         it seems APT's designers, who made up many of the authors
         of the 2008 paper, thought (and perhaps still think) that
         other CES architectures such as LISP and Ivip had this goal
         too - but this is not the case.

Short version:   Core-Edge  Separation / Elimination
                 \________/ \______________________/
                 Adjectives  Verbs

                 Are the adjectives referring to:


                 In "Core-Edge Elimination", the adjectives refer to
                 addresses: there being no more need for "edge"
                 addresses, meaning today's unscalable PI addresses,
                 or any other kind of  address for EUNs.

                 If the adjectives referred to "networks", it would
                 make no sense, since neither the core network nor
                 edge networks are being eliminated - nor are the
                 distinctions between them being eliminated

                 If the adjectives in "Core-Edge Separation" are
                 assumed to refer to "addresses" then this is a
                 meaningful and useful term:  Separation of global
                 unicast addresses into "edge" and "core" subsets
                 makes sense and is a useful distinction for
                 categorising the CES class of scalable routing
                 proposals, the other class (CEE) eliminates the
                 need for "edge addresses".

                 If the adjectives referred to "networks", then
                 "Core-Edge Separation" makes no sense, since these
                 networks are already separate from each other and
                 will always remain so.

                 However, it seems that the term "Core-Edge
                 Separation" is sometimes used in a way which

                    Separation of a global unicast addresses into a
                    subset of scalable "edge" addresses, leaving the
                    remainder as "core" addresses.


                    Isolation of all "edge" networks from the "core"
                    network (the DFZ, ISP routers, ITRs and ETRs) so
                    that hosts in "edge" networks cannot send packets
                    to "core" addresses.

                 This leads to lots of confusion.

                 Comments on Lixia's and colleagues' 2008 paper
                 on Core-Edge Separation vs. Core-Edge Elimination.

                 I think they were mistaken to class GSE as a
                 Core-Edge Separation architecture.

                 I think this paper refers generally to the
                 core-edge "separation" being the separation of
                 the global unicast *address* space into "core"
                 and "edge" subsets.  This is the defining
                 characteristic of CES architectures.

                 However, I believe they were also thinking of, and
                 sometimes referring to with the same "core-edge
                 separation"  term, something very different:
                 Core-Edge *network* "separation", but really meaning
                 "Core-Edge Network *Isolation*".  This was a goal of
                 APT, but is not a goal of other CES architectures -
                 but the authors may have thought that other CES
                 architectures did have this goal.

                 Many of the authors of this 2008 paper were the
                 designers of APT, which separated not just the
                 address space into two subsets, but had a long-term
                 goal of actually "separating" (really *isolating*)
                 all the edge  networks as a set, from the core, so
                 no edge host could physically send a packet to any
                 core address.

                 I think this double sense of the term "separate":

                    Separation of global unicast address space into
                    "edge" and "core" subsets.

                    ("Separation" = Isolation) of all edge networks
                    from the core network.

                 plus the paper's mistaken classification of GSE as
                 CES contributed to the GLI-Split authors considering
                 their proposal to be a CES architecture too.  Yet
                 GLI-Split is clearly CEE, since it implements the
                 Locator/Identification Separation naming model.

                 These terminological and classification problems
                 are compounded by LISP being inappropriately named -
                 since it is a CES architecture which does not
                 implement Locator/Identifier Separation.

                 Six/One Router has some similarities to a CES
                 architecture, but uses address rewriting (AKA
                 "translation") rather than tunneling.  I think it
                 is not a proper CES architecture, in part because
                 hosts in upgraded networks can't be reached by
                 hosts in non-upgraded networks on their real
                 addresses, requiring DNS tricks to return different
                 IP addresses to hosts in non-upgraded networks,
                 which is the only way the latter type of host can
                 initiate communications with a host in a Six/One
                 Router network.

                 GSE and GLI-Split are translation-based CEE
                 architectures.  They are unusual among CEE
                 architectures in that they only require the
                 new router functions and changes to the stack -
                 they do not require applications to be modified.
                 (However I just learnt that GLI-Split could
                 support new application functionality via a new

                 Comparison charts of:

                 CES: LISP-ALT/NERD, APT, Ivip, TRRP, TIDR, and

                 CEE: GSE, GLI-Split, ILNP, Name-Based Sockets.

                 I believe the GLI-Split designers were mistaken to
                 class their proposal as a Core-Edge Separation (CES)
                 architecture.  I am sure it is a CEE (Core-Edge
                 Elimination) architecture.

Here is some discussion further to my understanding of the
distinctions between Core-Edge Elimination (CEE) architectures and
Core-Edge Separation (CES) architectures:

  CES & CEE are completely different (graphs)

Please also see these messages for discussion of the current IP
naming model, why I believe the IP address plays both the Identifier
and Locator roles - and how this is still 100% true when a CES
architecture such as LISP or Ivip etc. is used and the destination
address is an EID (LISP) or SPI address (Ivip):

  LEIDs, SPI & ordinary IP addresses as both IDs & Locs
  and other messages in this thread.

This paper:

  Towards a Future Internet Architecture: Arguments for Separating
  Edges from Transit Core
  Dan Jen, Lixia Zhang, Lan Wang, Beichuan Zhang

is really important to the RRG discussions, because it formalises the
architectural terminology for two important classes of scalable
routing solution.  To see my attempt to trace the origins of these
concepts and terms, please see:

  Re: [rrg] Terminology

I support this paper, but have some comments to make:

  1 - The definition of exactly what core and edge things are
      being separated or eliminated could be improved - to
      refer to separation into two subsets of addresses and to
      use a different term for what I refer to as "CENI - Core-
      Edge Network Isolation".

  2 - The suggestion that GSE is a CES architecture is mistaken.

  3 - The paper doesn't mention what I consider to be the biggest
      objection to CEE: that it implements the Locator/Identifier
      Separation naming model which will generally slow down the
      establishment of communication sessions and place extra
      burdens on all hosts - especially those on slow, unreliable
      wireless links.

The paper refers in general to:

    "separating edge networks from the transit core"

but really both CEE and CES maintain the already existent separation
between edge networks from the transit core.  (See discussion of
section 4.3 below for my understanding of why the authors may have
emphasised the "separation of networks" (really CENI) rather than the
separation of a new subset of scalable "edge" addresses from the
remaining "core" addresses.  They intended APT to ultimately
"separate" (really *isolate*) the two systems so much that an edge
host would be physically unable to send a packet to a core router.
They also seem to have assumed that LISP and other CES architectures
were intended to do this as well, which is not the case

What is being separated in a CES architecture is a new, scalable
"edge" subset of *addresses* (SPI addresses in Ivip - EID addresses
in LISP) from the remainder of the global unicast addresses, which
are then known as "core" addresses.  This is a different concept from
"separating edge *networks* from the core", since the edge networks
are already separate from each other and separate from the DFZ, and
this use of "separation" really means isolation between things which
are already separate.

At present, any EUN which is actually getting portability,
multihoming and inbound TE is getting it with PI prefixes - which are
advertised in the core routing system (DFZ) just like ISP's prefixes.
 While these PI prefixes are for "edge" networks (since these are
EUNs, rather than ISP or transit provider networks), these PI
prefixes are part of the "core" address space, since all DFZ routers
need to develop best paths for each such PI prefix.

>From the abstract, it can be seen that the CES definition is not the
opposite of the CEE definition:

CES:    The first direction, which we dub separation, calls for
        separating edge networks from the transit core, and
        engineering a control and management layer in between.

CEE:    The other direction, which we dub elimination, calls for edge
        networks to adopt multiple provider-assigned addresses to
        enable provider-based address aggregation.

(See below for discussion of the paper's more formal description of CEE.)

In fact, the CEE definition could reasonably include CES
architectures, since CES architectures generally use multiple ETRs,
each on a PA ("provider-assigned") address, for the purposes of
enabling the DFZ to only handle prefixes which are subject to
"provider-based address aggregation".

The most important architectural distinction between CES and CEE is:

  CES involves end-user networks' hosts using global unicast
  addresses from a specially handled "edge" (AKA SPI, AKA EID) subset
  of the global unicast address space, with the remainder being known
  as "core" addresses.

  Global unicast IP addresses in both the "core" and "edge" subsets
  will continue to be used, as are all today, both for Identifying
  hosts and for the Routing system to decide how to forward packets
  (i.e. as a "Locator").  See msg06070 and msg06082.

  But note that CES has a new, fancy, way of implementing the
  Locator semantics of destination addresses whose IP address is
  in the new scalable "edge" (SPI, EID) subset.  msg06082 has a
  detailed example of this 2nd algorithm, which is only performed
  by ITRs.  The original 1st algorithm, as used by all routers except
  ITRs, still works as normal on these "edge" destination addresses.

  CEE involves end-user networks' hosts using something other
  than a global unicast IP address to Identify themselves.  Thus
  CEE implements the Locator/Identifier Separation naming model.
  Applications use this new Identifier, which is in a different
  namespace to IP addresses and any Locators used in the system,
  to identify each host.  Hosts can be reached via one or more
  potentially unstable Locators, and the applications are unaffected
  due to their reliance purely on Identifiers to specify which hosts
  are communicating.

CES preserves the existing IP naming model:

       Role             Level

       Text name  <---- FQDN
       Identifier <---] IP address
       Locator    <---]

while CEE creates one of several models, in which the Locator and
Identifier roles are played by different levels - different objects
in different namespaces.

CEE architectures always implement Locator/Identifier Separation.

No CES architecture implements Locator/Identifier Separation.  What
they separate is two subsets of the global unicast address space:
"edge" from the remaining "core".  Both types of address are used
both as Locators and as Identifiers.

Both subsets are used as Identifiers by all hosts.  Hosts make no
distinction between addresses in the two subsets.

Most routers make no distinction between how they handle packets with
destination addresses in these two subsets.  The exception is ITRs,
which make a big distinction.  ITRs implement the Locator
functionality of destination "edge" addresses in a totally different
way: by looking up mapping and tunneling the packet to an ETR, rather
than by forwarding the packet to a neighbour.

More on naming models:

  Today's "IP addr. = ID = Loc" naming model should be retained

CES architectures never require changes to host stacks or applications.

Many CEE architectures require changes to both stacks and apps, but
at least two - GLI-Split and I think GSE - only require changes to
the stacks.

Here are some cases:


      There's no debate about this - they all create a special
      "edge" subset of the global unicast address space for EUNs
      to use for portability, multihoming, TE and perhaps mobility
      - and add things to the routing system to scalably get packets
      addressed to "edge" addresses to the destination networks
      without excessive burdens on the DFZ control plane.

      Except for Ivip's long-term "Modified Header Forwarding" modes,
      all these use encapsulation in order to tunnel packets from
      an ITR or ITR-like device to an ETR or ETR-like device.  This
      raises some thorny PMTUD problems.

      These CES architectures are all intended to work for both IPv4
      and IPv6.  The descend directly or indirectly from the original
      map-and-encaps architecture.  No host stack or application
      changes are required.  Things are added to the routing system -
      but in general routers stay the same.

      These will work in  principle and in practice for both IPv4 and

CES-like - Six/One Router. See footnote.

CEE: GSE - Mike O'Dell, 1997


     This is an extension of his earlier 8+8 architecture - both are
     for IPv6 only due to the way multiple items are combined to
     create the final IP address.

     GSE is clearly a CEE architecture because it has separate
     objects for Identifying hosts and for Locating them.  There is
     no separate "edge" subset of the global unicast address space by
     which hosts in EUNs identify themselves.  The Identifier is the
     64 bit ESD.

     There needs to be new things in the network: an address
     rewriting function in the BRs of EUNs which adopt GSE.

     Host stacks needs to be somewhat modified, but I think this
     is mainly so header checksums are computed only on the ESD
     and not on other parts of the IPv6 address.

     As far as I know, applications do not need to be modified.

     Multihomed EUNs get a prefix from each of their two or more
     ISPs, and the identity of hosts is unrelated to this, or to
     any IP address.  These prefixes are provided by the ISP as
     PA space.  The ISP typically has a single large (short) prefix
     which it splits into many such PA prefixes for many such EUNs.
     Host Identity - and so all communications establishment and
     session continuity - is determined by the value each host uses
     for the 64 bit ESD, which exists in a separate namespace from
     IPv6 addresses.

CEE: GLI-Split


     I believe GLI-Split is a CEE architecture, because it introduces
     a separate "ID" object (Figure 2), with a separate namespace
     from IPv6 addresses or anything else, to Identify each host.
     This is different from the objects used to Locate them: the
     GL or LL Locators.

     The fact that both objects are combined with other things to
     form an IPv6 address does not affect the fact that these objects
     for Identifier and Locator are independent, and in separate
     namespaces, just as they are in ILNP, GSE and the other CEE

     GLI-Split implements Locator/Identifier Separation, as do all
     other CEE architectures.  From the Abstract:

         It implements the locator/identifier split and makes routing
         in the core of the Internet more scalable.

    More on GLI-Split at the end of this message.  The authors
    explicitly state that it is a CES architecture, but I am sure
    it is a CEE architecture.  I think they are following a mistake
    made which I believe was made in the the 2008 sep./elim. paper.
    See also my draft critique of GLI-Split and what the designers
    will hopefully soon write in response: (msg06056).

CEE: ILNP - Ran Atkinson.

     This is clearly a CEE proposal, because it has separate levels
     for Identifier and Locator.  It is a purely host-based
     descendent of GSE.

     There is no need for new routers, but the host stacks and
     applications all need to be upgraded, because the applications
     use Identifiers and Locators separately via the API to the
     stack (or do the applications use only Identifiers?).

     As with GSE and all other CEE architectures, a multihomed
     network uses two or more PA prefixes from its two or more
     ISPs, and the PA nature of those prefixes means each ISP can
     supply many such prefixes from a single prefix the ISP
     advertises in the DFZ.

CEE: Name Based Sockets - Christian Vogt.

     This is a CEE architecture because the Identity of each host
     is a FQDN and its one or more Locators are IPv6 addresses,
     again drawn from PA prefixes of the EUNs one or more ISPs.

     Name Based Sockets is unusual for a CEE architecture in that
     it appears to have a two-level naming model.  However, I am
     not sure how it could have this and still be able to support
     DNS lookups leading to multiple separate hosts.  I hope
     Christian will clarify this.


     See my recent questions and Xiaohu Xu's reply.

???: Name Overlay Service (NOL)

     See (msg06062).  Neither CEE nor CES.  It is NAT-like and I
     do not think it is at all practical.

???: hIPv4 - Patrick Frejborg

     Now in version 05, which is different from what I critiqued.
     See: "hIPv4-05: new header" thread.

     I previously thought it was a CEE but I think it is neither
     CEE or CES.

???: Aggregation with Increasing Scopes.

     I don't yet understand this, but the authors state that the
     end-result somewhat resembles their previous APT CES

Here are some charts:

CES            IPv4   IPv6   Tunneling  Local            Mapping
architecture                            full-            speed

LISP-NERD        v4     v6   Encaps.    ITRs are full    Slow

LISP-ALT         v4     v6   Encaps.    No               Global

ALT              v4     v6   Encaps.    Yes              Slow

Ivip             v4     v6   Encaps.    Yes              Fast
                             or MHF

TRRP             v4     v6   Encaps.    No               Global DNS

TIDR             v4     v6   Encaps.    ITR-like         Slow - via
                                        DFZ routers      DFZ control
                                        have full DB.    plane.

IRON-RANGER      v4     v6   Encaps.    (Neither question applies
                                         directly to IRON-RANGER.)

CEE            IPv4   IPv6   App       Stack      Router
                             changes?  changes?   changes?

GSE                     v6   -         Stack      Router rewrites

GLI-Split               v6   -*        Stack      Router rewrites

ILNP                    v6   App       Stack      -

Name Based
Sockets                 v6   App       Stack      -

* GLI-Split does not require host changes, but Michael Menth wrote
  to me off-list that new applications could be used, for instance
  for multipath protocols, if the stack had a suitable new API:

     "Applications usually communicate with identifier addresses,
      but also a more advanced API must be offered in addition to
      allow active multipath routing through applications or at
      least by transport layer protocols.

  He indicated that this was not in the current documentation but
  would be added in the future.

Back to the 2008 paper: on page 2, 2nd last paragraph:

     There are also other types of separation
     solutions besides Map & Encap. For example,
     Six-One Router [23] and GSE [19] use address
     rewriting, which rewrites the packet header
     to include information about the
     destination’s attachment point to the transit

For the reasons listed above, I believe GSE is a CEE architecture.

The fact that the somewhat CES-like architecture Six/One Router has
routers rewriting addresses does not mean that GSE is a CES
architecture simply because it too has routers rewriting addresses.

In Section 3, there is a more formal description of CEE:

     In order to achieve routing scalability, the
     elimination approach enforces provider-based
     address aggregation by eliminating all PI
     prefixes and de-aggregated PA prefixes.

Note that the verb "eliminate" has an object: a class of *address*.
This is congruent with my claim that the important CES and CEE
distinctions concern separation or elimination of classes of address
- not "separation" (meaning isolation between) two classes of *network*.

I wouldn't state it so strongly - a CEE architecture doesn't
"enforce" anything or forcibly "eliminate" PI prefixes.

I think it is more accurate to say that when all hosts in all
networks adopt CEE, the CEE mechanisms can be relied upon entirely
for portability, multihoming and inbound TE.  Then, and only then,
this means that no network needs PI space to achieve these things.
This doesn't mean that PI space (the current, unscalable, form of
"edge" space) would automatically be eliminated - just that it could
be eliminated without any network missing out on portability,
multihoming or inbound TE for all its hosts and communications.

     Each multihomed edge network will receive
     from each of its providers an address block out
     of a larger, aggregated block announced by the

I would rephrase it slightly.  Each EUN which adopts CEE would
generally use space like this.  Each such EUN *could* use PI space.
Its just that PI space is no better than PA space for use with a CEE
architecture.  (I am including in my definition of "PI" usage a EUN
using a prefix it gets as PA space from one ISP via another ISP as a
"more specific PA prefix" - which also has the effect of adding
another prefix to the DFZ control plane.)

The degree to which they can multihome etc. depends on how many of
the hosts they are communicating with are also using the CEE
architecture, as noted in 4.1.

     The multihomed site does not inject PI prefixes
     or more specific PA prefixes into the routing
     system. Instead, each host in a multihomed site
     is given multiple PA addresses. For example, as
     shown in Figure 2, the host obtains two addresses,
     one from each of its network’s ISPs.

Typically, this is what would happen.

The "elimination" in this description refers to the elimination of a
class of address space, the current unscalable "edge" subset which is
primarily PI prefixes and also, to some extent (I don't know how
much) these "more specific PA prefixes".

The paper's initial use of "separation" was not in terms of address
space, but in terms of separating edge *networks* from the core.
>From the abstract:

   The first direction, which we dub separation, calls for separating
   edge networks from the transit core, and engineering a control and
   management layer in between.

Edge networks are already separate from the core.  This initial use
of "separation" probably was intended to mean "isolating all edge
networks from being able to send packets to any core address" - which
is clearly something the APT designers intended for APT.  It seems
the authors also thought that all other CES architectures had the
same intention - but this is not the case.

So I think the authors at times conflate two completely different things:

   CES   Core-Edge (address) Separation
   CENI  Core-Edge Network Isolation

referring to both with the same term "Core-Edge Separation" and
furthermore imply that CES architectures other than APT - LISP, Ivip
and others - also intend to do both of the above.

Nonetheless, most of their usage of "Core-Edge Separation" seems to
refer to the separation of a new, scalable, "edge" subset of the
global unicast *address* space from what remains as the "core" - and
does not refer to the "Separation" (isolation between) core and edge

In section 4.3, I think I see why the authors refer more to
separating edge networks from the core.  I think what they are
referring to is the end-point of APT adoption:

  1 - All EUNs use APT, and so use the scalable "edge" subset of
      the global unicast address space.  Furthermore, all hosts in
      ISPs which need to communicate with hosts in EUNs are also
      using SPI space.

      This is not just what I now call: "EUN-SPI Full Adoption".

      It goes well beyond this - well beyond what is required to
      maximise scalable routing benefits - to what I call "Universal
      SPI Adoption".

  2 - Therefore, no EUN needs PI space or "more specific PA
      prefixes".  So any space which was previously a PI prefix
      has been converted either to the new scalable "edge" subset
      of space or to the remainder - the "core" subset.

  3 - Therefore, ISPs advertise prefixes into the DFZ which are
      all in the "core" subset.  ISPs use these addresses for
      ITR and ETRs, and for internal and DFZ routers.

  4 - Because all ISPs and EUNs use APT, there is no need for
      APT's equivalent of LISP's Proxy Tunnel Routers or Ivip's
      DITRs.  (APT's arrangement was never well documented, but
      it too could provide full support for packets sent from
      non-upgraded networks.)  Therefore, there is no need to
      advertise any "edge" space in the DFZ.

  5 - Now, if it was desired - as the APT designers desired -
      it would be possible, in theory, to prevent any host
      in an EUN from being able to send a packet to any host
      or router on a "core" address.  Hosts in EUNs only need
      to communicate with other hosts in EUNs.  (Traceroute
      and ping wouldn't concern DFZ routers, since all packets
      between EUN hosts would be tunneled across the DFZ.)

      This is what I now call "CENI - Core-Edge Network Isolation".

Most of the APT designers wrote this 2008 paper.

AFAIK, no other CES architecture had this complete, physical,
(separation) isolation of EUN hosts or routers in EUNs - all on
scalable (SPI = EID) "edge" addresses - from being able to send or
receive packets to or from "core" addresses as a goal.  I specify it
as a non-goal for Ivip.

>From section 4.3:

   Separating edges from the transit core provides
   additional features that are sorely missing in
   today’s Internet.  With separation, an end host
   can send packets through the transit core, but
   can no longer address a packet to any specific
   device inside the transit core. Although the
   separation does not eliminate any specific
   security threat, it raises the bar against
   malicious attacks targeted at the global routing
   infrastructure. In addition, the mapping layer
   between edge and core networks can serve as a
   mounting point for badly-needed control and
   protection mechanisms, and can also act as a
   cushion layer between the edge and core,
   allowing each side to deploy innovations without
   any involvement of the other side. We now
   elaborate on each of these benefits.

They go on to discuss "Rolling out new protocols", "DDoS
mitigation" and "Ingress Traffic Engineering" - all of which they
argue would be possible or easier only with complete "separation" of
the "edge" networks from the "core" networks.

I think "Ingress Traffic Engineering" is possible with any level of
adoption of APT (assuming that APT, like LISP with PTRs and Ivip with
DITRs supports packets sent from non-upgraded networks, which it can)
- so this depends on Core-Edge (address) Separation, but not on
"EUN-SPI Full Adoption".  Support for "Ingress Traffic Engineering"
certainly doesn't depend on "Universal SPI Adoption" or "CENI
Core-Edge Network Isolation".

   Side-note on Ingress Traffic Engineering:

      Inbound TE works fine due to any amount of adoption of the
      CES architecture.  As long as the CES architecture supports
      all packets sent from non-upgraded networks, inbound TE will
      be for packets arriving in all the adopting networks, no
      matter how many other networks have so-far adopted the CES

      What the authors write in the second paragraph about Site1
      having preferences for which of Site2's two or more ETRs to
      use doesn't strike me as part of most CES architectures.  I
      think the aim of most CES architectures is to to have the
      recipient site control which of its ISPs the packets arrive
      via.  This is inbound TE.  Maybe APT had some concept of the
      sending EUN controlling this, but I guess most sending EUN
      wouldn't care which ISP used by the destination EUN the
      packets went through.

In their discussion of the other two goals: "Rolling out new
protocols" and "DDoS mitigation", to the extent that these goals are
facilitated by any kind of core-edge separation, they too are
dependent on "Core-Edge (address) Separation" - and do not depend at
all on "EUN-SPI Full Adoption", "Universal SPI Adoption" or "CENI
Core-Edge Network Isolation".

So I think the phrase at the start of this paragraph "Separating
edges from the transit core" is misleading if it means "isolating"
edge networks from being able to send packets to core addresses.  It
would be better to use the phrase: "Deploying an architecture in
which a scalable subset of 'edge' space is separated from the
remaining 'core' global unicast addresses" - AKA "Deploying a CES

I think the discussion in section 5 about multipath being used with
CES is not really about multipath as some other people conceive of
it.  My understanding of true multipath, such as MPTCP, is the
sending host (or perhaps a router)and perhaps the receiving host (or
its router?) being able to choose which of multiple ISPs in the
sending host's network packets may go out from, at the same time as
choosing which of multiple ISPs in the recipient host's network the
packets will travel through.   With CES, I don't see how a sending
host could affect either.

The routers in the sending host's network can choose which ISP link
to send out the packets on (assuming the ISPs accept packets with
"edge" addresses, which they will have to when CES is widely used).
If those routers are ITRs, then in some or many CES architectures
(but not Ivip) then perhaps the ITR might be locally programmed to
prefer one of multiple destination ETRs over others - so in principle
the ITRs in the sending network could affect the incoming load
balance of the destination network.  However, as far as I know, the
intention with all CES architectures is to allow the sending network
to choose the outgoing ISP and the recipient network to choose
(inbound TE) its incoming ISP(s).  In the case of Ivip, the recipient
network only provides a single ETR address, so there is no option for
the sending network to have its ITRs tunnel traffic to some other ETR
of the destination network.

I think this paper doesn't mention what I regard as the greatest
objection to CEE architectures - their adoption of the
Locator/Identifier Separation naming model, as I discuss in (msg05864).

Many of the authors of the 2008 paper also wrote the 2009 RRG
proposal: AIS / Evolution AKA "Aggregation with Increasing Scopes: An
Evolutionary Path Towards Global Routing Scalability".  In this they
seem also to be conflating CES (address) with what I now call CENI -
Core-Edge Network Isolation.  I will write more about that in my
forthcoming discussion / critique of the AIS proposal.

I think the GLI-Split authors were mistaken to state that GLI-Split
is a CES architecture.

The summary:


includes text indicating that perhaps the authors consider GLI-Slit
to be a CES architecture:

    GLI-Split implements a separation between global routing
    (in the global Internet outside edge networks) and local
    routing (inside edge networks) and using global and local
    locators (GLs, LLs).

Yes, but all EUNs have separate routing from the DFZ - so this is not
 a function of GLI-Split.

     o  Hierarchical aggregation of routing information in the
        global Internet through separation of edge and core

These phrases seem to imply that GLI-Split is a Core-Edge Separation

Back to the GLI-Split paper, there is explicit recognition of how it
involves the separation of Locator and Identifier functions.  This is
a feature of all CEE architectures, and is not (despite LISP's name)
something which CES architectures do:

    Separating current IP addresses into two independent
    pieces of reachability and identification information
    helps to reduce this growth and is called locator/
    identifier split (Loc/ID split) [4]. The stable
    identifier (ID) gives a global name to a node. A
    changeable locator (Loc) describes how the node can
    currently be reached through the global Internet.
    Furthermore, a mapping system (MS) is needed to map
    locators to identifiers. This principle makes routing
    in the stable Internet core more scalable because core
    routing is not affected by changed attachment points
    and multihoming of edge networks. The deployment of
    Loc/ID split in the Internet requires modifications to
    the current routing and addressing architecture.

In section 6, there is a further mention of "separation" of
"networks" rather than of the scalable "edge" address subset.  This
is a reference [32] to the 2008 sep./elim. paper and so simply
repeats its terminology:

    The authors of [32] identified two different
    strategies: separation of core and edge networks
    and elimination of de-aggregated provider-independent
    and provider-aggregatable addresses from BGP routing

The next paragraph continues:

    Proposals implementing separation can be subdivided
    into address rewriting, map-and-encaps, and source
    routing approaches.

Except for the mention of "source routing" "separation" proposals
(which I don't understand at all), this repeats the 2008 paper's
taxonomy of considering a proposal which does address rewriting as a
"separation" (CES) architecture: both Six/One Router and GSE.

But I am sure it is a mistake to characterize GSE as a CES
architecture, which is stated two sentences later.

The GLI-Split designers state that their architecture was "evolved
from the early ideas of GSE".  (Quotation below.)  They then assert
that ILNP and Six/One router are likewise "evolved from the early
ideas of GSE".  I agree this is the case with ILNP - and Ran Atkinson
states this clearly: http://ilnp.cs.st-andrews.ac.uk/

However, I disagree that Six/One Router "evolved" from GSE.  There is
no mention of GSE in Christian Vogt's paper.  While both Six/One
Router and GSE involve address rewriting at the borders of EUNs,
there are fundamental differences between Six/One Router and any CEE
architecture, as mentioned in the footnote below.

GSE is clearly a CEE architecture and is implementing
"Locator/Identifier Separation".

Six/One Router is not.  The word "separation" only appears in the
references of the Six/One Router paper.  The words "Locator" and
"Identifier" don't appear at all.

    With address rewriting, border routers add global
    locator information to packets destined for a
    different domain by coding this information into
    source and destination addresses for transit purposes.

    GLI-Split falls into that class.

GLI-Split does address rewriting, as does GSE (CEE, but mistakenly
classified as a CES architecture) and Six/One Router.

But this does not mean that GLI-Split is a CES architecture.  It is
clearly a CEE architecture.

    Also Six/One Router [9, 33] uses address rewriting.

This is true.

    Identifiers are only locally routable addresses.

This statement appears to be about Six/One Router, but it cannot be
true to say this of Six/One Router.  The term "Identifier" does not
appear in the Six/One Router paper I am quoting from, which I recall
is the latest version.  The last time we discussed Six/One Router on
the RRG was in August 2008, referring to this no-longer existent file:


I don't seem to have an electronic copy of this, but Christian
announced it on 2008-07-12:


and the MobiArch08 paper I am quoting from:


was last updated on 2008-07-24, so I figure this is the final version.

I think it is wrong to state: "Identifiers are only locally routable

The "edge" addresses used by hosts in Six/One Router networks (which
I guess is what the GLI-Split authors were referring to) are indeed
only "locally routable" in a direct sense, but they are globally
unique, are returned by any DNS query from within any Six/One Router
network.  In a sense these addresses are "globally routable" in that
packets with these source and destination addresses can be
transported to any Six/One Router network in the world, via the
writing and rewriting of the upper 64 bits.  This in no way makes
them "Identifiers" - they are IP addresses and hosts and at least
some routers treat them just like IP addresses today: both as
Identifiers and Locators.

   When communicating with nodes in different domains,
   the addresses are rewritten 1-to-1 through stateless
   NAT in border routers to globally routable transit
   addresses. A major focus of Six/One Router is
   improved multi-homing support.

This is all fine.

Then there is some discussion of ILNP, which looks fine to me.  Then:

   GLI-Split, ILNP, and Six/One Router have evolved from the
   early ideas of GSE (global, site, and end-system address
   elements) [36, 37].

This is true of GLI-Split and ILNP, but not of Six/One Router.

   It essentially codes a global locator, a local
   locator, and an identifier into an IPv6 address.
   Addresses are dynamically combined from these parts.
   It uses only the identifier for TCP checksum
   calculation and requires host upgrades for deployment.

This is a brief description of GLI-Slit, but it is a mistake to think
GSE, ILNP or GLI-Split are Core-Edge Separation architectures.

 - Robin

Footnote on Six/One Router:

CES:  Six/One Router - by Christian Vogt, though AFAIK it is no
      longer under development.


      I previously stated that this was "clearly a CES architecture"
      but now I think it is not one, despite some resemblances.

      Between hosts in upgraded networks, it appears these end-user
      networks operate from a single prefix drawn from an "edge"
      subset of the global unicast address range.  (Sect. 2.1.)

         Figure 1 illustrates this setup and addressing for an
         edge network that is multi-homed with two providers.  This
         edge network has one border link to each provider, so two
         Six/One routers are in use. The edge addresses are from the
         prefix ABC::/64. The Six/One router on the border link to
         provider 1 translates between those and transit addresses
         from the prefix 1000::/64; the Six/One router on the border
         link to provider 2 translates between the same edge
         addresses and transit addresses from the prefix 2000::/64.

      A host HA in one Six/One Router network, addresses packets to
      a host HB in another Six/One Router network another using the
      same IP address in the destination field as HB uses as its
      own "edge" address.

         Edge addresses are for use within this and other upgraded
         edge networks.  They are globally unique, but do not appear
         in the global routing table because they cannot be
         efficiently aggregated.

      All hosts in Six/One routers use a form of DNS which returns
      these "edge" addresses for any host in any Six/One Router
      network.  So this is a genuine scalable "edge" subset of the
      global unicast address space.

      However, because Six/One Router has no equivalent to DITRs
      (Ivip) or PTRs (LISP), hosts in ordinary IPv6 networks can't
      send packets to the Six/One Router hosts using these scalable
      "edge" addresses.   So, from a non-Six/One Router network, a
      DNS lookup on the same FQDN of host HB will not return its
      "edge" address, but one or more another addresses, based on
      one or more addresses within the PA prefixes HB's network
      obtains from its one or more ISPs.  The address-rewriting
      router at the border of HB's network will rewrite the
      destination address of a  packet coming from some ordinary IPv6
      host HX, to the "edge" address of HB.

      This is explained in section 2.4.  No other CES architecture
      has this "split-horizon" (I think this is the right term) DNS
      arrangement.  I guess it was impossible to implement an
      equivalent of DITRs and PTRs.

      So there isn't a single address by which a host in a Six/One
      Router network can be contacted or communicated with by both
      other such hosts, or hosts in ordinary IPv6 networks.  To those
      hosts in ordinary networks, the "edge" subset of addresses
      doesn't work.  No other CES architecture is like this, and I
      now think it is not correct to think of Six/One Router as being
      as much a CES architecture as LISP, APT, Ivip, TRRP, TIDR or

      The hosts in Six/One Router behave as usual.  There are no
      stack or application changes.  The naming model used in the
      packets they send and receive is the same as with today's IPv4
      and IPv6, as illustrated above.

      Rather than tunnel packets across the DFZ with encapsulation or
      Modified Header Forwarding, Six/One Router uses address
      rewriting (AKA address translation) at EUN BRs or perhaps
      in a router in their ISP(s).

      Unfortunately, Six/One Router can't be used for IPv4 due
      (I guess) to the more chaotic EUN prefix lengths and (I am
      sure) due to the need for a 2 ISP multihomed EUN to consume 3
      times the amount of global unicast address space it really
      needs.  (One for its "edge" prefix and then the same amount
      of PA space from each of its ISPs.)  Also, when Christian and
      I last discussed it on the RRG, I recall that he wrote that he
      needed a flag bit from the IPv6 header, which would require a
      modified IPv6 header format and so (I guess) require updates to
      all DFZ and many other routers.