Re: [lisp] LISP Interworking: Proxy Egress Tunnel Routers

[Robin Whittle <[e-mail]>]

Short version:    What are the business models for PETRs?

                  When would PETRs be used?  Darrel's text mentions
                  "LISP-NR" sites, but these are sites with EID
                  addresses not covered by a PTR advertisement.  I
                  can't imagine why any end-user site would want
                  to be a LISP-NR site, since this means that their
                  hosts would unreachable from all non-LISP networks.

                  PETRs have nothing much in common with ETRs.  How
                  can the ITR, or whatever it is which sends packets
                  to the PETR, make sure the PETR is alive, getting
                  most or all its packets and is able to handle them?

Hi Noel,

Thanks for your reply, in which you wrote:

>     > AFAIK, you haven't specified the commercial relationship between the
>     > operator of the LISP node and the operator of the PETR which it uses.
> 
> Right, but obvious commercial arrangements are outside the IETF scope. If
> your point was that you can't see that there will be any economically
> workable arrangement, that I think would be something that should concern us.
> As to that, I have no idea...

I understand that specific business arrangements are not a matter for
the IETF, but in developing a scalable routing proposal, we
definitely need to plan the types of business relationships which we
anticipate will develop to support the technical arrangements we are
devising.

I am not saying there isn't a viable business model for PETRs.  I am
just saying that I don't know what the business model is for PETRs
and that I think that at least one seemingly viable model should be
proposed before the whole PETR concept is accepted.


> > Do you plan to have provision for two or more PETRs, for redundancy?
> 
> You mean, 'would each non-LISP destination site be served by two or more
> PETRs'? Or did you mean 'would each LISP source site be served by two or more
> PETRs'?

The latter, since any PETR can send packets to all non-LISP sites.

> Because I think the original post said something about wanting PETRs
> scattered around the system, to minimize extra hops. If a PETR is close to
> either the source, or the destination, it will minimize the path length.

Darrel wrote:

    In order to minimize stretch on communications between LISP
    and non-LISP sites, its very likely that they should be located
    in separate places in the topology.

but there is no reference to the location of PETRs in the text he
proposed for discussion.

Since the typical situation is for a given source to be sending
packets to destinations all around the Net, the only way to minimise
stretch (longer total packet paths than would be required without the
PETR) is to locate the PETR close to the source.

So my question was whether a given LISP site using PETRs would have
two or more of them.  Implicitly, it would be best if they were close
to the site.

This raises questions about business models.  Since the purpose of
LISP or any other core-edge separation solution to the routing
scaling problem is to allow each end-user network to keep its address
range and use any ISP in the world, if the PETR is outside its
current ISP and needs to be "close", then we need to think about
business arrangements by which the site chooses one or more PETRs.

Part of this involves the possibility of two or more PETRs, both
ideally nearby (but not absolutely necessarily nearby) which are
operated by different companies and therefore are likely to be
different networks - to provide some network-level and business-level
redundancy.  The network-level redundancy is obvious.  The
business-level redundancy is to do with what happens if there is an
administrative glitch, or the operator of the PETR decides the site
hasn't paid its bill.

The business arrangements for the TTR model are discussed in:

  http://www.firstpr.com.au/ip/ivip/#mobile
  TTR Mobility Extensions for Core-Edge Separation Solutions to the
  Internet's Routing Scaling Problem
  Robin Whittle, Steven Russert 2008-08-25


> Which model you pick probably depends on the economic relationship; i.e.
> whether a PETR is associated with a source, or a destination. For the former,
> you'd want it close to the source so it can serve multiple destinations
> without increased path length, and vice versa for the other relationship.

Yes - I can't see any sense in having multiple PETRs distant from the
 LISP site, with the LISP site's router having to choose between them
for packets depending on the physical location of the destination of
those packets.

> Anyway, getting back to your original point, the system should _allow_
> either, but I'm not sure it should _mandate_ anything.

I imagine the LISP PETR specification will allow all sorts of things,
but the key thing is to show the desired arrangements are attractive
from both technical and business perspectives.


> > I understand that the concept of a PETR is quite separate from that of
> > the one or more ETRs a LISP node may be using. For LISP nodes which
> > need a PETR, does the LISP node perform its own ETR function?
> 
> The terminology may be a little confusing. PETRs serve non-LISP destinations,
> i.e. they only handle traffic flowing from a LISP site, to a non-LISP site.
> The ETR of a LISP site would, instead, be associated with traffic coming to
> the LISP site. How the ETR service for the LISP node is handled is not
> something this proposal needs to cover.

OK.

> > In the PETR model, I understand the LISP node simply encapsulates
> > packets to send to the PETR
> 
> The 'LISP node' might be either i) a LISP site, in which unmodified hosts are
> sending packet to an ITR which is doing the LISP encapsulation, or ii) some
> other source of LISP-encapsulated packets. Again, the detail there, of where
> the LISP-encapsulated packets are coming from, is not something this proposal
> needs to cover.

I don't understand either i) or ii).  I think the PETR specification
needs to identify when PETRs would be needed and when not.  As part
of this it should give some principles and ideally examples of the
sorts of "LISP-node", "LISP-site" or whatever which will be sending
packets to the PETR.


> > The PETR only receives packets from LISP nodes which are addressed to
> > non-LISP addresses.
> 
> Right.

OK.


> > So there is no need for the PETR to contain or be close to an ITR.
> 
> That would likely depend on the economic model/relationship between the PETR,
> the non-LISP hosts being served, and the source of the LISP-encapsulated
> traffic - see the comment above about PETR placement.

OK.


> > Can you list the sorts of scenarios in which the LISP node would be
> > using a PETR?
> 
> I'm not sure an exhaustive list is possible. It's just a pretty basic,
> generic building block, and those kind of things tend to get used in many
> ways that were not forseen. As to a list of possible/plausible ones, Darrel's
> original message gave two, perhaps others can come up with more?

Darrel's messages gave two purposes, or functions to be performed.

I think it would help me and other people if there were some specific
examples of when PETRs would be needed.  If there are multiple types
of scenario when they are needed, it would be good to have two or
more examples of each kind of scenario, together with a more formal
definition of the scenario.

Without this, the discussion inevitably becomes messy since I and
others are trying to imagine why the main LISP developers want PETRs
- but are probably getting it wrong because the developers have not
yet clearly explained, with examples, when they would be needed.


> > Must the LISP node be on a global unicast address, or can it be behind
> > one or more layers of NAT?
> 
> Again, the term "LISP node" confuses me a little - are you talking of
> the original source of the packets, or the ITR, or what?

A am asking about whatever it is which is sending packets to the PETR.

I see the term "LISP-NR" is used - but I didn't know what this meant.

   http://tools.ietf.org/html/draft-ietf-lisp-interworking-00

     LISP Non-Routable (LISP-NR) Site:

        A LISP site whose addresses are EIDs only, these EIDs are
        not found in the legacy Internet routing table.

So to me, this means addresses used by an end-user network where the
addresses are LISP-mapped, and will be handled by ITRs in the sending
network, and tunneled to the ETR of the end-user network, wherever it
is - *and* that this address range is not covered by an advertisement
by an Proxy Tunnel Router.  (If it was in a PTR advertisement, then
these EID addresses used by the end-user network would be encompassed
by one of the BGP-advertised prefixes.)

This makes no sense to me.  Why would anyone adopt LISP-mapped
addresses without them being covered by PTRs?  To do so would mean
that these addresses are only reachable from hosts in networks with
ITRs.

This contrasts with another definition:

     LISP Routable (LISP-R) Site:

        A LISP site whose addresses are used as both globally
        routable IP addresses and LISP EIDs.

To me, this means a LISP site with EID addresses which are covered by
PTR advertisements.  This is the only way I imagine anyone might want
to adopt LISP-mapped addresses, since the PTRs are the only way their
hosts would be reachable from hosts in networks without ITRs.  (In
Ivip, all Ivip-mapped addresses are covered by OITRDs, which do the
same job as PTRs.)

So are PETRs really only needed for this subset of end-user networks
which don't have their addresses covered by PTRs?  If so, I would
like to understand why - and why anyone thinks LISP would be adopted
on this basis by any end-user networks.


> The ITR will have to have a global unicast address, but that does not prevent
> it being behind a NAT - it just has to figure out what the _NAT_'s global
> unicast address is, and register that as the RLOC associated with the EIDs
> which which that ITR is responsible.

Is this formally specified anywhere yet?  It seems unworkable to me
that an ITR would have to figure out the address of its NAT box.
What if it is behind multiple layers of NAT?

I think there really needs to be a full description of why anyone
expects a LISP ITR to be behind NAT - with all the business and
technical reasons why this is the case.

Having the ITR behind NAT violates one of the original principles of
LISP, which is there in the very first I-D, I am sure, about ETRs and
ITRs always being on "RLOC" global unicast addresses, not on "EID"
addresses and implicitly not behind NAT, since behind NAT means their
actual address is not global unicast.


> > Do you expect the LISP node to be connected to the Net via a slow,
> > expensive and unreliable last mile link such as a wireless mobile
> > network? 
> 
> I don't think we're making any assumptions about the kind of link it will
> have. Like I said, it's a pretty generic building block, and will likely be
> used in many different ways.

I think you need to make some specific examples.  The original LISP
concept did not anticipate ITRs being on slow, expensive or
unreliable links.  If this is being contemplated, it needs to be
recognised clearly - and reasons given for this change.  Then, this
particular class of ITRs may require different arrangements for
PMTUD, handling dropped packets etc.

> > How do you plan to cope with PMTUD and packet loss problems between the
> > LISP node and the PETR?
> 
> The same way we would between the ITR and any other ETR, I would assume.

Just because this new network element is called a Proxy ETR doesn't
mean it has anything in common with an ordinary LISP ETR.  With the
original ETR concept, you knew it was either:

  1 - In the network of the ISP which was one of the ISPs by which
      the end-user network is using for Internet access.

or

  2 - It is a Provider Edge router in the end-user network, right
      next to the ISP (but presumably at the end-user site, not
      at the ISP).

I think there was ongoing debate about where ETRs were located and I
don't recall it being settled either way.

These PETRs do not in any way resemble ETRs.  They are not at or near
the destination end-user network and they are not in a business sense
associated with any destination end-user network.  They are, I
suggest, more likely to be close to and in a business sense
associated with the end-user network which is sending packets.

With the basic ETR concept, if the ETR is dead, this means (for a
multihomed destination end-user site) that each sending ITRs somehow
discover this and then decides to tunnel packets to another ETR which
is specified in the mapping.

So with this basic concept of an ETR, the ITR is not generally
concerned about two-way communication to make sure all packets it
tunnels to it are received.

If a PETR is dead, this has nothing to do with mapping or any
particular end-user site.  The ITR really needs to be sure all the
packets (or almost all of them) it tunnels to the PETR are received
there and that the PETR can handle them properly.  A PETR failure is
not at all like an ETR failure, since there is no mapping involved,
no alternative ETR to choose etc.


> > How does the LISP node ensure that the PETR is still alive?
> 
> That's an interesting point. Some of the techniques used to detect
> reachability/liveness of the ETR won't necessarily work with PETRs. E.g.
> unless a PETR is co-located in the same box as a PITR, there won't be any
> reverse user-data traffic to piggyback nonces on.

OK - as I wrote above.


> However, reachability/liveness testing does fall back to explicit probing,
> when none of the other mechanisms are giving a positive signal, so there will
> be a backstop. And some of the other ones might work (e.g. PITR/PETR
> colocation), there's no way to know about a particular configuration until
> it's examined.

OK.

  - Robin