Re: [lisp] Review draft-ietf-lisp-te

[Dino Farinacci <[e-mail]>]

I browsed your comments. They are easier to interpret now. Thanks for that. However, your <move> references are not helpful because they indicate you want text to be moved but you don’t say where. So please clarify that before I make any changes. 

Dino

> On Apr 26, 2024, at 9:16 AM, Luigi Iannone <[e-mail]> wrote:
> 
> 
> Comments inline.
> 
> Ciao
> 
> L.
> 
> 
>> 
>> 
>> 
>> 
>> Internet Engineering Task Force                             D. Farinacci
>> Internet-Draft                                               lispers.net
>> Intended status: Experimental                                   M. Kowal
>> Expires: 24 October 2024                                   cisco Systems
>>                                                                P. Lahiri
>>                                                            22 April 2024
>> 
>> 
>>                         LISP Traffic Engineering
>>                          draft-ietf-lisp-te-15
>> 
>> Abstract
>> 
>>    This document describes how LISP re-encapsulating tunnels can be used
>>    for Traffic Engineering purposes.  The mechanisms described in this
>>    document require no LISP protocol changes but do introduce a new
>>    locator (RLOC) encoding.  The Traffic Engineering features provided
>>    by these LISP mechanisms can span intra-domain, inter-domain, or
>>    combination of both.
>> 
>> Status of This Memo
>> 
>>    This Internet-Draft is submitted in full conformance with the
>>    provisions of BCP 78 and BCP 79.
>> 
>>    Internet-Drafts are working documents of the Internet Engineering
>>    Task Force (IETF).  Note that other groups may also distribute
>>    working documents as Internet-Drafts.  The list of current Internet-
>>    Drafts is at https://datatracker.ietf.org/drafts/current/.
>> 
>>    Internet-Drafts are draft documents valid for a maximum of six months
>>    and may be updated, replaced, or obsoleted by other documents at any
>>    time.  It is inappropriate to use Internet-Drafts as reference
>>    material or to cite them other than as "work in progress."
>> 
>>    This Internet-Draft will expire on 24 October 2024.
>> 
>> Copyright Notice
>> 
>>    Copyright (c) 2024 IETF Trust and the persons identified as the
>>    document authors.  All rights reserved.
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024                [Page 1]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>    This document is subject to BCP 78 and the IETF Trust's Legal
>>    Provisions Relating to IETF Documents (https://trustee.ietf.org/
>>    license-info) in effect on the date of publication of this document.
>>    Please review these documents carefully, as they describe your rights
>>    and restrictions with respect to this document.  Code Components
>>    extracted from this document must include Revised BSD License text as
>>    described in Section 4.e of the Trust Legal Provisions and are
>>    provided without warranty as described in the Revised BSD License.
>> 
>> Table of Contents
>> 
>>    1.  Requirements Language . . . . . . . . . . . . . . . . . . . .   3
>>    2.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
>>    3.  Definition of Terms . . . . . . . . . . . . . . . . . . . . .   3
>>    4.  Overview  . . . . . . . . . . . . . . . . . . . . . . . . . .   4
>>    5.  Explicit Locator Paths  . . . . . . . . . . . . . . . . . . .   5
>>      5.1.  ELP Re-optimization . . . . . . . . . . . . . . . . . . .   7
>>      5.2.  Using Recursion . . . . . . . . . . . . . . . . . . . . .   8
>>      5.3.  ELP Selection based on Class of Service . . . . . . . . .   8
>>      5.4.  Packet Loop Avoidance . . . . . . . . . . . . . . . . . .   9
>>    6.  Service Chaining  . . . . . . . . . . . . . . . . . . . . . .  10
>>    7.  RLOC Probing by RTRs  . . . . . . . . . . . . . . . . . . . .  10
>>    8.  ELP Probing . . . . . . . . . . . . . . . . . . . . . . . . .  10
>>    9.  Interworking Considerations . . . . . . . . . . . . . . . . .  11
>>    10. Multicast Considerations  . . . . . . . . . . . . . . . . . .  12
>>    11. Security Considerations . . . . . . . . . . . . . . . . . . .  14
>>    12. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  14
>>    13. References  . . . . . . . . . . . . . . . . . . . . . . . . .  14
>>      13.1.  Normative References . . . . . . . . . . . . . . . . . .  14
>>      13.2.  Informative References . . . . . . . . . . . . . . . . .  15
>>    Appendix A.  Acknowledgments  . . . . . . . . . . . . . . . . . .  16
>>    Appendix B.  Document Change Log  . . . . . . . . . . . . . . . .  16
>>      B.1.  Changes to draft-ietf-lisp-te-15  . . . . . . . . . . . .  16
>>      B.2.  Changes to draft-ietf-lisp-te-14  . . . . . . . . . . . .  16
>>      B.3.  Changes to draft-ietf-lisp-te-13  . . . . . . . . . . . .  16
>>      B.4.  Changes to draft-ietf-lisp-te-12  . . . . . . . . . . . .  16
>>      B.5.  Changes to draft-ietf-lisp-te-11  . . . . . . . . . . . .  16
>>      B.6.  Changes to draft-ietf-lisp-te-10  . . . . . . . . . . . .  17
>>      B.7.  Changes to draft-ietf-lisp-te-09  . . . . . . . . . . . .  17
>>      B.8.  Changes to draft-ietf-lisp-te-08  . . . . . . . . . . . .  17
>>      B.9.  Changes to draft-ietf-lisp-te-07  . . . . . . . . . . . .  17
>>      B.10. Changes to draft-ietf-lisp-te-06  . . . . . . . . . . . .  17
>>      B.11. Changes to draft-ietf-lisp-te-05  . . . . . . . . . . . .  17
>>      B.12. Changes to draft-ietf-lisp-te-04  . . . . . . . . . . . .  17
>>      B.13. Changes to draft-ietf-lisp-te-03  . . . . . . . . . . . .  17
>>      B.14. Changes to draft-ietf-lisp-te-02  . . . . . . . . . . . .  18
>>      B.15. Changes to draft-ietf-lisp-te-01  . . . . . . . . . . . .  18
>>      B.16. Changes to draft-ietf-lisp-te-00  . . . . . . . . . . . .  18
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024                [Page 2]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>      B.17. Changes to draft-farinacci-lisp-te-02 through -12 . . . .  18
>>      B.18. Changes to draft-farinacci-lisp-te-01.txt . . . . . . . .  18
>>      B.19. Changes to draft-farinacci-lisp-te-00.txt . . . . . . . .  18
>>    Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  18
>> 
>> 1.  Requirements Language
>> 
>>    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
>>    "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
>>    "OPTIONAL" in this document are to be interpreted as described in BCP
>>    14 [RFC2119][RFC8174] when, and only when, they appear in all
>>    capitals, as shown here.
>> 
>> 2.  Introduction
>> 
>>    This document describes extensions to the Locator/Identifier
>>    Separation Protocol (LISP) for traffic engineering features.
>> 
>>    When LISP routers encapsulate packets to other LISP routers, the path
>>    stretch is typically 1, meaning the packet travels on a direct path
>>    from the encapsulating ITR to the decapsulating ETR at the
>>    destination site.  The direct path is determined by the underlying
>>    routing protocol and metrics it uses to find the shortest path.
>> 
>>    This specification will examine how re-encapsulating tunnels
>>    [RFC9300] can be used so a packet can take an administratively
>>    specified path, a congestion avoidance path, a failure recovery path,
>>    or multiple load-shared paths, as it travels from ITR to ETR.  By
>>    introducing an Explicit Locator Path (ELP) locator encoding
>>    [RFC8060], an ITR can encapsulate a packet to a Re-Encapsulating
>>    Tunnel Router (RTR) which decapsulates the packet, then encapsulates
>>    it to the next locator in the ELP.
>> 
>> 3.  Definition of Terms
>> 
>>    Refer to [RFC9300] for authoritative definitions for terms EID, RLOC,
>>    RTR, and Recursive Tunneling.  The other terms defined in this
>>    section add to the canonical definition to reflect the design
>>    considerations in this specification.
>> 
>>    Explicit Locator Path (ELP):  The ELP is an explicit list of RLOCs
>>       for each RTR a packet must travel to along its path toward a final
>>       destination ETR (or PETR).  The list is a strict ordering where
>>       each RLOC in the list is visited.  However, the path from one RTR
>>       to another is determined by the underlying routing protocol and
>>       how the infrastructure assigns metrics and policies for the path.
>> 
>>    Re-Encapsulating Tunnel Router (RTR):  An RTR is a router that acts
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024                [Page 3]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>       as an ETR (or PETR) by decapsulating packets where the destination
>>       address in the "outer" IP header is one of its own RLOCs.  Then
>>       acts as an ITR (or PITR) by making a decision where to encapsulate
>>       the packet based on the next locator in the ELP towards the final
>>       destination ETR.
> For that you do not need to re-define the RTR term. You just need to state that RTR uses ELPs in the body of this document, which you do.
> The must be only one document authoritative on the terms, and 9300 is already there.
>> 4.  Overview
>> 
>>    Typically, a packet's path from source EID to destination EID travels
>>    through the locator core via the encapsulating ITR directly to the
>>    decapsulating ETR as the following diagram illustrates:
>> 
>>    Legend:
>> 
>>    seid:  Packet is originated by source EID 'seid'.
>> 
>>    deid:  Packet is consumed by destination EID 'deid'.
>> 
>>    A,B,C,D :  Core routers in different ASes.
>> 
>>    ---> :  The physical topological path between two routers.
>> 
>>    ===> :  A multi-hop LISP dynamic tunnel between LISP routers.
>> 
>> 
>>                               Core Network
>>    Source site       (----------------------------)    Destination Site
>>    +--------+        (                            )         +---------+
>>    |         \       (                            )        /          |
>>    | seid     ITR ---(---> A --> B --> C --> D ---)---> ETR      deid |
>>    |         / ||    (                            )     ^^ \          |
>>    +--------+  ||    (                            )     ||  +---------+
>>                ||    (----------------------------)     ||
>>                ||                                       ||
>>                ===========================================
>>                                 LISP Tunnel
>> 
>> 
>>                 Figure 1: Typical Data Path from ITR to ETR
>> 
>> 
>>    Let's introduce RTRs 'X' and 'Y' so that, for example, if it is
>>    desirable to route around the path from B to C, one could provide an
>>    ELP of (X,Y,etr):
>> 
>> 
>> 
> In the current figure 2 there is no physical path, between X and Y, that "routes around” the path from B to C. 
> Packets will still go through the path B to C, in contradiction with your objective “route around the path from B to C”.
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024                [Page 4]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>                               Core Network
>>    Source site       (----------------------------)    Destination Site
>>    +--------+        (                            )         +---------+
>>    |         \       (                            )        /          |
>>    | seid     ITR ---(---> A --> B --> C --> D ---)---> ETR      deid |
>>    |         / ||    (          /      ^          )     ^^ \          |
>>    |        /  ||    (         |        \         )     ||  \         |
>>    +-------+   ||    (         v         |        )     ||   +--------+
>>                ||    (         X ======> Y        )     ||
>>                ||    (        ^^         ||       )     ||
>>                ||    (--------||---------||-------)     ||
>>                ||             ||         ||             ||
>>                =================         =================
>>                  LISP Tunnel                 LISP Tunnel
>> 
>> 
>>    Figure 2: ELP tunnel path ITR ==> X, then X ==> Y, and then Y ==> ETR
>> 
>> 
>>    There are various reasons why the path from 'seid' to 'deid' may want
>>    to avoid the path from B to C.  To list a few:
>> 
>>    *  There may not be sufficient capacity provided by the networks that
>>       connect B and C together.
>> 
>>    *  There may be a policy reason to avoid the ASes that make up the
>>       path between B and C.
>> 
>>    *  There may be a failure on the path between B and C which makes the
>>       path unreliable.
>> 
>>    *  There may be monitoring or traffic inspection resources close to
>>       RTRs X and Y that do network accounting or measurement.
>> 
>>    *  There may be a chain of services performed at RTRs X and Y
>>       regardless if the path from ITR to ETR is through B and C.
>> 
>> 5.  Explicit Locator Paths
> 
> This is the main technical contribution of this document and should be separated from the use cases. However, some of the technical details is spread in the use cases. 
> Paragraphs that should be appended to this sections are enclosed by the tags <move> </move>.
>>    The notation for a general formatted ELP is (x, y, etr) which
>>    represents the list of RTRs a packet SHOULD travel through to reach
>>    the final tunnel hop to the ETR.
>> 
>>    The procedure for using an ELP at each tunnel hop is as follows:
>> 
>>    1.  The ITR will retrieve the ELP from the mapping database.
>> 
>>    2.  The ITR will encapsulate the packet to RLOC 'x'.
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024                [Page 5]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>    3.  The RTR with RLOC 'x' will decapsulate the packet.  It will use
>>        the decapsulated packet's destination address as a lookup into
>>        the mapping database to retrieve the ELP.
>> 
>>    4.  RTR 'x' will encapsulate the packet to RTR with RLOC 'y'.
>> 
>>    5.  The RTR with RLOC 'y' will decapsulate the packet.  It will use
>>        the decapsulated packet's destination address as a lookup into
>>        the mapping database to retrieve the ELP.
>> 
>>    6.  RTR 'y' will encapsulate the packet on the final tunnel hop to
>>        ETR with RLOC 'etr'.
>> 
>>    7.  The ETR will decapsulate the packet and deliver the packet to the
>>        EID inside of its site.
>> 
>>    The specific encoding format for the ELP can be found in [RFC8060].
>>    It is defined that an ELP will appear as a single encoded locator in
>>    a locator-set.  Say for instance, we have a mapping entry for EID-
>>    prefix 10.0.0.0/8 that is reachable via 4 locators.  Two locators are
>>    being used as active/active and the other two are used as active/
>>    active if the first two go unreachable (as noted by the priority
>>    assignments below).  This is what the mapping entry would look like:
>> 
>> 
>>    EID-prefix:   10.0.0.0/8
>>    Locator-set:  ETR-A: priority 1, weight 50
>>                  ETR-B: priority 1, weight 50
>>                  ETR-C: priority 2, weight 50
>>                  ETR-D: priority 2, weight 50
>> 
>> 
>>    If an ELP is going to be used to have a policy path to ETR-A and
>>    possibly another policy path to ETR-B, the locator-set would be
>>    encoded as follows:
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024                [Page 6]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>    EID-prefix:   10.0.0.0/8
>>    Locator-set:  (x, y, ETR-A): priority 1, weight 50
>>                  (q, r, ETR-B): priority 1, weight 50
>>                  ETR-C:         priority 2, weight 50
>>                  ETR-D:         priority 2, weight 50
>> 
>> 
>>    The mapping entry with ELP locators is registered to the mapping
>>    database system just like any other mapping entry would.  The
>>    registration is typically performed by the ETR(s) that are assigned
>>    and own the EID-prefix. 
> Add reference to RFC9301.
> You state that ELP are registered as any other mapping but you do not state how they are retrieved by the RTRs.
> Put a sentence and a reference to RFC9300.
> 
>>  That is, the destination site makes the
>>    choice of the RTRs in the ELP.  Alternatively, it may be common
>>    practice for a provisioning system to program the mapping database
>>    with ELPs.
> Do not understand the role of this provisioning system. Clarify or delete.
> 
>>    Another case where a locator-set can be used for flow-based load-
>>    sharing across multiple paths to the same destination site:
>> 
>> 
>>    EID-prefix:   10.0.0.0/8
>>    Locator-set:  (x, y, ETR-A): priority 1, weight 75
>>                  (q, r, ETR-A): priority 1, weight 25
>> 
>> 
>>    Using this mapping entry, an ITR would load split 75% of the EID
>>    flows on the (x, y, ETR-A) ELP path and 25% of the EID flows on the
>>    (q, r, ETR-A) ELP path.  If any of the ELPs go down, then the other
>>    can take 100% of the load.
>>  
> Dino, you correct text mixes specifications and use cases. By concentrating the specifications in one section (namely section 5) you will improve readability and clarity of the document. 
> 
> Put here:
> 
> 6. Use cases
> 
> And renumber the subsections.
> 
>> 5.1.  ELP Re-optimization
> 
> <move>
>>    ELP re-optimization is a process of changing the RLOCs of an ELP due
>>    to underlying network change conditions.  Just like when there is any
>>    locator change for a locator-set, the procedures from the main LISP
>>    specification [RFC9300] are followed.
>> 
>>    When a RLOC from an ELP is changed, Map-Notify messages [RFC9301] can
>>    be used to inform the existing RTRs in the ELP so they can do a
>>    lookup to obtain the latest version of the ELP.  Map-Notify messages
>>    can also be sent to new RTRs in an ELP so they can get the ELP in
>>    advance to receiving packets that will use the ELP.  This can
>>    minimize packet loss during mapping database lookups in RTRs.
>> 
>> 
> </move>
> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024                [Page 7]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>> 5.2.  Using Recursion
>> 
>>    In the previous examples, we showed how an ITR encapsulates using an
>>    ELP of (x, y, etr).  When a packet is encapsulated by the ITR to RTR
>>    'x', the RTR may want a policy path to RTR 'y' and run another level
>>    of re-encapsulating tunnels for packets destined to RTR 'y'.  In this
>>    case, RTR 'x' does not encapsulate packets to 'y' but rather performs
>>    a mapping database lookup on the address 'y', requests the ELP for
>>    RTR 'y', 
> What really request is a mapping, which may or may not be an ELP.
> What happens if it receives a negative map-reply?
> 
>> and encapsulates packets to the first-hop of the returned
>>    ELP.  This can be done when using a public or private mapping
>>    database.  
> You mean that this second lookup can be done on a mapping system that is different from the one who delivered the initial ELP, right?
> If yes, can you state so?
> 
>> The decision to use address 'y' as an encapsulation
>>    address versus a lookup address is based on the L-bit 
> How the S-bit, L-bit, and the P-bit are used is not covered at all and should be described in section 5.
> 
> 
>> setting for 'y'
>>    in the ELP entry.  The decision and policy of ELP encodings are local
>>    to the entity which registers the EID-prefix associated with the ELP.
>> 
>>    Another example of recursion is when the ITR uses the ELP (x, y, etr)
>>    to first prepend a header with a destination RLOC of the ETR and then
>>    prepend another header and encapsulate the packet to RTR 'x'.  When
>>    RTR 'x' decapsulates the packet, rather than doing a mapping database
>>    lookup on RTR 'y' the last example showed, instead RTR 'x' does a
>>    mapping database lookup on ETR 'etr'.  In this scenario, RTR 'x' can
>>    choose an ELP from the locator-set by considering the source RLOC
>>    address of the ITR versus considering the source EID.
>> 
>>    This additional level of recursion also brings advantages for the
>>    provider of RTR 'x' to store less state.  Since RTR 'x' does not need
>>    to look at the inner most header, it does not need to store EID
>>    state.  It only stores an entry for RTR 'y' which many EID flows
>>    could share for scaling benefits.  The locator-set for entry 'y'
>>    could either be a list of typical locators, a list of ELPs, or
>>    combination of both.  Another advantage is that packet load-splitting
>>    can be accomplished by examining the source of a packet.  If the
>>    source is an ITR versus the source being the last-hop of an ELP the
>>    last-hop selected, different forwarding paths can be used.
>> 
>> 5.3.  ELP Selection based on Class of Service
>> 
>>    Paths to an ETR may want to be selected based on different classes of
>>    service.  Packets from a set of sources that have premium service can
>>    use ELP paths that are less congested where normal sources use ELP
>>    paths that compete for less resources or use longer paths for best
>>    effort service.
>> 
>>    Using source/destination lookups into the mapping database can yield
>>    different ELPs.  For example, a premium service flow with
>>    (source=1.1.1.1, dest=10.1.1.1) can be described by using the
>>    following mapping entry:
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024                [Page 8]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>    EID-prefix:   (1.0.0.0/8, 10.0.0.0/8)
>>    Locator-set:  (x, y, ETR-A): priority 1, weight 50
>>                  (q, r, ETR-A): priority 1, weight 50
>> 
>> 
>>    And all other best-effort sources would use different mapping entry
>>    described by:
>> 
>> 
>>    EID-prefix:   (0.0.0.0/0, 10.0.0.0/8)
>>    Locator-set:  (x, x', y, y', ETR-A): priority 1, weight 50
>>                  (q, q', r, r', ETR-A): priority 1, weight 50
>> 
>> 
>>    If the source/destination lookup is coupled with recursive lookups,
>>    then an ITR can encapsulate to the ETR, prepending a header that
>>    selects source address ITR-1 based on the premium class of service
>>    source, or selects source address ITR-2 for best-effort sources with
>>    normal class of service.  The ITR then does another lookup in the
>>    mapping database on the prepended header using lookup key
>>    (source=ITR-1, dest=10.1.1.1) that returns the following mapping
>>    entry:
>> 
>> 
>>    EID-prefix:   (ITR-1, 10.0.0.0/8)
>>    Locator-set:  (x, y, ETR-A): priority 1, weight 50
>>                  (q, r, ETR-A): priority 1, weight 50
>> 
>> 
>>    And all other sources would use different mapping entry with a lookup
>>    key of (source=ITR-2, dest=10.1.1.1):
>> 
>> 
>>    EID-prefix:   (ITR-2, 10.0.0.0/8)
>>    Locator-set:  (x, x', y, y', ETR-A): priority 1, weight 50
>>                  (q, q', r, r', ETR-A): priority 1, weight 50
>> 
>> 
>>    This will scale the mapping system better by having fewer source/
>>    destination combinations.  Refer to the Source/Dest LCAF type
>>    described in [RFC8060] for encoding EIDs in Map-Request and Map-
>>    Register messages.
>> 
>> 5.4.  Packet Loop Avoidance
>> 
> <move>
>>    An ELP that is first used by an ITR must be inspected for encoding
>>    loops.  If any RLOC appears twice in the ELP, it MUST not be used.
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024                [Page 9]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>    Since it is expected that multiple mapping systems will be used,
>>    there can be a loop across ELPs when registered in different mapping
>>    systems.  The TTL copying procedures for re-encapsulating tunnels and
>>    recursive tunnels in [RFC9300] MUST be followed.
>> 
> </move>
> 
>> 6.  Service Chaining
>> 
>>    An ELP can be used to deploy services at each reencapsulation point
>>    in the network.  One example is to implement a honey-pot service when
>>    a destination EID is being DoS attacked.  That is, when a DoS attack
>>    is recognized when the encapsulation path is between ITR and ETR, an
>>    ELP can be registered for a destination EID to the mapping database
>>    system.  The ELP can include an RTR so the ITR can encapsulate
>>    packets to the RTR which will decapsulate and deliver packets to a
>>    scrubber service device.  The scrubber could decide if the offending
>>    packets are dropped or allowed to be sent to the destination EID.  In
>>    which case, the scurbber delivers packets back to the RTR which
>>    encapsulates to the ETR.
>> 
>> 7.  RLOC Probing by RTRs
>> 
> <move>
>>    Since an RTR knows the next tunnel hop to encapsulate to, it can
>>    monitor the reachability of the next-hop RTR RLOC by doing RLOC-
>>    probing according to the procedures in [RFC9300].  When the RLOC is
>>    determined unreachable by the RLOC-probing mechanisms, the RTR can
>>    use another locator in the locator-set.  That could be the final ETR,
>>    a RLOC of another RTR, or an ELP where it must search for itself and
>>    use the next RLOC in the ELP list to encapsulate to.
>> 
>>    RLOC-probing can also be used to measure delay on the path between
>>    RTRs and when it is desirable switch to another lower delay ELP.
>> 
> </move>
>> 8.  ELP Probing
>> 
>>    Since an ELP-node knows the reachabiliy of the next ELP-node in a ELP
>>    by using RLOC probing, the sum of reachability can determine the
>>    reachability of the entire path.  A head-end ITR/RTR/PITR can
>>    determine the quality of a path and decide to select one path from
>>    another based on the telemetry data gathered by RLOC-probing for each
>>    encapsulation hop.
>> 
>>    ELP-probing mechanism details can be found in
>>    [I-D.filyurin-lisp-elp-probing].
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024               [Page 10]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>> 9.  Interworking Considerations
>> 
>>    [RFC6832] defines procedures for how non-LISP sites talk to LISP
>>    sites.  The network elements defined in the Interworking
>>    specification, the proxy ITR (PITR) and proxy ETR (PETR) (as well as
>>    their multicast counterparts defined in [RFC6831]) can participate in
>>    LISP-TE.  That is, a PITR and a PETR can appear in an ELP list and
>>    act as an RTR.
>> 
> <move>
>>    Note when an RLOC appears in an ELP, it can be of any address-family.
>>    There can be a mix of IPv4 and IPv6 locators present in the same ELP.
>>    This can provide benefits where islands of one address-family or the
>>    other are supported and connectivity across them is necessary.  For
>>    instance, an ELP can look like:
>> 
>>    (x4, a46, b64, y4, etr)
>> 
>>    Where an IPv4 ITR will encapsulate using an IPv4 RLOC 'x4' and 'x4'
>>    could reach an IPv4 RLOC 'a46', but RTR 'a46' encapsulates to an IPv6
>>    RLOC 'b64' when the network between them is IPv6-only.  Then RTR
>>    'b64' encapsulates to IPv4 RLOC 'y4' if the network between them is
>>    dual-stack.
>> 
> </move>
>>    Note that RTRs can be used for NAT-traversal scenarios
>>    [I-D.ermagan-lisp-nat-traversal] as well to reduce the state in both
>>    an xTR that resides behind a NAT and the state the NAT needs to
>>    maintain.  In this case, the xTR only needs a default map-cache entry
>>    pointing to the RTR for outbound traffic and all remote ITRs can
>>    reach EIDs through the xTR behind a NAT via a single RTR (or a small
>>    set RTRs for redundancy).
>> 
>>    RTRs have some scaling features to reduce the number of locator-set
>>    changes, the amount of state, and control packet overhead:
>> 
>>    *  When ITRs and PITRs are using a small set of RTRs for
>>       encapsulating to "orders of magnitude" more EID-prefixes, the
>>       probability of locator-set changes are limited to the RTR RLOC
>>       changes versus the RLOC changes for the ETRs associated with the
>>       EID-prefixes if the ITRs and PITRs were directly encapsulating to
>>       the ETRs.  This comes at an expense in packet stretch, but
>>       depending on RTR placement, this expense can be mitigated.
>> 
>>    *  When RTRs are on-path between many pairwise EID flows, ITRs and
>>       PITRs can store a small number of coarse EID-prefixes.
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024               [Page 11]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>    *  RTRs can be used to help scale RLOC-probing.  Instead of ITRs
>>       RLOC-probing all ETRs for each destination site it has cached, the
>>       ITRs can probe a smaller set of RTRs which in turn, probe the
>>       destination sites.
>> 
>> 10.  Multicast Considerations
>> 
>>    ELPs have application in multicast environments.  Just like RTRs can
>>    be used to provide connectivity across different address family
>>    islands, RTRs can help concatenate a multicast region of the network
>>    to one that does not support native multicast.
>> 
>>    Note there are various combinations of connectivity that can be
>>    accomplished with the deployment of RTRs and ELPs:
>> 
>>    *  Providing multicast forwarding between IPv4-only-unicast regions
>>       and IPv4-multicast regions.
>> 
>>    *  Providing multicast forwarding between IPv6-only-unicast regions
>>       and IPv6-multicast regions.
>> 
>>    *  Providing multicast forwarding between IPv4-only-unicast regions
>>       and IPv6-multicast regions.
>> 
>>    *  Providing multicast forwarding between IPv6-only-unicast regions
>>       and IPv4-multicast regions.
>> 
>>    *  Providing multicast forwarding between IPv4-multicast regions and
>>       IPv6-multicast regions.
>> 
>>    An ITR or PITR can do a (S-EID,G) lookup into the mapping database.
>>    What can be returned is a typical locator-set that could be made up
>>    of the various RLOC addresses:
>> 
>> 
>>    Multicast EID key:  (S-EID, G)
> The document uses a mix of “seid” and “S-EID”, choose one.
> 
>>    Locator-set:        ETR-A: priority 1, weight 25
>>                        ETR-B: priority 1, weight 25
>>                        g1:    priority 1, weight 25
>>                        g2:    priority 1, weight 25
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024               [Page 12]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>     Figure 3: An entry for host 'S-EID' sending to application group 'G'
>> 
>> 
>>    The locator-set above can be used as a replication list.  That is
>>    some RLOCs listed can be unicast RLOCs and some can be delivery group
>>    RLOCs.  A unicast RLOC in this case is used to encapsulate a
>>    multicast packet originated by a multicast source EID into a unicast
>>    packet for unicast delivery on the underlying network.  ETR-A could
>>    be an IPv4 unicast RLOC address and ETR-B could be a IPv6 unicast
>>    RLOC address.
>> 
>>    A delivery group address is used when a multicast packet originated
>>    by a multicast source EID is encapsulated in a multicast packet for
>>    multicast delivery on the underlying network.  Group address 'g1'
>>    could be an IPv4 delivery group RLOC and group address 'g2' could be
>>    an IPv6 delivery group RLOC.
>> 
>>    Flexibility for these various types of connectivity combinations can
>>    be achieved and provided by the mapping database system.  And the RTR
>>    placement allows the connectivity to occur where the differences in
>>    network functionality is located.
>> 
>>    Extending this concept by allowing ELPs in locator-sets, one could
>>    have this locator-set registered in the mapping database for (S-EID,
>>    G).  For example:
>> 
>> 
>>    Multicast EID key:  (S-EID, G)
>>    Locator-set:        (x, y, ETR-A):    priority 1, weight 50
>>                        (a, g, b, ETR-B): priority 1, weight 50
>> 
>>                   Figure 4: Using ELPs for multicast flows
>> 
>> 
>>    In the above situation, an ITR would encapsulate a multicast packet
>>    originated by a multicast source EID to the RTR with unicast RLOC
>>    'x'.  Then RTR 'x' would decapsulate and unicast encapsulate to RTR
>>    'y' ('x' or 'y' could be either IPv4 or IPv6 unicast RLOCs), which
>>    would decapsulate and unicast encapsulate to the final RLOC 'ETR-A'.
>>    The ETR 'ETR-A' would decapsulate and deliver the multicast packet
>>    natively to all the receivers joined to application group 'G' inside
>>    the LISP site.
>> 
>>    Let's look at the ITR using the ELP (a, g, b, ETR-B).  Here the
>>    encapsulation path would be the ITR unicast encapsulates to unicast
>>    RLOC 'a'.  RTR 'a' multicast encapsulates to delivery group 'g'.  The
>>    packet gets to all ETRs that have joined delivery group 'g' so they
>>    can deliver the multicast packet to joined receivers of application
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024               [Page 13]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>    group 'G' in their sites.  RTR 'b' is also joined to delivery group
>>    'g'. 
> What if it isn’t? Or what if there are several RTR that should re-encap in unicast? It look underspecified to me.
> 
>>  Since it is in the ELP, it will be the only RTR that unicast
>>    encapsulates the multicast packet to ETR 'ETR-B'.  Lastly, 'ETR-B'
>>    decapsulates and delivers the multicast packet to joined receivers to
>>    application group 'G' in its LISP site.
>> 
>>    As one can see there are all sorts of opportunities to provide
>>    multicast connectivity across a network with non-congruent support
>>    for multicast and different address-families.  One can also see how
>>    using the mapping database can allow flexible forms of delivery
>>    policy, rerouting, and congestion control management in multicast
>>    environments.
>> 
>> 11.  Security Considerations
>> 
>>    When an RTR receives a LISP encapsulated packet, it can look at the
>>    outer source address to verify that RLOC is the one listed as the
>>    previous hop in the ELP list.  If the outer source RLOC address
>>    appears before the RLOC which matches the outer destination RLOC
>>    address, the decapsulating RTR (or ETR if last hop), MAY choose to
>>    drop the packet.
> Why not a SHOULD?
> Flexibility is not a sufficient answer. The MAY opens the door to security issues. 
> How does this work with LISP-SEC? IMO there is no changes needed, it just works out of the box. Would be good to state it explicitly.
> 
> I would add a sentence about new attacks. Refer to RFC7835 and state if there are additional attack. If not, just state explicitly that no new attack vectors are introduced by this mechanism.
>> 12.  IANA Considerations
>> 
>>    This document does not make any request to IANA.
>> 
>> 13.  References
>> 
>> 13.1.  Normative References
>> 
>>    [RFC0791]  Postel, J., "Internet Protocol", STD 5, RFC 791,
>>               DOI 10.17487/RFC0791, September 1981,
>>               <https://www.rfc-editor.org/info/rfc791>.
>> 
>>    [RFC1034]  Mockapetris, P., "Domain names - concepts and facilities",
>>               STD 13, RFC 1034, DOI 10.17487/RFC1034, November 1987,
>>               <https://www.rfc-editor.org/info/rfc1034>.
>> 
>>    [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
>>               Requirement Levels", BCP 14, RFC 2119,
>>               DOI 10.17487/RFC2119, March 1997,
>>               <https://www.rfc-editor.org/info/rfc2119>.
>> 
>>    [RFC2460]  Deering, S. and R. Hinden, "Internet Protocol, Version 6
>>               (IPv6) Specification", RFC 2460, DOI 10.17487/RFC2460,
>>               December 1998, <https://www.rfc-editor.org/info/rfc2460>.
>> 
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024               [Page 14]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>    [RFC3261]  Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston,
>>               A., Peterson, J., Sparks, R., Handley, M., and E.
>>               Schooler, "SIP: Session Initiation Protocol", RFC 3261,
>>               DOI 10.17487/RFC3261, June 2002,
>>               <https://www.rfc-editor.org/info/rfc3261>.
>> 
>>    [RFC6831]  Farinacci, D., Meyer, D., Zwiebel, J., and S. Venaas, "The
>>               Locator/ID Separation Protocol (LISP) for Multicast
>>               Environments", RFC 6831, DOI 10.17487/RFC6831, January
>>               2013, <https://www.rfc-editor.org/info/rfc6831>.
>> 
>>    [RFC6832]  Lewis, D., Meyer, D., Farinacci, D., and V. Fuller,
>>               "Interworking between Locator/ID Separation Protocol
>>               (LISP) and Non-LISP Sites", RFC 6832,
>>               DOI 10.17487/RFC6832, January 2013,
>>               <https://www.rfc-editor.org/info/rfc6832>.
>> 
>>    [RFC8060]  Farinacci, D., Meyer, D., and J. Snijders, "LISP Canonical
>>               Address Format (LCAF)", RFC 8060, DOI 10.17487/RFC8060,
>>               February 2017, <https://www.rfc-editor.org/info/rfc8060>.
>> 
>>    [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
>>               2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
>>               May 2017, <https://www.rfc-editor.org/info/rfc8174>.
>> 
>>    [RFC9300]  Farinacci, D., Fuller, V., Meyer, D., Lewis, D., and A.
>>               Cabellos, Ed., "The Locator/ID Separation Protocol
>>               (LISP)", RFC 9300, DOI 10.17487/RFC9300, October 2022,
>>               <https://www.rfc-editor.org/info/rfc9300>.
>> 
>>    [RFC9301]  Farinacci, D., Maino, F., Fuller, V., and A. Cabellos,
>>               Ed., "Locator/ID Separation Protocol (LISP) Control
>>               Plane", RFC 9301, DOI 10.17487/RFC9301, October 2022,
>>               <https://www.rfc-editor.org/info/rfc9301>.
>> 
>> 13.2.  Informative References
>> 
>>    [I-D.ermagan-lisp-nat-traversal]
>>               Ermagan, V., Farinacci, D., Lewis, D., Maino, F.,
>>               Portoles-Comeras, M., Skriver, J., White, C., Brescó, A.
>>               L., and A. Cabellos-Aparicio, "NAT traversal for LISP",
>>               Work in Progress, Internet-Draft, draft-ermagan-lisp-nat-
>>               traversal-19, 7 May 2021,
>>               <https://datatracker.ietf.org/doc/html/draft-ermagan-lisp-
>>               nat-traversal-19>.
>> 
>> 
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024               [Page 15]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>    [I-D.filyurin-lisp-elp-probing]
>>               Filyurin, Y., Raszuk, R., Boyes, T., and D. Farinacci,
>>               "LISP Explicit Locator Path (ELP) Probing", Work in
>>               Progress, Internet-Draft, draft-filyurin-lisp-elp-probing-
>>               01, 14 May 2018, <https://datatracker.ietf.org/doc/html/
>>               draft-filyurin-lisp-elp-probing-01>.
>> 
>> Appendix A.  Acknowledgments
>> 
>>    The authors would like to thank the following people for their ideas
>>    and comments.  They are Albert Cabellos, Khalid Raza, and Vina
>>    Ermagan, Gregg Schudel, Yan Filyurin, Robert Raszuk, and Truman
>>    Boyes.
>> 
>> Appendix B.  Document Change Log
>> 
>> B.1.  Changes to draft-ietf-lisp-te-15
>> 
>>    *  Posted April 2024.
>> 
>>    *  Made changes to reflect comments from Luigi as we ready document
>>       for standards track.
>> 
>> B.2.  Changes to draft-ietf-lisp-te-14
>> 
>>    *  Posted February 2024.
>> 
>>    *  Update references and document timer.
>> 
>> B.3.  Changes to draft-ietf-lisp-te-13
>> 
>>    *  Posted August 2023.
>> 
>>    *  Update references (to proposed standard documents) and document
>>       timer.
>> 
>> B.4.  Changes to draft-ietf-lisp-te-12
>> 
>>    *  Posted March 2023.
>> 
>>    *  Update references (to propsed standard documents) and document
>>       timer.
>> 
>> B.5.  Changes to draft-ietf-lisp-te-11
>> 
>>    *  Posted September 2022.
>> 
>>    *  Update document timer and references.
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024               [Page 16]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>> B.6.  Changes to draft-ietf-lisp-te-10
>> 
>>    *  Posted March 2022.
>> 
>>    *  Update document timer and references.
>> 
>> B.7.  Changes to draft-ietf-lisp-te-09
>> 
>>    *  Posted September 2021.
>> 
>>    *  Update document timer and references.
>> 
>> B.8.  Changes to draft-ietf-lisp-te-08
>> 
>>    *  Posted March 2021.
>> 
>>    *  Update document timer and references.
>> 
>> B.9.  Changes to draft-ietf-lisp-te-07
>> 
>>    *  Posted October 2020.
>> 
>>    *  Update document timer and references.
>> 
>> B.10.  Changes to draft-ietf-lisp-te-06
>> 
>>    *  Posted April 2020.
>> 
>>    *  Update document timer and references.
>> 
>> B.11.  Changes to draft-ietf-lisp-te-05
>> 
>>    *  Posted October 2019.
>> 
>>    *  Update document timer and references.
>> 
>> B.12.  Changes to draft-ietf-lisp-te-04
>> 
>>    *  Posted April 2019.
>> 
>>    *  Update document timer and references.
>> 
>> B.13.  Changes to draft-ietf-lisp-te-03
>> 
>>    *  Posted October 2018.
>> 
>>    *  Update document timer and references.
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024               [Page 17]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>> B.14.  Changes to draft-ietf-lisp-te-02
>> 
>>    *  Posted April 2018.
>> 
>>    *  Update document timer and references.
>> 
>> B.15.  Changes to draft-ietf-lisp-te-01
>> 
>>    *  Posted October 2017.
>> 
>>    *  Added section on ELP-probing that tells an ITR/RTR/PITR the
>>       feasibility and reachability of an Explicit Lcoator Path.
>> 
>> B.16.  Changes to draft-ietf-lisp-te-00
>> 
>>    *  Posted April 2017.
>> 
>>    *  Changed draft-farinacci-lisp-te-12 to working group document.
>> 
>> B.17.  Changes to draft-farinacci-lisp-te-02 through -12
>> 
>>    *  Many postings from January 2013 through February 2017.
>> 
>>    *  Update references and document timer.
>> 
>> B.18.  Changes to draft-farinacci-lisp-te-01.txt
>> 
>>    *  Posted July 2012.
>> 
>>    *  Add the Lookup bit to allow an ELP to be a list of encapsulation
>>       and/or mapping database lookup addresses.
>> 
>>    *  Indicate that ELPs can be used for service chaining.
>> 
>>    *  Add text to indicate that Map-Notify messages can be sent to new
>>       RTRs in a ELP so their map-caches can be pre-populated to avoid
>>       mapping database lookup packet loss.
>> 
>>    *  Fixes to editorial comments from Gregg.
>> 
>> B.19.  Changes to draft-farinacci-lisp-te-00.txt
>> 
>>    *  Initial draft posted March 2012.
>> 
>> Authors' Addresses
>> 
>> 
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024               [Page 18]
>> 
>> Internet-Draft          LISP Traffic Engineering              April 2024
>> 
>> 
>>    Dino Farinacci
>>    lispers.net
>>    San Jose, California
>>    United States of America
>>    Phone: [phone]
>>    Email: [e-mail]
>> 
>> 
>>    Michael Kowal
>>    cisco Systems
>>    111 Wood Avenue South
>>    ISELIN, NJ
>>    United States of America
>>    Email: [e-mail]
>> 
>> 
>>    Parantap Lahiri
>>    United States of America
>>    Email: [e-mail]
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Farinacci, et al.        Expires 24 October 2024               [Page 19]