[bess] Re: [Shepherding AD review] review of draft-ietf-bess-evpn-irb-extended-mobility-17

[Neeraj Malhotra <[e-mail]>]

Hi Gunter,

Thanks so much for the detailed review and suggested edits. Many apologies
for the delay, I somehow missed the email earlier.

Will incorporate and update.

Thanks,
Neeraj


On Tue, Aug 6, 2024 at 10:11 AM Gunter van de Velde (Nokia)
<gunter.van_de_velde=[e-mail]> wrote:

> # Gunter Van de Velde, RTG AD, comments for
> draft-ietf-bess-evpn-irb-extended-mobility-17
>
> Many thanks to Donald Eastlake for his Early RTGDIR review for
> draft-ietf-bess-evpn-irb-extended-mobility-10. The review improved the
> document quality. The enhancements to the document have been resolved to
> satisfaction according
> https://mailarchive.ietf.org/arch/msg/bess/ZAqF_aVRjbYi9H2hs6SgeZVU2-8/
>
> Thanks to Stephane Litkowski for his detailed Shepherd write-up.
>
> The document technical content seems complete, although it was not always
> easy to process. I tried to proposed alternate textblobs to improve
> readability while trying to keep the original message content. During this
> review process i noted observations when something was not clear and needed
> potentially clarification or correction.
>
> Great success to learn that there are at least 2 implementations of the
> draft
>
> #issues
> #======
> # This document is not mentioning SRv6 anywhere. Is that the intent? If
> yes, then maybe that should be explicit mentioned early in the document.
>
> # figure 3 seems to be missing some components: PE3, PE4 and ESI-2?
>
> # The abstract seems overly detailed. Please consider the suggested
> alternate abstract, higher level and more generic, with focus upon
> detailing more abstract the objective of
> draft-ietf-bess-evpn-irb-extended-mobility-17
>
>
> #DETAILED COMMENTS
> #=================
> ##classified as [minor] and [major] and [re-edit]
>
> 17      Abstract
>
> 19         The procedure to handle host mobility in a layer 2 Network with
> EVPN
> 20         control plane is defined as part of RFC7432.  EVPN has since
> evolved
> 21         to find wider applicability across various IRB use cases that
> include
> 22         distributing both MAC and IP reachability via a common EVPN
> control
> 23         plane.  MAC Mobility procedures defined in RFC7432 are
> extensible to
> 24         IRB use cases if a fixed 1:1 mapping between host IP and MAC is
> 25         assumed across host moves.  Generic mobility support for IP and
> MAC
> 26         addresses that allows these bindings to change across moves IS
> 27         REQUIRED to support a broader set of EVPN IRB use cases.  EVPN
> all-
> 28         active multi-homing further introduces scenarios that require
> 29         additional consideration from mobility perspective.  This
> document
> 30         enumerates a set of design considerations applicable to mobility
> 31         across these EVPN IRB use cases and updates sequence number
> 32         assignment procedures defined in RFC7432 to address these IRB
> use
> 33         cases.
>
> [major]
> This abstract is very detailed and makes it hard to understand on a high
> level what exactly the content of the draft is all about. I view upon a
> abstract as the textblob one gives to your people manager to make him/het
> understand what the document is all about. What about the following
> abstract textblob proposal, making high level draft intent better
> understandable for non-EVPN technology wizards
>
> "
> This document specifies extensions to RFC7432 Ethernet VPN (EVPN)
> Integrated Routing and Bridging (IRB) procedures to enhance the mobility
> mechanisms for EVPN IRB-based networks. The proposed extensions improve the
> handling of IP address mobility across EVPN networks by introducing a
> mechanism to track the movement of IP addresses and ensure seamless
> forwarding. These enhancements address the limitations in the existing EVPN
> IRB mobility procedures by providing more efficient and scalable solutions.
> The extensions are backward compatible with existing EVPN IRB
> implementations and aim to optimize network performance in scenarios
> involving frequent IP address mobility.
> "
>
> 127     1.  Introduction
> 128
> 129        EVPN-IRB enables advertising both MAC and IP routes via a single
> 130        MAC+IP RT-2 advertisement.  The MAC address is imported into the
> 131        local bridge MAC table and enables L2 bridged traffic across the
> 132        network overlay.  The IP address is imported into the local ARP
> table
> 133        in an asymmetric IRB design or imported into the IP routing
> table in
> 134        a symmetric IRB design, and enables routed traffic across the
> layer 2
> 135        network overlay.  Please refer to [RFC9135] for more background
> on
> 136        EVPN IRB forwarding modes.
>
> [re-edit]
> EVPN-IRB facilitates the advertisement of both MAC and IP routes via a
> single MAC+IP Route Type 2 (RT-2) advertisement. The MAC address is
> integrated into the local bridge MAC table, enabling Layer 2 (L2) bridged
> traffic across the network overlay. The IP address is incorporated into the
> local ARP table in an asymmetric IRB design, or into the IP routing table
> in a symmetric IRB design, facilitating routed traffic across the L2
> network overlay. For additional context on EVPN IRB forwarding modes, refer
> to [RFC9135].
>
> 138        To support EVPN mobility procedure, a single sequence number
> mobility
> 139        attribute is advertised with the combined MAC+IP route.  A
> single
> 140        sequence number advertised with the combined MAC+IP route to
> resolve
> 141        both MAC and IP reachability implicitly assumes a 1:1 fixed
> mapping
> 142        between IP and MAC.  While a fixed 1:1 mapping between IP and
> MAC is
> 143        a common use case that is addressed via existing MAC mobility
> 144        procedure defined in [RFC7432], additional IRB scenarios need
> to be
> 145        considered, that don't necessarily adhere to this assumption.
> Such
> 146        use cases are common in a virtualized host environment where
> hosts
> 147        attached to an EVPN network are virtual machines (VM) or
> 148        containerized workloads.  Following IRB mobility scenarios are
> 149        considered:
> 150
> 151        *  VM move results in VM IP and MAC moving together
> 152
> 153        *  VM move results in VM IP moving to a new MAC association
> 154
> 155        *  VM move results in VM MAC moving to a new IP association
>
> [re-edit]
> To support the EVPN mobility procedure, a single sequence number mobility
> attribute is advertised with the combined MAC+IP route. This approach,
> which resolves both MAC and IP reachability with a single sequence number,
> inherently assumes a fixed 1:1 mapping between IP and MAC. While this fixed
> 1:1 mapping is a common use case and is addressed via the existing MAC
> mobility procedure defined in [RFC7432], there are additional IRB scenarios
> that do not adhere to this assumption. Such scenarios are prevalent in
> virtualized host environments where hosts connected to an EVPN network are
> virtual machines (VMs) or containerized workloads. The following IRB
> mobility scenarios are considered:
>
> * A VM move results in the VM's IP and MAC moving together.
>
> * A VM move results in the VM's IP moving to a new MAC association.
>
> * A VM move results in the VM's MAC moving to a new IP association.
>
> 157        While existing MAC mobility procedure can be used for MAC+IP
> move in
> 158        the first scenario, subsequent scenarios result in a new MAC- IP
> 159        association.  As a result, a single sequence number assigned
> 160        independently per-{MAC, IP} is not sufficient to determine most
> 161        recent reachability for both MAC and IP, unless the sequence
> number
> 162        assignment algorithm allows for changing MAC-IP bindings across
> 163        moves.
> 163
> 165        This document updates sequence number assignment procedures
> defined
> 166        in [RFC7432] to adequately address mobility support across
> EVPN-IRB
> 167        overlay use cases that allow MAC-IP bindings to change across VM
> 168        moves and can support mobility for both MAC and IP components
> carried
> 169        in an EVPN RT-2 for these use cases.
>
> [re-edit]
> While the existing MAC mobility procedure can manage the MAC+IP move in
> the first scenario, the subsequent scenarios lead to new MAC-IP
> associations. Therefore, a single sequence number assigned independently
> per-{MAC, IP} is insufficient to determine the most recent reachability for
> both MAC and IP unless the sequence number assignment algorithm allows for
> changing MAC-IP bindings across moves.
>
> This document updates the sequence number assignment procedures defined in
> [RFC7432] to adequately address mobility support across EVPN-IRB overlay
> use cases that permit MAC-IP bindings to change across VM moves and support
> mobility for both MAC and IP components carried in an EVPN RT-2 for these
> use cases.
>
> 171        In addition, for hosts on an ESI multi-homed to multiple PE
> devices,
> 172        additional procedures are specified to ensure synchronized
> sequence
> 173        number assignments across the multi-homing devices.
> 174
> 175        This document covers mobility for the following cases,
> independent of
> 176        the overlay encapsulation (e.g.: MPLS, NVO Tunnel):
> 177
> 178        *  Symmetric EVPN IRB overlay
> 179
> 180        *  Asymmetric EVPN IRB overlay
> 181
> 182        *  Routed EVPN overlay
>
> [re-edit]
> Additionally, for hosts on an ESI multi-homed to multiple PE devices,
> additional procedures are specified to ensure synchronized sequence number
> assignments across the multi-homing devices.
>
> This document addresses mobility for the following cases, independent of
> the overlay encapsulation (e.g., MPLS, NVO Tunnel):
>
> * Symmetric EVPN IRB overlay
>
> * Asymmetric EVPN IRB overlay
>
> * Routed EVPN overlay
>
> 184     1.1.  Document Structure
> 185
> 186        Following sections of the document are informative:
> 187
> 188        *  section 3 provides the necessary background and problem
> statement
> 189           being addressed in this document.
> 190
> 191        *  section 4 lists the resulting design considerations for the
> 192           document.
> 193
> 194        Following sections of the document are normative:
> 195
> 196        *  section 6 describes the mobility and sequence number
> assigment
> 197           procedures in an EVPN-IRB overlay required to address the
> 198           scenarios described in section 4.
> 199
> 200        *  section 7 describes the mobility procedures for a routed
> overlay
> 201           network as opposed to an IRB overlay.
> 202
> 203        *  section 8 describes corresponding duplicate detection
> procedures
> 204           for EVPN-IRB and routed overlays.
>
> [minor]
> What about section 5? it exists in the draft. I assume the intent is
> informational
>
> 217        *  Underlay: IP or MPLS fabric core network that provides IP or
> MPLS
> 218           routed reachability between EVPN PEs.
>
> [major]
> Is SRv6 intentionally missing from this list?
>
> 220        *  Overlay: VPN or service layer network consisting of EVPN PEs
> OR
> 221           VPN provider-edge (PE) switch-router devices that runs on
> top of
> 222           an underlay routed core.
>
> [major]
> I believe that this is ambigious terminology. add RFC references to the
> base RFC that documents each type of overlay PE
>
> 233        *  Symmetric EVPN-IRB: An overlay fabric first-hop routing
> 234           architecture as defined in [RFC9135], wherein, overlay
> host-to-
> 235           host routed inter-subnet flows are routed at both ingress and
> 236           egress EVPN PEs.
>
> [re-edit]
> Symmetric EVPN-IRB: is a specific design approach used in EVPN-based
> networks [RFC9135] to handle both Layer 2 (L2) and Layer 3 (L3) forwarding
> within the same network infrastructure. The key characteristic of symmetric
> EVPN-IRB is that both ingress and egress PE routers perform routing for
> inter-subnet traffic.
>
> 238        *  Asymmetric EVPN-IRB: An overlay fabric first-hop routing
> 239           architecture as defined in [RFC9135], wherein, overlay
> host-to-
> 240           host routed inter-subnet flows are routed and bridged at
> ingress
> 241           PE and bridged at egress PEs.
>
> [re-edit]
> Asymmetric EVPN-IRB: is a design approach used in EVPN-based networks
> [RFC9135] to handle Layer 2 (L2) and Layer 3 (L3) forwarding. In this
> approach, only the ingress Provider Edge (PE) router performs routing for
> inter-subnet traffic, while the egress PE router performs bridging.
>
> 248        *  Ethernet-Segment: physical Ethernet or LAG port that
> connects an
> 249           access device to an EVPN PE, as defined in [RFC7432].
>
> [minor]
> s/physical Ethernet/Physical ethernet/
>
> 251        *  EVPN all-active multi-homing: PE-CE all-active multi-homing
> 252           achieved via a multi-homed layer-2 LAG interface on a CE with
> 253           member links to multiple PEs and related EVPN procedures on
> the
> 254           PEs.
>
> [re-edit]
> EVPN all-active multi-homing: is a redundancy and load-sharing mechanism
> used in EVPN networks. This method allows multiple PE devices to
> simultaneously provide Layer 2 and Layer 3 connectivity to a single CE
> device or network segment.
>
> 256        *  RT-2: EVPN route type 2 carrying both MAC and IP
> reachability.
> 257
> 258        *  RT-5: EVPN route type 5 carrying IP prefix reachability.
>
> [minor]
> add references to RFC7432
>
> 260        *  MAC-IP: IP association for a MAC, referred to in this
> document may
> 261           be IPv4, IPv6 or both.
>
> [minor]
> Is this specified in a document somewhere, or is this explicit to this
> document itself?
>
>
> 263        *  SYNC MAC route: In the context of EVPN multi-homing, this
> refers
> 264           to a local MAC route SYNCed from another PE sharing the same
> ESI.
> 265
> 266        *  SYNC MAC-IP route: In the context of EVPN multi-homing, this
> 267           refers to a local MAC-IP route SYNCed from another PE
> sharing the
> 268           same ESI.
> 269
> 270        *  SYNC MAC sequence number: In the context of EVPN
> multi-homing,
> 271           this refers to sequence number received with a SYNC MAC
> route.
> 272
> 273        *  SYNC MAC-IP sequence number: In the context of EVPN
> multi-homing,
> 274           this refers to sequence number received with a SYNC MAC-IP
> route.
>
>
> [minor]
> Is the SYNC something outlined in this draft itself, or is this procedure
> specified in another document?
> I assume this is based upon the priciples of RFC7432 using the MAC
> Mobility Extended Community
>
> 279     3.1.  Optional MAC only RT-2
> 280
> 281        In an EVPN IRB scenario, where a single MAC+IP RT-2
> advertisement
> 282        carries both IP and MAC routes, a MAC only RT-2 advertisement is
> 283        redundant for host MACs that are advertised via MAC+IP RT-2.
> As a
> 284        result, advertisement of a local MAC only RT-2 is an optional
> at an
> 285        EVPN PE.  This is an important consideration for mobility
> scenarios
> 286        discussed in subsequent sections.  Note that a local MAC and its
> 287        assigned sequence number is still maintained locally on a PE,
> and it
> 288        is just the advertisement of this route to other PEs that is
> 289        optional.
>
> 291        MAC only RT-2 may still be advertised for non-IP host MACs that
> are
> 292        not advertised via MAC+IP RT-2.
>
> [re-edit]
> In an EVPN IRB scenario, where a single MAC+IP RT-2 advertisement carries
> both IP and MAC routes, a MAC-only RT-2 advertisement becomes redundant for
> host MACs already advertised via MAC+IP RT-2. Consequently, the
> advertisement of a local MAC-only RT-2 is optional at an EVPN PE. This
> consideration is important for mobility scenarios discussed in subsequent
> sections. It is noteworthy that a local MAC and its assigned sequence
> number are still maintained locally on a PE, and only the advertisement of
> this route to other PEs is optional.
>
> MAC-only RT-2 advertisements may still be issued for non-IP host MACs that
> are not included in MAC+IP RT-2 advertisements.
>
> 294     3.2.  Mobility Use Cases
> 295
> 296        This section describes the IRB mobility use cases considered in
> this
> 297        document.  Procedures to address them are covered later in
> section 6
> 298        and section 7.
> 299
> 300        *  Host move results in Host IP and MAC moving together
> 301
> 302        *  Host move results in Host IP moving to a new MAC association
> 303
> 304        *  Host move results in Host MAC moving to a new IP association
>
> [re-edit]
> This section outlines the IRB mobility use cases addressed in this
> document. Detailed procedures to handle these scenarios are provided in
> Sections 6 and 7.
>
> * A host move results in both the host's IP and MAC addresses moving
> together.
>
> * A host move results in the host's IP address moving to a new MAC address
> association.
>
> * A host move results in the host's MAC address moving to a new IP address
> association.
>
> 306     3.2.1.  Host MAC+IP Move
> 307
> 308        This is the baseline case, wherein a host move results in both
> host
> 309        MAC and IP moving together with no change in MAC-IP binding
> across a
> 310        move.  Existing MAC mobility defined in [RFC7432] may be
> leveraged to
> 311        apply to corresponding MAC+IP route to support this mobility
> 312        scenario.
> 313
> 314     3.2.2.  Host IP Move to new MAC
> 315
> 316        This is the case, where a host move results in VM IP moving to
> a new
> 317        MAC binding.
> 318
> 319     3.2.2.1.  VM Reload
> 320
> 321        A host reload or an orchestrated host move that results in a
> host
> 322        being re-spawned at a new location may result in host getting a
> new
> 323        MAC assignment, while maintaining its existing IP address.  This
> 324        results in a host IP move to a new MAC binding:
> 325
> 326        IP-a, MAC-a ---> IP-a, MAC-b
> 327
> 328     3.2.2.2.  MAC Sharing
> 329
> 330        This takes into account scenarios, where multiple hosts, each
> with a
> 331        unique IP, may share a common MAC binding, and a host move
> results in
> 332        a new MAC binding for the host IP.
> 333
> 334        As an example, hosts running on a single physical server, each
> with a
> 335        unique IP, may share the same physical server MAC.  In yet
> another
> 336        scenario, an L2 access network may be behind a firewall, such
> that
> 337        all hosts IPs on the access network are learnt with a common
> firewall
> 338        MAC.  In all such "shared MAC" use cases, multiple local MAC-IP
> ARP
> 339        entries may be learnt with the same MAC.  A host IP move, in
> such
> 340        scenarios (for example, to a new physical server), could result
> in
> 341        new MAC association for the host IP.
> 342
> 343     3.2.2.3.  Problem
> 344
> 345        In both of the above scenarios, a combined MAC+IP EVPN RT-2
> 346        advertised with a single sequence number attribute implicitly
> assumes
> 347        a fixed IP to MAC mapping.  A host IP move to a new MAC breaks
> this
> 348        assumption and results in a new MAC+IP route.  If this new
> MAC+IP
> 349        route is independently assigned a new sequence number, the
> sequence
> 350        number can no longer be used to determine most recent host IP
> 351        reachability in a symmetric EVPN-IRB design OR the most recent
> IP to
> 352        MAC binding in an asymmetric EVPN-IRB design.
> 353
> 354                             +------------------------+
> 355                             | Underlay Network Fabric|
> 356                             +------------------------+
>
> 358          +-----+   +-----+      +-----+   +-----+      +-----+
>  +-----+
> 359          | PE1 |   | PE2 |      | PE3 |   | PE4 |      | PE5 |   | PE6
> |
> 360          +-----+   +-----+      +-----+   +-----+      +-----+
>  +-----+
> 361             \         /            \         /            \         /
> 362              \ ESI-1 /              \ ESI-2 /              \ ESI-3 /
> 363               \     /                \     /                \     /
> 364               +\---/+                +\---/+                +\---/+
> 365               | \ / |                | \ / |                | \ / |
> 366               +--+--+                +--+--+                +--+--+
> 367                  |                      |                      |
> 368             Server-M1              Server-M2              Server-M3
> 369                  |                      |                      |
> 370           VM-IP1, VM-IP2         VM-IP3, VM-IP4         VM-IP5, VM-IP6
> 371
> 372                                       Figure 1
> 373
> 374        As an example, consider a topology shown in Figure 1, with host
> VMs
> 375        sharing the physical server MAC.  In steady state, IP1-M1 route
> is
> 376        learnt at PE1, PE2 and advertised to remote PEs with a sequence
> 377        number N.  Now, VM-IP1 is moved to MAC Server-M2.  ARP or ND
> based
> 378        local learning at PE3, PE4 would now result in a new IP1-M2
> route
> 379        being learnt.  If route IP1-M2 is learnt as a new MAC+IP route
> and
> 380        assigned a new sequence number of say 0, mobility procedure for
> VM-
> 381        IP1 will not trigger across the overlay network.
> 382
> 383        A sequence number assignment procedure needs to be defined to
> 384        unambiguously determine the most recent IP reachability, IP to
> MAC
> 385        binding, and MAC reachability for such a MAC sharing scenario.
>
> [re-edit]
> 3.2.1. Host MAC+IP Move
> This is the baseline scenario where a host move results in both the host's
> MAC and IP addresses moving together without altering the MAC-IP binding.
> The existing MAC mobility procedures defined in [RFC7432] can be leveraged
> to support this MAC+IP mobility scenario.
>
> 3.2.2. Host IP Move to a New MAC
> This scenario involves a host move where the host's IP address is
> reassigned to a new MAC address.
>
> 3.2.2.1. VM Reload
> A host reload or orchestrated move may cause a host to be re-spawned at a
> new location, resulting in a new MAC assignment while retaining the
> existing IP address. This results in the host's IP moving to a new MAC
> binding, as shown below:
>
> IP-a, MAC-a ---> IP-a, MAC-b
>
> 3.2.2.2. MAC Sharing
> This scenario considers cases where multiple hosts, each with a unique IP
> address, share a common MAC address. A host move results in a new MAC
> binding for the host IP. For example, hosts running on a single physical
> server might share the same MAC. Alternatively, an L2 access network behind
> a firewall may have all host IPs learned with a common firewall MAC. In
> these "shared MAC" scenarios, multiple local MAC-IP ARP entries may be
> learned with the same MAC. A host IP move to a new physical server could
> result in a new MAC association for the host IP.
>
> 3.2.2.3. Problem
> In the aforementioned scenarios, a combined MAC+IP EVPN RT-2 advertised
> with a single sequence number attribute assumes a fixed IP-to-MAC mapping.
> A host IP move to a new MAC breaks this assumption and results in a new
> MAC+IP route. If this new route is independently assigned a new sequence
> number, the sequence number can no longer determine the most recent host IP
> reachability in a symmetric EVPN-IRB design or the most recent IP-to-MAC
> binding in an asymmetric EVPN-IRB design.
>
>                                 +------------------------+
>                                 | Underlay Network Fabric|
>                                 +------------------------+
>
>              +-----+   +-----+      +-----+   +-----+      +-----+
>  +-----+
>              | PE1 |   | PE2 |      | PE3 |   | PE4 |      | PE5 |   | PE6
> |
>              +-----+   +-----+      +-----+   +-----+      +-----+
>  +-----+
>                 \         /            \         /            \         /
>                  \ ESI-1 /              \ ESI-2 /              \ ESI-3 /
>                   \     /                \     /                \     /
>                   +\---/+                +\---/+                +\---/+
>                   | \ / |                | \ / |                | \ / |
>                   +--+--+                +--+--+                +--+--+
>                      |                      |                      |
>                 Server-M1              Server-M2              Server-M3
>                      |                      |                      |
>               VM-IP1, VM-IP2         VM-IP3, VM-IP4         VM-IP5, VM-IP6
>
>                                           Figure 1
>
> Figure 1 illustrates a topology with host VMs sharing the physical server
> MAC. In steady state, the IP1-M1 route is learned at PE1 and PE2 and
> advertised to remote PEs with a sequence number N. If VM-IP1 moves to
> Server-M2, ARP or ND-based local learning at PE3 and PE4 would result in a
> new IP1-M2 route. If this new route is assigned a sequence number of 0, the
> mobility procedure for VM-IP1 will not trigger across the overlay network.
>
> A sequence number assignment procedure must be defined to unambiguously
> determine the most recent IP reachability, IP-to-MAC binding, and MAC
> reachability for such MAC sharing scenarios.
>
> 387     3.2.3.  Host MAC move to new IP
> 388
> 389        This is a scenario where a host move or re-provisioning behind
> a new
> 390        gateway location may result in the host getting a new IP address
> 391        assigned, while keeping the same MAC.
> 392
> 393     3.2.3.1.  Problem
> 394
> 395        The complication with this scenario is that MAC reachability
> could be
> 396        carried via a combined MAC+IP route while a MAC only route may
> not be
> 397        advertised at all.  A single sequence number association with
> the
> 398        MAC+IP route again implicitly assumes a fixed mapping between
> MAC and
> 399        IP.  A MAC move resulting in a new IP association for the host
> MAC
> 400        breaks this assumption and results in a new MAC+IP route.  If
> this
> 401        new MAC+IP route independently assumes a new sequence number,
> this
> 402        mobility attribute can no longer be used to determine the most
> recent
> 403        host MAC reachability.
> 404
> 405                             +------------------------+
> 406                             | Underlay Network Fabric|
> 407                             +------------------------+
> 408          +-----+   +-----+      +-----+   +-----+      +-----+
>  +-----+
> 409          | PE1 |   | PE2 |      | PE3 |   | PE4 |      | PE5 |   | PE6
> |
> 410          +-----+   +-----+      +-----+   +-----+      +-----+
>  +-----+
> 411             \         /            \         /            \         /
> 412              \ ESI-1 /              \ ESI-2 /              \ ESI-3 /
> 413               \     /                \     /                \     /
> 414               +\---/+                +\---/+                +\---/+
> 415               | \ / |                | \ / |                | \ / |
> 416               +--+--+                +--+--+                +--+--+
> 417                  |                      |                      |
> 418               Server1                Server2                Server3
> 419                  |                      |                      |
> 420         VM-IP1-M1, VM-IP2-M2   VM-IP3-M3, VM-IP4-M4   VM-IP5-M5,
> VM-IP6-M6
> 421                                       Figure 2
> 422
> 423        As an example, consider a host VM IP1-M1 that is learnt locally
> at
> 424        PE1, PE2 and advertised to remote hosts with a sequence number
> N.
> 425        Consider a scenario where this VM with MAC M1 is re-provisioned
> at
> 426        server 2, however, as part of this re-provisioning, assigned a
> 427        different IP address say IP7.  IP7-M1 is learnt as a new route
> at
> 428        PE3, PE4 and advertised to remote PEs with a sequence number of
> 0.
> 429        As a result, L3 reachability to IP7 would be established across
> the
> 430        overlay, however, MAC mobility procedure for M1 will not
> trigger as a
> 431        result of this MAC-IP route advertisement.  If an optional MAC
> only
> 432        route is also advertised, sequence number associated with the
> MAC
> 433        only route would trigger MAC mobility as per [RFC7432].
> However, in
> 434        the absence of an additional MAC only route advertisement, a
> single
> 435        sequence number advertised with a combined MAC+IP route may not
> be
> 436        sufficient to update MAC reachability across the overlay.
> 437
> 438        A MAC-IP sequence number assignment procedure needs to be
> defined to
> 439        unambiguously determine the most recent MAC reachability in
> such a
> 440        scenario without a MAC only route being advertised.
> 441
> 442        Further, PE1/PE2, on learning new reachability for IP7-M1 via
> PE3/PE4
> 443        MUST probe and delete any local IPs associated with MAC M1,
> such as
> 444        IP1-M1 in the above example.
> 445
> 446        Arguably, MAC mobility sequence number defined in [RFC7432],
> could be
> 447        interpreted to apply only to the MAC part of MAC-IP route, and
> would
> 448        hence cover this scenario.  This interpretation could be
> considered a
> 449        clarification to [RFC7432] and one of the reasons for the common
> 450        sequence number assignment procedure across all MAC-IP mobility
> 451        scenarios detailed in this document.
>
> [re-edit]
> 3.2.3.  Host MAC move to new IP
> The complication in this scenario arises because MAC reachability can be
> carried via a combined MAC+IP route, whereas a MAC-only route may not be
> advertised. Associating a single sequence number with the MAC+IP route
> implicitly assumes a fixed MAC-to-IP mapping. A MAC move that results in a
> new IP association breaks this assumption and creates a new MAC+IP route.
> If this new route independently receives a new sequence number, the
> sequence number can no longer reliably indicate the most recent host MAC
> reachability.
>
>                                 +------------------------+
>                                 | Underlay Network Fabric|
>                                 +------------------------+
>              +-----+   +-----+      +-----+   +-----+      +-----+
>  +-----+
>              | PE1 |   | PE2 |      | PE3 |   | PE4 |      | PE5 |   | PE6
> |
>              +-----+   +-----+      +-----+   +-----+      +-----+
>  +-----+
>                 \         /            \         /            \         /
>                  \ ESI-1 /              \ ESI-2 /              \ ESI-3 /
>                   \     /                \     /                \     /
>                   +\---/+                +\---/+                +\---/+
>                   | \ / |                | \ / |                | \ / |
>                   +--+--+                +--+--+                +--+--+
>                      |                      |                      |
>                   Server1                Server2                Server3
>                      |                      |                      |
>             VM-IP1-M1, VM-IP2-M2   VM-IP3-M3, VM-IP4-M4   VM-IP5-M5,
> VM-IP6-M6
>                                           Figure 2
>
> For instance, consider host VM IP1-M1 learned locally at PE1 and PE2 and
> advertised to remote hosts with sequence number N. If this VM with MAC M1
> is re-provisioned at Server2 and assigned a different IP address (e.g.,
> IP7), the new IP7-M1 route learned at PE3 and PE4 would be advertised with
> sequence number 0. Consequently, L3 reachability to IP7 would be
> established across the overlay, but the MAC mobility procedure for M1 would
> not trigger due to the new MAC-IP route advertisement. Advertising an
> optional MAC-only route with its sequence number would trigger MAC mobility
> per [RFC7432]. However, without this additional advertisement, a single
> sequence number associated with a combined MAC+IP route may be insufficient
> to update MAC reachability across the overlay.
>
> A MAC-IP sequence number assignment procedure is required to unambiguously
> determine the most recent MAC reachability in such scenarios without
> advertising a MAC-only route.
>
> Furthermore, PE1 and PE2, upon learning new reachability for IP7-M1 via
> PE3 and PE4, must probe and delete any local IPs associated with MAC M1,
> such as IP1-M1.
>
> It could be argued that the MAC mobility sequence number defined in
> [RFC7432] applies only to the MAC part of a MAC-IP route, thus covering
> this scenario. This interpretation could serve as a clarification to
> [RFC7432] and supports the need for a common sequence number assignment
> procedure across all MAC-IP mobility scenarios detailed in this document.
>
> 453     3.3.  EVPN All Active multi-homed ES
> 454                                +------------------------+
> 455                                | Underlay Network Fabric|
> 456                                +------------------------+
> 457
> 458                                    +-----+   +-----+
> 459                                    | PE1 |   | PE2 |
> 460                                    +-----+   +-----+
> 461                                      \\         //
> 462                                       \\ ESI-1 //
> 463                                        \\     /X
> 464                                        +\\---//+
> 465                                        | \\ // |
> 466                                        +---+---+
> 467                                            |
> 468                                           CE1
> 469
> 470                                       Figure 3
> 471
> 472        Consider an EVPN-IRB overlay network shown in Figure 2, with
> hosts
> 473        multi-homed to two or more PE devices via an all-active
> multi-homed
> 474        ES.  MAC and ARP entries learnt on a local ES may also be
> 475        synchronized across the multi-homing PE devices sharing this ES.
> 476        This MAC and ARP SYNC enables local switching of intra and inter
> 477        subnet ECMP traffic flows from remote hosts.  In other words,
> local
> 478        MAC and ARP entries on a given ES may be learnt via local
> learning
> 479        and / or via sync from another PE device sharing the same ES.
> 480
> 481        For a host that is multi-homed to multiple PE devices via an
> all-
> 482        active ES interface, local learning of host MAC and MAC-IP at
> each PE
> 483        device is an independent asynchronous event, that is dependent
> on
> 484        traffic flow and or ARP / ND response from the host hashing to a
> 485        directly connected PE on the MC-LAG interface.  As a result,
> sequence
> 486        number mobility attribute value assigned to a locally learnt
> MAC or
> 487        MAC-IP route at each device may not always be the same,
> depending on
> 488        transient states on the device at the time of local learning.
> 489
> 490        As an example, consider a host VM that is deleted from ESI-2 and
> 491        moved to ESI-1.  It is possible for host to be learnt on PE1
> 492        following deletion of the remote route from PE3, PE4, while
> being
> 493        learnt on PE2 prior to deletion of remote route from PE3, PE4.
> If
> 494        so, PE1 would process local host route learning as a new route
> and
> 495        assign a sequence number of 0, while PE2 would process local
> host
> 496        route learning as a remote to local move and assign a sequence
> number
> 497        of N+1, N being the existing sequence number assigned at PE3,
> PE4.
> 498
> 499        Inconsistent sequence numbers advertised from multi-homing
> devices
> 500        introduces:
> 501
> 502        *  Ambiguity with respect to how the remote PEs should handle
> paths
> 503           with same ESI and different sequence numbers.  A remote PE
> might
> 504           not program ECMP paths if it receives routes with different
> 505           sequence numbers from a set of multi-homing PEs sharing the
> same
> 506           ESI.
> 507
> 508        *  Breaks consistent route versioning across the network
> overlay that
> 509           is needed for EVPN mobility procedures to work.
> 510
> 511        As an example, in this inconsistent state, PE2 would drop a
> remote
> 512        route received for the same host with sequence number N (as its
> local
> 513        sequence number is N+1), while PE1 would install it as the best
> route
> 514        (as its local sequence number is 0).
> 515
> 516        There is need for a mechanism to ensure consistency of sequence
> 517        numbers advertised from a set of multi-homing devices for EVPN
> 518        mobility to work reliably.
> 519
> 520        In order to support mobility for multi-homed hosts using the
> sequence
> 521        number mobility attribute, local MAC and MAC-IP routes learnt
> on a
> 522        multi-homed ES MUST be advertised with the same sequence number
> by
> 523        all PE devices that the ES is multi-homed to.  There is need
> for a
> 524        mechanism to ensure consistency of sequence numbers assigned
> across
> 525        these PEs.
>
> [major]
> * The text talks about PE3 and PE4 and about ESI-2, but the figure does
> not show this.
> Can figure be corrected to show these components?
> This will make it more clear how inconsistency with sequence numbers
> manifests.
>
> [minor]
> unsure why in thi informational section in the last paragraph uppercase
> MUST is used. BCP14 language does not apply to informational textblobs
>
> [re-edit]
> 3.3.  EVPN All Active multi-homed ES
>
>                            +------------------------+
>                            | Underlay Network Fabric|
>                            +------------------------+
>
>                                +-----+   +-----+
>                                | PE1 |   | PE2 |
>                                +-----+   +-----+
>                                  \\         //
>                                   \\ ESI-1 //
>                                    \\     /X
>                                    +\\---//+
>                                    | \\ // |
>                                    +---+---+
>                                        |
>                                       CE1
>
>                                   Figure 3
>
> Consider an EVPN-IRB overlay network illustrated in Figure 3, where hosts
> are multi-homed to two or more PE devices via an all-active multi-homed ES.
> MAC and ARP entries learned on a local ES may also be synchronized across
> the multi-homing PE devices sharing this ES. This synchronization enables
> local switching of intra- and inter-subnet ECMP traffic flows from remote
> hosts. Thus, local MAC and ARP entries on a given ES may be learned through
> local learning and/or synchronization from another PE device sharing the
> same ES.
>
> For a host that is multi-homed to multiple PE devices via an all-active ES
> interface, the local learning of host MAC and MAC-IP at each PE device is
> an independent asynchronous event, dependent on traffic flow or ARP/ND
> response from the host hashing to a directly connected PE on the MC-LAG
> interface. Consequently, the sequence number mobility attribute value
> assigned to a locally learned MAC or MAC-IP route at each device may not
> always be the same, depending on transient states on the device at the time
> of local learning.
>
> For example, consider a host VM that is deleted from ESI-2 and moved to
> ESI-1. It is possible for the host to be learned on PE1 following the
> deletion of the remote route from PE3 and PE4, while being learned on PE2
> prior to the deletion of the remote route from PE3 and PE4. In this case,
> PE1 would process local host route learning as a new route and assign a
> sequence number of 0, while PE2 would process local host route learning as
> a remote-to-local move and assign a sequence number of N+1, where N is the
> existing sequence number assigned at PE3 and PE4.
>
> Inconsistent sequence numbers advertised from multi-homing devices
> introduce:
>
> * Ambiguity regarding how remote PEs should handle paths with the same ESI
> but different sequence numbers. A remote PE might not program ECMP paths if
> it receives routes with different sequence numbers from a set of
> multi-homing PEs sharing the same ESI.
> * Disruption of consistent route versioning across the network overlay,
> which is necessary for EVPN mobility procedures to function correctly.
>
> For instance, in this inconsistent state, PE2 would drop a remote route
> received for the same host with sequence number N (since its local sequence
> number is N+1), while PE1 would install it as the best route (since its
> local sequence number is 0).
>
> To support mobility for multi-homed hosts using the sequence number
> mobility attribute, local MAC and MAC-IP routes learned on a multi-homed ES
> must be advertised with the same sequence number by all PE devices to which
> the ES is multi-homed. There is a need for a mechanism to ensure the
> consistency of sequence numbers assigned across these PEs.
>
> 527     4.  Design Considerations
> 528
> 529        To summarize, sequence number assignment scheme and
> implementation
> 530        must take following considerations into account:
> 531
> 532        *  MAC+IP may be learnt on an ES multi-homed to multiple PE
> devices,
> 533           hence requires sequence numbers to be synchronized across
> multi-
> 534           homing PE devices.
> 535
> 536        *  MAC only RT-2 is optional in an IRB scenario and may not
> 537           necessarily be advertised in addition to MAC+IP RT-2.
> 538
> 539        *  A single MAC may be associated with multiple IPs, i.e.,
> multiple
> 540           host IPs may share a common MAC.
> 541
> 542        *  A host IP move could result in host moving to a new MAC,
> resulting
> 543           in a new IP to MAC association and a new MAC+IP route.
> 544
> 545        *  A host MAC move to a new location could result in host MAC
> being
> 546           associated with a different IP address, resulting in a new
> MAC to
> 547           IP association and a new MAC+IP route.
> 548
> 549        *  Local MAC-IP learn via ARP would always accompanied by a
> local MAC
> 550           learn event resulting from the ARP packet.  MAC and MAC-IP
> 551           learning, however, could happen in any order.
> 552
> 553        *  Use cases discussed earlier that do not maintain a constant
> 1:1
> 554           MAC-IP mapping across moves could potentially be addressed by
> 555           using separate sequence numbers associated with MAC and IP
> 556           components of MAC+IP route.  Maintaining two separate
> sequence
> 557           numbers however adds significant overhead with respect to
> 558           complexity, debugability, and backward compatibility.
> Hence, this
> 559           document addresses these requirements via a single sequence
> number
> 560           attribute.
>
> [re-edit]
> To summarize, the sequence number assignment scheme and implementation
> must consider the following:
>
> * Synchronization Across Multi-Homing PE Devices: MAC+IP may be learned on
> an ES multi-homed to multiple PE devices, requiring synchronized sequence
> numbers across these devices.
>
> * Optional MAC-Only RT-2: In an IRB scenario, MAC-only RT-2 is optional
> and may not be advertised alongside MAC+IP RT-2.
>
> * Multiple IPs Associated with a Single MAC: A single MAC may be linked to
> multiple IP addresses, indicating multiple host IPs sharing a common MAC.
>
> * Host IP Movement: A host IP move may result in a new MAC association,
> necessitating a new IP to MAC association and a new MAC+IP route.
>
> * Host MAC Movement: A host MAC move may result in a new IP association,
> requiring a new MAC to IP association and a new MAC+IP route.
>
> * Local MAC-IP Learning via ARP: Local MAC-IP learning via ARP always
> accompanies a local MAC learning event resulting from the ARP packet.
> However, MAC and MAC-IP learning can occur in any order.
>
> * Separate Sequence Numbers for MAC and IP: Use cases that do not maintain
> a constant 1:1 MAC-IP mapping across moves could potentially be addressed
> by using separate sequence numbers for MAC and IP components of the MAC+IP
> route. However, maintaining two separate sequence numbers adds significant
> complexity, debugging challenges, and backward compatibility issues.
> Therefore, this document addresses these requirements using a single
> sequence number attribute.
>
> 562     5.  Solution Components
> 563
> 564        This section goes over the main components of the EVPN IRB
> mobility
> 565        solution specified in this document.  Later sections will
> specify
> 566        exact sequence number assignment procedures resulting from
> concepts
> 567        described in this section.
> 568
> 569     5.1.  Sequence Number Inheritance
> 570
> 571        The main idea presented here is to view a local MAC-IP route as
> a
> 572        child of the corresponding local MAC route within the local
> context
> 573        of a PE, such that the local MAC-IP route inherits the sequence
> 574        number attribute from the parent local MAC only route:
> 575
> 576        Mx-IPx -----> Mx (seq# = N)
> 578
> 578        As a result, both parent MAC and child MAC-IP routes share one
> common
> 579        sequence number associated with the parent MAC route.  Doing so
> 580        ensures that a single sequence number attribute carried in a
> combined
> 581        MAC+IP route represents sequence number for both a MAC only
> route as
> 582        well as a MAC+IP route, and hence makes advertisement of the
> MAC only
> 583        route truly optional.  As a result, optional MAC only route
> with its
> 584        own sequence number is not required to establish the most recent
> 585        reachability for a MAC in the overlay network.  Specifically,
> this
> 586        enables a MAC to assume a different IP address on a move, and
> still
> 587        be able to establish the most recent reachability to the MAC
> across
> 588        the overlay network via the mobility attribute associated with
> the
> 589        MAC+IP route advertisement.  As an example, when Mx moves to a
> new
> 590        location, it would result in local Mx being assigned a higher
> 591        sequence number at its new location as per [RFC7432].  If this
> move
> 592        results in Mx assuming a different IP address, IPz, local Mx+IPz
> 593        route would inherit the new sequence number from Mx.
> 594
> 595        Local MAC and local MAC-IP routes would typically be sourced
> from
> 596        data plane learning and ARP learning respectively, and could get
> 597        learnt in the control plane in any order.  Implementation could
> 598        either replicate the inherited sequence number in each MAC-IP
> entry
> 599        OR maintain a single attribute in the parent MAC by creating a
> 600        forward reference local MAC object for cases where a local
> MAC-IP is
> 601        learnt before the local MAC.
> 602
> 603     5.2.  MAC Sharing
>
> 605        Further, for the shared MAC scenario, this results in multiple
> local
> 606        MAC-IP siblings inheriting a sequence number attribute from the
> 607        common parent MAC route:
> 608
> 609          Mx-IP1 -----
> 610           |          |
> 611          Mx-IP2 -----
> 612            .         |
> 613            .         +---> Mx (seq# = N)
> 614            .         |
> 615          Mx-IPw -----
> 616            |         |
> 617          Mx-IPx -----
> 627
> 619                                       Figure 4
> 620
> 621        In such a case, a host-IP move to a different physical server
> would
> 622        result in IP moving to a new MAC binding.  A new MAC-IP route
> 623        resulting from this move must now be advertised with a sequence
> 624        number that is higher than the previous MAC-IP route for this
> IP,
> 625        advertised from the prior location.  As an example, consider a
> route
> 626        Mx-IPx that is currently advertised with sequence number N from
> PE1.
> 627        IPx moving to a new physical server behind PE2 results in IPx
> being
> 628        associated with MAC Mz.  A new local Mz-IPx route resulting
> from this
> 629        move at PE2 must now be advertised with a sequence number
> higher than
> 630        N and higher than the previous Mz sequence number M.  This is
> so that
> 631        PE devices, including PE1, PE2, and other remote PE devices
> that are
> 632        part of the overlay can clearly determine and program the most
> recent
> 633        MAC binding and reachability for the IP.  PE1, on receiving
> this new
> 634        Mz-IPx route with sequence number say, N+1, for symmetric IRB
> case,
> 635        would update IPx reachability via PE2 in forwarding, for
> asymmetric
> 636        IRB case, would update IPx's ARP binding to Mz.  In addition,
> PE1
> 637        would clear and withdraw the stale Mx-IPx route with the lower
> 638        sequence number.
> 639
> 640        This also implies that sequence number associated with local
> MAC Mz
> 641        and all local MAC-IP children of Mz at PE2 must now be
> incremented to
> 642        N+1 or to M+1 if the previous Mz sequence number M is greater
> than N,
> 643        and re-advertised across the overlay.  While this
> re-advertisement of
> 644        all local MAC-IP children routes affected by the parent MAC
> route is
> 645        an overhead, it avoids the need for two separate sequence number
> 646        attributes to be maintained and advertised for IP and MAC
> components
> 647        of MAC+IP RT-2.  Implementation would need to be able to lookup
> MAC-
> 648        IP routes for a given IP and update sequence number for it's
> parent
> 649        MAC and its MAC-IP children.
> 650
> 651     5.3.  Multi-homing Mobility Synchronization
> 652
> 653        In order to support mobility for multi-homed hosts, local MAC
> and
> 654        MAC-IP routes learnt on a shared ES MUST be advertised with the
> same
> 655        sequence number by all PE devices that the ES is multi-homed to.
> 656        This also applies to local MAC only routes. local MAC and
> MAC-IP may
> 657        be learnt natively via data plane and ARP/ND respectively as
> well as
> 658        via SYNC from another multi-homing PE to achieve local
> switching.
> 659        Local and SYNC route learning can happen in any order.  Local
> MAC-IP
> 660        routes advertised by all multi-homing PE devices sharing the ES
> must
> 661        carry the same sequence number, independent of the order in
> which
> 662        they are learnt.  This implies:
> 663
> 664        *  On local or SYNC MAC-IP route learning, sequence number for
> the
> 665           local MAC-IP route MUST be compared and updated to the higher
> 666           value.
> 667
> 668        *  On local or SYNC MAC route learning, sequence number for the
> local
> 669           MAC route MUST be compared and updated to the higher value.
> 670
> 671        If an update to local MAC-IP sequence number is required as a
> result
> 672        of the above comparison with SYNC MAC-IP route, it would
> essentially
> 673        amount to a sequence number update on the parent local MAC,
> resulting
> 674        in inherited sequence number update on the MAC-IP route.
>
> [major]
> * is the arrow used in the small figure correct? Should it not be the
> other way around if the sequence number is inherited? w.o.w. Mx (seq# = N)
> -----> Mx-IPx ?
> * similar with the other figure in section 5.2
>
> [re-edit]
> 5. Solution Components
> This section outlines the main components of the EVPN IRB mobility
> solution specified in this document. Subsequent sections will detail the
> exact sequence number assignment procedures based on the concepts described
> here.
>
> 5.1. Sequence Number Inheritance
> The key concept presented here is to treat a local MAC-IP route as a child
> of the corresponding local MAC route within the local context of a PE. This
> ensures that the local MAC-IP route inherits the sequence number attribute
> from the parent local MAC-only route:
>
>            Mx-IPx -----> Mx (seq# = N)
>
> Thus, both the parent MAC and child MAC-IP routes share a common sequence
> number associated with the parent MAC route. This ensures that a single
> sequence number attribute carried in a combined MAC+IP route represents the
> sequence number for both a MAC-only route and a MAC+IP route, making the
> advertisement of the MAC-only route truly optional. This enables a MAC to
> assume a different IP address upon moving and still establish the most
> recent reachability to the MAC across the overlay network via the mobility
> attribute associated with the MAC+IP route advertisement. For instance,
> when Mx moves to a new location, it would be assigned a higher sequence
> number at its new location per [RFC7432]. If this move results in Mx
> assuming a different IP address, IPz, the local Mx+IPz route would inherit
> the new sequence number from Mx.
>
> Local MAC and local MAC-IP routes are typically sourced from data plane
> learning and ARP learning, respectively, and can be learned in the control
> plane in any order. Implementation can either replicate the inherited
> sequence number in each MAC-IP entry or maintain a single attribute in the
> parent MAC by creating a forward reference local MAC object for cases where
> a local MAC-IP is learned before the local MAC.
>
> 5.2. MAC Sharing
> For the shared MAC scenario, multiple local MAC-IP siblings inherit the
> sequence number attribute from the common parent MAC route:
>
>  Mx-IP1 -----
>   |          |
>  Mx-IP2 -----
>    .         |
>    .         +---> Mx (seq# = N)
>    .         |
>  Mx-IPw -----
>    |         |
>  Mx-IPx -----
>
> In such cases, a host-IP move to a different physical server results in
> the IP moving to a new MAC binding. A new MAC-IP route resulting from this
> move must be advertised with a sequence number higher than the previous
> MAC-IP route for this IP, advertised from the prior location. For example,
> consider a route Mx-IPx currently advertised with sequence number N from
> PE1. If IPx moves to a new physical server behind PE2 and is associated
> with MAC Mz, the new local Mz-IPx route must be advertised with a sequence
> number higher than N and the previous Mz sequence number M. This allows PE
> devices, including PE1, PE2, and other remote PE devices, to determine and
> program the most recent MAC binding and reachability for the IP. PE1, upon
> receiving this new Mz-IPx route with sequence number N+1, would update IPx
> reachability via PE2 for symmetric IRB and update IPx's ARP binding to Mz
> for asymmetric IRB, while clearing and withdrawing the stale Mx-IPx route
> with the lower sequence number.
>
> This implies that the sequence number associated with local MAC Mz and all
> local MAC-IP children of Mz at PE2 must be incremented to N+1 or M+1 if the
> previous Mz sequence number M is greater than N and re-advertised across
> the overlay. While this re-advertisement of all local MAC-IP children
> routes affected by the parent MAC route adds overhead, it avoids the need
> for maintaining and advertising two separate sequence number attributes for
> IP and MAC components of MAC+IP RT-2. Implementation must be able to look
> up MAC-IP routes for a given IP and update the sequence number for its
> parent MAC and its MAC-IP children.
>
> 5.3. Multi-Homing Mobility Synchronization
> To support mobility for multi-homed hosts, local MAC and MAC-IP routes
> learned on a shared ES must be advertised with the same sequence number by
> all PE devices to which the ES is multi-homed. This applies to local
> MAC-only routes as well. Local MAC and MAC-IP may be learned natively via
> data plane and ARP/ND respectively, as well as via SYNC from another
> multi-homing PE to achieve local switching. Local and SYNC route learning
> can occur in any order. Local MAC-IP routes advertised by all multi-homing
> PE devices sharing the ES must carry the same sequence number, independent
> of the order in which they are learned. This implies:
>
> * On local or SYNC MAC-IP route learning, the sequence number for the
> local MAC-IP route must be compared and updated to the higher value.
>
> * On local or SYNC MAC route learning, the sequence number for the local
> MAC route must be compared and updated to the higher value.
>
> If an update to the local MAC-IP sequence number is required as a result
> of the comparison with the SYNC MAC-IP route, it essentially amounts to a
> sequence number update on the parent local MAC, resulting in an inherited
> sequence number update on the MAC-IP route.
>
> 676     6.  Requirements for Sequence Number Assignment
> 677
> 678        Following sections specify sequence number assignment procedure
> 679        needed on local and SYNC MAC and MAC-IP route learning events in
> 680        order to accomplish the above.
> 681
> 682     6.1.  Local MAC-IP learning
> 683
> 684        A local Mx-IPx learning via ARP or ND should result in
> computation OR
> 685        re-computation of the parent MAC Mx's sequence number, following
> 686        which the MAC-IP route Mx-IPx would simply inherit parent MAC's
> 687        sequence number.  The parent MAC Mx Sequence number MUST be
> computed
> 688        as follows:
> 689
> 690        *  MUST be higher than any existing remote MAC route for Mx, as
> per
> 691           [RFC7432].
> 692
> 693        *  MUST be at least equal to corresponding SYNC MAC sequence
> number
> 694           if one is present.
> 695
> 696        *  If the IP is also associated with a different remote MAC
> "Mz",
> 697           MUST be higher than the "Mz" sequence number.
> 698
> 699        Once the new sequence number for MAC route Mx is computed as per
> 700        above, all local MAC-IPs associated with MAC Mx MUST inherit the
> 701        updated sequence number.
> 702
> 703     6.2.  Local MAC learning
> 704
> 705        The local MAC Mx Sequence number MUST be computed as follows:
> 705
> 707        *  MUST be higher than any existing remote MAC route for Mx, as
> per
> 708           [RFC7432].
> 709
> 710        *  MUST be at least equal to the corresponding SYNC MAC sequence
> 711           number if one is present.
> 712
> 713        *  Once the new sequence number for MAC route Mx is computed as
> per
> 714           above, all local MAC-IPs associated with MAC Mx MUST inherit
> the
> 715           updated sequence number.
> 716
> 717        Note that the local MAC sequence number might already be
> present if
> 718        there was a local MAC-IP learnt prior to the local MAC, in
> which case
> 719        the above may not result in any change in local MAC's sequence
> 720        number.
> 721
> 722     6.3.  Remote MAC or MAC-IP Update
> 723
> 724        On receiving a remote MAC OR MAC-IP route update associated
> with a
> 725        MAC Mx with a sequence number that is
> 726
> 727        *  either higher than the sequence number assigned to a local
> route
> 728           for MAC Mx,
> 729
> 730        *  or equal to the sequence number assigned to a local route
> for MAC
> 731           Mx, but the remote route is selected as best because of
> lower VTEP
> 732           IP as per [RFC7432],
> 733
> 734        following handling IS REQUIRED on the receiving PE:
> 735
> 736        *  the PE MUST trigger probe and deletion procedure for all
> local IPs
> 737           associated with MAC Mx.
> 738
> 739        *  the PE MUST trigger deletion procedure for local MAC route
> for Mx.
> 740
> 741     6.4.  REMOTE (SYNC) MAC update
> 742
> 743        On receiving a REMOTE SYNC, the corresponding local MAC Mx (if
> 744        present) sequence number should be re- computed as follows:
> 745
> 746        *  If the current sequence number is less than the received
> SYNC MAC
> 747           sequence number, it MUST be increased to be equal to
> received SYNC
> 748           MAC sequence number.
> 749
> 750        *  If a local MAC sequence number is updated as a result of the
> 751           above, all local MAC-IPs associated with MAC Mx MUST inherit
> the
> 752           updated sequence number.
> 753
> 754     6.5.  REMOTE (SYNC) MAC-IP update
> 755
> 756        Receiving a SYNC MAC-IP for a locally attached host results in a
> 757        derived SYNC MAC Mx route entry, as MAC only RT-2 advertisement
> is
> 758        optional.  The corresponding local MAC Mx (if present) sequence
> 759        number should be re-computed as follows:
> 760
> 761        *  If the current sequence number is less than the received
> SYNC MAC
> 762           sequence number, it MUST be increased to be equal to
> received SYNC
> 763           MAC sequence number.
> 764
> 765        *  If a local MAC sequence number is updated as a result of the
> 766           above, all local MAC-IPs associated with MAC Mx MUST inherit
> the
> 767           updated sequence number.
> 768
> 769     6.6.  Interoperability
> 770
> 771        In general, if all PE nodes in the overlay network follow the
> above
> 772        sequence number assignment procedures, and the PE is
> advertising both
> 773        MAC+IP and MAC routes, sequence numbers advertised with the MAC
> and
> 774        MAC+IP routes with the same MAC would always be the same.
> However,
> 775        an inter-op scenario with a different implementation could
> arise,
> 776        where a PE implementation non-compliant with this document or
> with
> 777        [RFC7432] assigns and advertises independent sequence numbers
> to MAC
> 778        and MAC+IP routes.  To handle this case, if different sequence
> 779        numbers are received for remote MAC+IP and corresponding remote
> MAC
> 780        routes from a remote PE, sequence number associated with the
> remote
> 781        MAC route MUST be computed as:
> 782
> 783        *  Highest of all the received sequence numbers with remote
> MAC+IP
> 784           and MAC routes with the same MAC.
> 785
> 786        *  MAC sequence number would be re-computed on a MAC or MAC+IP
> route
> 787           withdraw as per above.
> 788
> 789        A MAC and / or IP move to the local PE would now result in the
> MAC
> 790        (and hence all MAC-IP) sequence numbers being incremented from
> the
> 791        above computed remote MAC sequence number.
> 792
> 793        If MAC only routes are not advertised at all, and different
> sequence
> 794        numbers are received with multiple MAC+IP routes for a given
> MAC, the
> 795        sequence number associated with the derived remote MAC route
> should
> 796        still be computed as the highest of all of the received MAC+IP
> 797        sequence numbers with the same MAC.
> 798
> 799     6.7.  MAC Sharing Race Condition
> 800
> 801        In a MAC sharing use case described in section 5.2, a race
> condition
> 802        is possible with simultaneous host moves between a pair of
> PEs.  As
> 803        an example, consider PE1 with local host IPs I1 and I2 sharing
> MAC
> 804        M1, and PE2 with local host IPs I3 and I4 sharing MAC M2.  A
> 805        simultaneous move of I1 from PE1 to PE2 and of I3 from PE2 to
> PE1,
> 806        such that I3 is learnt on PE1 before I1's local entry has been
> probed
> 807        out on PE1 and/or I1 is learnt on PE2 before I3's local entry
> has
> 808        been probed out on PE2 may trigger a race condition.  This race
> 809        condition together with MAC sequence number assignment rules
> defined
> 810        in section 6.1 can cause new mac-ip routes I1-M2 and I3-M1 to
> bounce
> 811        a couple of times with an incremented sequence number until
> stale
> 812        entries I1-M1 and I3-M2 have been probed out from PE1 and PE2
> 813        respectively.  An implementation MUST ensure proper probing
> 814        procedures to remove stale ARP, ND, and local MAC entries,
> following
> 815        a move, on learning remote routes as defined in section 6.3
> (and as
> 816        per [RFC9135]) to minimize exposure to this race condition.
> 817
> 818     6.8.  Mobility Convergence
> 819
> 820        This sections is optional and details ARP and ND probing
> procedures
> 821        that MAY be implemented to achieve faster host re- learning and
> 822        convergence on mobility events.
> 822
> 824        *  Following a host move from PE1 to PE2, the host's MAC is
> 825           discovered at PE2 as a local MAC via a data frames received
> from
> 826           the host.  If PE2 has a prior remote MAC-IP host route for
> this
> 827           MAC from PE1, an ARP/ND probe MAY be triggered at PE2 to
> learn the
> 828           MAC-IP as a local adjacency and trigger EVPN RT-2
> advertisement
> 829           for this MAC-IP across the overlay with new reachability via
> PE2.
> 830           This results in a reliable "event based" host IP learning
> 831           triggered by a "MAC learning event" across the overlay, and
> hence
> 832           faster convergence of overlay routed flows to the host.
> 833
> 834        *  Following a host move from PE1 to PE2, once PE1 receives a
> MAC or
> 835           MAC-IP route from PE2 with a higher sequence number, an
> ARP/ND
> 836           probe MAY be triggered at PE1 to clear the stale local MAC-IP
> 837           neighbor adjacency or to re-learn the local MAC-IP in case
> the
> 838           host has moved back or is duplicate.
> 838
> 840        *  Following a local MAC age-out, if there is a local IP
> adjacency
> 841           with this MAC, an ARP/ND probe MAY be triggered for this IP
> to
> 842           either re-learn the local MAC and maintain local l3 and l2
> 843           reachability to this host or to clear the ARP/ND entry in
> case the
> 844           host is indeed no longer local.  Note that this accomplishes
> 845           clearance of stale ARP entries, triggered by a MAC age-out
> event
> 846           even when the ARP refresh timer was longer than the MAC
> age-out
> 847           timer.  Clearing of stale IP neighbor entries in-turn
> facilitates
> 848           traffic convergence in the event that the host was silent
> and not
> 849           discovered at its new location.  Once the stale neighbor
> entry for
> 850           the host is cleared, routed traffic flow destined for the
> host can
> 851           re-trigger ARP/ND discovery for this host at the new
> location.
> 852
> 853     6.8.1.  Generalized Probing Logic
> 854
> 855        The above probing logic may be generalized as probing for an IP
> 856        neighbor anytime a resolving parent MAC route is "inconsistent"
> with
> 857        the MAC- IP neighbor route, where being inconsistent is defined
> as
> 858        being not present or conflicting in terms of the route source
> being
> 859        local OR remote.  The MAC-IP to MAC parent relationship
> described
> 860        earlier in this document in section 5.1 MAY be used to achieve
> this
> 861        logic.
>
> [major]
> * for my own understanding: in section 6.2 first bullet point, make me
> wonder if the connected ESI is share between two PEs. Would the requirement
> potentially lead to a count to infinity when two PEs connect to a shared
> ESI?
>
> * section 6.6: How would an implementation detect that the remote
> implementation does not support the behavior? Could that be explicit
> explained in the text?
>
> * section 6.7: THis section i did not understand. Too many moving parts.
> Can this be explained more explicit or elaborative?
>
> * section 6.8: What network figure is referenced towards?
>
> [re-edit]
> 6. Requirements for Sequence Number Assignment
> The following sections specify the sequence number assignment procedures
> required for local and SYNC MAC and MAC-IP route learning events to achieve
> the objectives outlined.
>
> 6.1. Local MAC-IP Learning
> A local Mx-IPx learning via ARP or ND should result in the computation or
> re-computation of the parent MAC Mx's sequence number, following which the
> MAC-IP route Mx-IPx inherits the parent MAC's sequence number. The parent
> MAC Mx sequence number MUST be computed as follows:
>
> * MUST be higher than any existing remote MAC route for Mx, as per
> [RFC7432].
>
> * MUST be at least equal to the corresponding SYNC MAC sequence number, if
> present.
>
> * If the IP is also associated with a different remote MAC "Mz," it MUST
> be higher than the "Mz" sequence number.
>
> Once the new sequence number for MAC route Mx is computed as per the above
> criteria, all local MAC-IPs associated with MAC Mx MUST inherit the updated
> sequence number.
>
> 6.2. Local MAC Learning
> The local MAC Mx sequence number MUST be computed as follows:
>
> * MUST be higher than any existing remote MAC route for Mx, as per
> [RFC7432].
>
> * MUST be at least equal to the corresponding SYNC MAC sequence number, if
> present.
>
> Once the new sequence number for MAC route Mx is computed as per the above
> criteria, all local MAC-IPs associated with MAC Mx MUST inherit the updated
> sequence number. Note that the local MAC sequence number might already be
> present if there was a local MAC-IP learned prior to the local MAC, in
> which case the above may not result in any change in the local MAC's
> sequence number.
>
> 6.3. Remote MAC or MAC-IP Update
> Upon receiving a remote MAC or MAC-IP route update associated with a MAC
> Mx with a sequence number that is:
>
> * Either higher than the sequence number assigned to a local route for MAC
> Mx,
>
> * Or equal to the sequence number assigned to a local route for MAC Mx,
> but the remote route is selected as best due to a lower VTEP IP as per
> [RFC7432],
>
> the following actions are REQUIRED on the receiving PE:
>
> * The PE MUST trigger a probe and deletion procedure for all local IPs
> associated with MAC Mx.
>
> * The PE MUST trigger a deletion procedure for the local MAC route for Mx.
>
> 6.4. REMOTE (SYNC) MAC Update
> Upon receiving a REMOTE SYNC, the corresponding local MAC Mx (if present)
> sequence number should be re-computed as follows:
>
> * If the current sequence number is less than the received SYNC MAC
> sequence number, it MUST be increased to be equal to the received SYNC MAC
> sequence number.
>
> * If a local MAC sequence number is updated as a result of the above, all
> local MAC-IPs associated with MAC Mx MUST inherit the updated sequence
> number.
>
> 6.5. REMOTE (SYNC) MAC-IP Update
> Receiving a SYNC MAC-IP for a locally attached host results in a derived
> SYNC MAC Mx route entry, as the MAC-only RT-2 advertisement is optional.
> The corresponding local MAC Mx (if present) sequence number should be
> re-computed as follows:
>
> * If the current sequence number is less than the received SYNC MAC
> sequence number, it MUST be increased to be equal to the received SYNC MAC
> sequence number.
>
> * If a local MAC sequence number is updated as a result of the above, all
> local MAC-IPs associated with MAC Mx MUST inherit the updated sequence
> number.
>
> 6.6. Interoperability
> Generally, if all PE nodes in the overlay network follow the above
> sequence number assignment procedures and the PE is advertising both MAC+IP
> and MAC routes, the sequence numbers advertised with the MAC and MAC+IP
> routes with the same MAC would always be the same. However, an
> interoperability scenario with a different implementation could arise,
> where a non-compliant PE implementation assigns and advertises independent
> sequence numbers to MAC and MAC+IP routes. To handle this case, if
> different sequence numbers are received for remote MAC+IP and corresponding
> remote MAC routes from a remote PE, the sequence number associated with the
> remote MAC route MUST be computed as:
>
> * The highest of all received sequence numbers with remote MAC+IP and MAC
> routes with the same MAC.
>
> * The MAC sequence number would be re-computed on a MAC or MAC+IP route
> withdraw as per the above.
>
> A MAC and/or IP move to the local PE would then result in the MAC (and
> hence all MAC-IP) sequence numbers being incremented from the above
> computed remote MAC sequence number.
>
> If MAC-only routes are not advertised at all, and different sequence
> numbers are received with multiple MAC+IP routes for a given MAC, the
> sequence number associated with the derived remote MAC route should still
> be computed as the highest of all received MAC+IP sequence numbers with the
> same MAC.
>
> 6.7. MAC Sharing Race Condition
> *************************************************************
> ****This section i was not able to process and understand****
> *************************************************************
> In a MAC sharing use case described in section 5.2, a race condition
> is possible with simultaneous host moves between a pair of PEs.  As
> an example, consider PE1 with local host IPs I1 and I2 sharing MAC
> M1, and PE2 with local host IPs I3 and I4 sharing MAC M2.  A
> simultaneous move of I1 from PE1 to PE2 and of I3 from PE2 to PE1,
> such that I3 is learnt on PE1 before I1's local entry has been probed
> out on PE1 and/or I1 is learnt on PE2 before I3's local entry has
> been probed out on PE2 may trigger a race condition.  This race
> condition together with MAC sequence number assignment rules defined
> in section 6.1 can cause new mac-ip routes I1-M2 and I3-M1 to bounce
> a couple of times with an incremented sequence number until stale
> entries I1-M1 and I3-M2 have been probed out from PE1 and PE2
> respectively.  An implementation MUST ensure proper probing
> procedures to remove stale ARP, ND, and local MAC entries, following
> a move, on learning remote routes as defined in section 6.3 (and as
> per [RFC9135]) to minimize exposure to this race condition.
>
> 6.8. Mobility Convergence
> This section is optional and details ARP and ND probing procedures that
> MAY be implemented to achieve faster host re-learning and convergence on
> mobility events.
>
> * Following a host move from PE1 to PE2, the host's MAC is discovered at
> PE2 as a local MAC via data frames received from the host. If PE2 has a
> prior remote MAC-IP host route for this MAC from PE1, an ARP/ND probe MAY
> be triggered at PE2 to learn the MAC-IP as a local adjacency and trigger
> EVPN RT-2 advertisement for this MAC-IP across the overlay with new
> reachability via PE2. This results in a reliable "event-based" host IP
> learning triggered by a "MAC learning event" across the overlay, and hence
> faster convergence of overlay routed flows to the host.
>
> * Following a host move from PE1 to PE2, once PE1 receives a MAC or MAC-IP
> route from PE2 with a higher sequence number, an ARP/ND probe MAY be
> triggered at PE1 to clear the stale local MAC-IP neighbor adjacency or to
> re-learn the local MAC-IP in case the host has moved back or is duplicated.
>
> * Following a local MAC age-out, if there is a local IP adjacency with
> this MAC, an ARP/ND probe MAY be triggered for this IP to either re-learn
> the local MAC and maintain local L3 and L2 reachability to this host or to
> clear the ARP/ND entry if the host is no longer local. This accomplishes
> the clearance of stale ARP entries triggered by a MAC age-out event even
> when the ARP refresh timer is longer than the MAC age-out timer. Clearing
> stale IP neighbor entries facilitates traffic convergence if the host was
> silent and not discovered at its new location. Once the stale neighbor
> entry for the host is cleared, routed traffic flow destined for the host
> can re-trigger ARP/ND discovery for this host at the new location.
>
> 6.8.1. Generalized Probing Logic
> The above probing logic may be generalized as probing for an IP neighbor
> anytime a resolving parent MAC route is inconsistent with the MAC-IP
> neighbor route, where inconsistency is defined as being not present or
> conflicting in terms of the route source being local or remote. The MAC-IP
> to MAC parent relationship described in section 5.1 MAY be used to achieve
> this logic.
>
> 863     7.  Routed Overlay
> 864
> 865        An additional use case is possible, such that traffic to an end
> host
> 866        in the overlay is always IP routed.  In a purely routed overlay
> such
> 867        as this:
> 868
> 869        *  A host MAC is never advertised in the EVPN overlay control
> plane.
> 870
> 871        *  Host /32 or /128 IP reachability is distributed across the
> overlay
> 872           via EVPN route type 5 (RT-5) along with a zero or non- zero
> ESI.
> 873
> 874        *  An overlay IP subnet may still be stretched across the
> underlay
> 875           fabric, however, intra-subnet traffic across the stretched
> overlay
> 876           is never bridged.
> 877
> 878        *  Both inter-subnet and intra-subnet traffic, in the overlay
> is IP
> 879           routed at the EVPN PE.
> 880
> 881        Please refer to [RFC7814] for more details.
> 882
> 883        Host mobility within the stretched subnet would still need to be
> 884        supported for this use.  In the absence of any host MAC routes,
> 885        sequence number mobility Extended Community specified in
> [RFC7432],
> 886        section 7.7 may be associated with a /32 OR /128 host IP prefix
> 887        advertised via EVPN route type 5.  MAC mobility procedures
> defined in
> 888        [RFC7432] can now be applied as is to host IP prefixes:
> 889
> 890        *  On local learning of a host IP, on a new ESI, the host IP
> MUST be
> 891           advertised with a sequence number attribute that is higher
> than
> 892           what is currently advertised with the old ESI.
> 893
> 894        *  On receiving a host IP route advertisement with a higher
> sequence
> 895           number, a PE MUST trigger ARP/ND probe and deletion
> procedures on
> 896           any local route for that IP with a lower sequence number.  A
> PE
> 897           would essentially move the forwarding entry to point to the
> remote
> 898           route with a higher sequence number and send an ARP/ND PROBE
> for
> 899           the local IP route.  If the IP has indeed moved, PROBE would
> 900           timeout and the local IP host route would be deleted.
> 901
> 902        Note that there is still only one sequence number associated
> with a
> 903        host route at any time.  For earlier use cases where a host MAC
> is
> 904        advertised along with the host IP, a sequence number is only
> 905        associated with a MAC.  Only if the MAC is not advertised at
> all, as
> 906        in this use case, is a sequence number associated with a host
> IP.
> 907
> 908        Note that this mobility procedure would not apply to "anycast
> IPv6"
> 909        hosts advertised via NA messages with 0-bit=0.  Please refer to
> 910        [RFC9161].
>
> [major]
> * Unsure what purpose of 0-bit=0 is and where it is explained in RFC9161.
> Some explicit reference and explanation could help the draft
>
> [re-edit]
> 7. Routed Overlay
> An additional use case involves traffic to an end host in the overlay
> being entirely IP routed. In such a purely routed overlay:
>
> * A host MAC is never advertised in the EVPN overlay control plane.
>
> * Host /32 or /128 IP reachability is distributed across the overlay via
> EVPN Route Type 5 (RT-5) along with a zero or non-zero ESI.
>
> * An overlay IP subnet may still be stretched across the underlay fabric;
> however, intra-subnet traffic across the stretched overlay is never bridged.
>
> * Both inter-subnet and intra-subnet traffic in the overlay is IP routed
> at the EVPN PE.
>
> Refer to [RFC7814] for more details.
>
> Host mobility within the stretched subnet still needs support. In the
> absence of host MAC routes, the sequence number mobility Extended Community
> specified in [RFC7432], section 7.7, MAY be associated with a /32 or /128
> host IP prefix advertised via EVPN Route Type 5. MAC mobility procedures
> defined in [RFC7432] can be applied to host IP prefixes as follows:
>
> * On local learning of a host IP on a new ESI, the host IP MUST be
> advertised with a sequence number higher than what is currently advertised
> with the old ESI.
>
> * On receiving a host IP route advertisement with a higher sequence
> number, a PE MUST trigger ARP/ND probe and deletion procedures on any local
> route for that IP with a lower sequence number. The PE will update the
> forwarding entry to point to the remote route with a higher sequence number
> and send an ARP/ND probe for the local IP route. If the IP has moved, the
> probe will time out, and the local IP host route will be deleted.
>
> Note that there is only one sequence number associated with a host route
> at any time. For previous use cases where a host MAC is advertised along
> with the host IP, a sequence number is only associated with the MAC. If the
> MAC is not advertised, as in this use case, a sequence number is associated
> with the host IP.
>
> This mobility procedure does not apply to "anycast IPv6" hosts advertised
> via NA messages with 0-bit=0. Refer to [RFC9161] for more details.
>
> 912     8.  Duplicate Host Detection
> 913
> 914        Duplicate host detection scenarios across EVPN IRB can be
> classified
> 915        as follows:
> 916
> 917        *  Scenario A: where two hosts have the same MAC (host IPs may
> or may
> 918           not be duplicate).
> 919
> 920        *  Scenario B: where two hosts have the same IP but different
> MACs.
> 920
> 922        *  Scenario C: where two hosts have the same IP and host MAC is
> not
> 923           advertised at all.
> 924
> 925        Duplicate detection procedures for scenario B and C would not
> apply
> 926        to "anycast IPv6" hosts advertised via NA messages with 0-bit=0.
> 927        Please refer to [RFC9161].
> 928
> 929     8.1.  Scenario A
> 920
> 931        For all use cases where duplicate hosts have the same MAC, the
> MAC is
> 932        detected as duplicate via the duplicate MAC detection procedure
> 933        described in [RFC7432].  Corresponding MAC-IP routes with the
> same
> 934        MAC do not require duplicate detection and MUST simply inherit
> the
> 935        duplicate property from the corresponding MAC route.  In other
> words,
> 936        if a MAC route is in duplicate state, all corresponding MAC-IP
> routes
> 937        MUST also be treated as duplicate.  Duplicate detection
> procedure
> 938        need only be applied to MAC routes.
> 939
> 940     8.2.  Scenario B
> 941
> 942        Due to misconfiguration, a situation may arise where hosts with
> 943        different MACs are configured with the same IP.  This scenario
> would
> 944        not be detected by [RFC7432] duplicate MAC detection procedures
> and
> 945        would result in incorrect forwarding of routed traffic destined
> to
> 946        this IP.
> 947
> 948        Such a situation, on local MAC-IP learning, would be detected
> as a
> 949        move scenario via the following local MAC sequence number
> computation
> 950        procedure described earlier in section 6.1:
> 951
> 952        *  If the IP is also associated with a different remote MAC
> "Mz",
> 953           MUST be higher than "Mz" sequence number.
> 954
> 955        Such a move that results in sequence number increment on local
> MAC
> 956        because of a remote MAC-IP route associated with a different
> MAC MUST
> 957        be counted as an "IP move" against the "IP" independent of the
> MAC.
> 958        Duplicate detection procedure described in [RFC7432] can now be
> 959        applied to an "IP" entity independent of MAC.  Once an IP is
> detected
> 960        as duplicate, corresponding MAC-IP route should be treated as
> 961        duplicate.  Associated MAC routes and any other MAC-IP routes
> 962        associated with this MAC should not be affected.
> 963
> 964     8.2.1.  Duplicate IP Detection Procedure for Scenario B
> 065
> 966        The duplicate IP detection procedure for such a scenario are
> 967        specified in [RFC9161].  What counts as an "IP move" in this
> scenario
> 968        is further clarified as follows:
> 969
> 970        *  On learning a local MAC-IP route Mx-IPx, check if there is an
> 971           existing remote or local route for IPx with a different MAC
> 972           association, say, Mz-IPx.  If so, count this as an "IP move"
> count
> 973           for IPx, independent of the MAC.
> 974
> 975        *  On learning a remote MAC-IP route Mz-IPx, check if there is
> an
> 976           existing local route for IPx with a different MAC
> association,
> 977           say, Mx-IPx.  If so, count this as an "IP move" count for
> IPx,
> 978           independent of the MAC.
> 979
> 980        A MAC-IP route SHOULD be treated as duplicate if either of the
> 981        following two conditions are met:
> 982
> 983        *  The corresponding MAC route is marked as duplicate via
> existing
> 984           duplicate detection procedure.
> 985
> 986        *  The corresponding IP is marked as duplicate via extended
> procedure
> 987           described above.
> 988
> 989     8.3.  Scenario C
> 990
> 991        For a purely routed overlay scenario described in section 7,
> where
> 992        only a host IP is advertised via EVPN RT-5, together with a
> sequence
> 993        number mobility attribute, duplicate MAC detection procedures
> 994        specified in [RFC7432] can be intuitively applied to IP only
> host
> 995        routes for the purpose of duplicate IP detection.
> 996
> 997        *  On learning a local host IP route IPx, check if there is an
> 998           existing remote or local route for IPx with a different ESI
> 999           association.  If so, count this as an "IP move" count for
> IPx.
> 1000
> 1001       *  On learning a remote host IP route IPx, check if there is an
> 1002          existing local route for IPx with a different ESI
> association.  If
> 1003          so, count this as an "IP move" count for IPx.
> 1004
> 1005       *  With configurable parameters "N" and "M", if "N" IP moves are
> 1006          detected within "M" seconds for IPx, treat IPx as duplicate.
> 1007
> 1008    8.4.  Duplicate Host Recovery
> 1009
> 1010       Once a MAC or IP is marked as duplicate and frozen, corrective
> action
> 1011       must be taken to un-provision one of the duplicate MAC or IP.
> Un-
> 1012       provisioning a duplicate MAC or IP in this context refers to a
> 1013       corrective action taken on the host side.  Once one of the
> duplicate
> 1014       MAC or IP is un-provisioned, normal operation would not resume
> until
> 1015       the duplicate MAC or IP ages out, following this correction,
> unless
> 1016       additional action is taken to speed up recovery.
> 1017
> 1018       This section lists possible additional corrective actions that
> could
> 1019       be taken to achieve faster recovery to normal operation.
> 1020
> 1021    8.4.1.  Route Un-freezing Configuration
> 1022
> 1023       Unfreezing the duplicate or frozen MAC or IP via a CLI can be
> used to
> 1024       recover from duplicate and frozen state following corrective un-
> 1025       provisioning of the duplicate MAC or IP.
> 1026
> 1027       Unfreezing the frozen MAC or IP via a CLI at a PE should result
> in
> 1028       that MAC or IP being advertised with a sequence number that is
> higher
> 1029       than the sequence number advertised from the other location of
> that
> 1030       MAC or IP.
> 1031
> 1032       Two possible corrective un-provisioning scenarios exist:
> 1033
> 1034       *  Scenario A: A duplicate MAC or IP may have been
> un-provisioned at
> 1035          the location where it was NOT marked as duplicate and frozen.
> 1036
> 1037       *  Scenario B: A duplicate MAC or IP may have been
> un-provisioned at
> 1038          the location where it was marked as duplicate and frozen.
> 1039
> 1040       Unfreezing the duplicate and frozen MAC or IP, following the
> above
> 1041       corrective un-provisioning scenarios would result in recovery to
> 1042       steady state as follows:
> 1043
> 1044       *  Scenario A: If the duplicate MAC or IP was un-provisioned at
> the
> 1045          location where it was NOT marked as duplicate, unfreezing the
> 1046          route at the frozen location will result in the route being
> 1047          advertised with a higher sequence number.  This would in-turn
> 1048          result in automatic clearing of local route at the PE
> location,
> 1049          where the host was un-provisioned via ARP/ND PROBE and DELETE
> 1050          procedure specified earlier in section 6 and in [RFC7432].
> 1051
> 1052       *  Scenario B: If the duplicate host is un-provisioned at the
> 1053          location where it was marked as duplicate, unfreezing the
> route
> 1054          will trigger an advertisement with a higher sequence number
> to the
> 1055          other location.  This would in-turn trigger re-learning of
> local
> 1056          route at the remote location, resulting in another
> advertisement
> 1057          with a higher sequence number from the remote location.
> Route at
> 1058          the local location would now be cleared on receiving this
> remote
> 1059          route advertisement, following the ARP/ND PROBE.
> 1060
> 1061       Note that the probes referred to in the above scenarios are
> event
> 1062       driven probes resulting from receiving a route with a higher
> sequence
> 1063       number.  Periodic probes resulting from refresh timers may also
> occur
> 1064       in addition as completely independent probes.
> 1065
> 1066    8.4.2.  Route Clearing Configuration
> 1067
> 1068       In addition to the above, route clearing CLIs may also be used
> to
> 1069       clear the local MAC or IP route, to be executed AFTER the
> duplicate
> 1070       host is un-provisioned:
> 1071
> 1072       *  clear MAC CLI: A clear MAC CLI can be used to clear a
> duplicate
> 1073          MAC route, to recover from a duplicate MAC scenario.
> 1074
> 1075       *  clear ARP/ND: A clear ARP/ND CLI may be used to clear a
> duplicate
> 1076          IP route to recover from a duplicate IP scenario.
> 1077
> 1078       Note that the route unfreeze CLI may still need to be run if the
> 1079       route was un-provisioned and cleared from the non-duplicate /
> non-
> 1080       frozen location.  Given that unfreezing of the route via the un-
> 1081       freeze CLI would any ways result in auto-clearing of the route
> from
> 1082       the "un- provisioned" location, as explained in the prior
> section,
> 1083       need for a route clearing CLI for recovery from duplicate /
> frozen
> 1084       state is truly optional.
>
> [major]
> * what is the 0-bit=0? please add a specific reference
>
> [re-edit]
> 8. Duplicate Host Detection
>
> Duplicate host detection scenarios across EVPN IRB can be classified as
> follows:
>
> * Scenario A: Two hosts have the same MAC address (host IPs may or may not
> be duplicates).
> * Scenario B: Two hosts have the same IP address but different MAC
> addresses.
> * Scenario C: Two hosts have the same IP address, and the host MAC is not
> advertised.
>
> Duplicate detection procedures for Scenarios B and C do not apply to
> "anycast IPv6" hosts advertised via NA messages with 0-bit=0, as per
> [RFC9161].
>
> 8.1. Scenario A
>
> In cases where duplicate hosts share the same MAC address, the MAC is
> detected as duplicate using the duplicate MAC detection procedure described
> in [RFC7432]. Corresponding MAC-IP routes with the same MAC do not require
> separate duplicate detection and MUST inherit the duplicate property from
> the MAC route. If a MAC route is marked as duplicate, all associated MAC-IP
> routes MUST also be treated as duplicates. Duplicate detection procedures
> need only be applied to MAC routes.
>
> 8.2. Scenario B
>
> Misconfigurations may lead to different MAC addresses being assigned the
> same IP address. This scenario is not detected by [RFC7432] duplicate MAC
> detection procedures and can result in incorrect routing of traffic
> destined for the IP address.
>
> Such situations, when detected locally, are identified as a move scenario
> through the local MAC sequence number computation procedure described in
> section 6.1:
>
> * If the IP is associated with a different remote MAC "Mz," the sequence
> number MUST be higher than the "Mz" sequence number.
>
> This move results in a sequence number increment for the local MAC due to
> the remote MAC-IP route associated with a different MAC, counting as an "IP
> move" against the IP, independent of the MAC. The duplicate detection
> procedure described in [RFC7432] can then be applied to the IP entity
> independent of the MAC. Once an IP is detected as duplicate, the
> corresponding MAC-IP route should be treated as duplicate. Associated MAC
> routes and any other MAC-IP routes related to this MAC should not be
> affected.
>
> 8.2.1. Duplicate IP Detection Procedure for Scenario B
>
> The duplicate IP detection procedure for this scenario is specified in
> [RFC9161]. An "IP move" is further clarified as follows:
>
> * Upon learning a local MAC-IP route Mx-IPx, check for existing remote or
> local routes for IPx with a different MAC association (Mz-IPx). If found,
> count this as an "IP move" for IPx, independent of the MAC.
>
> * Upon learning a remote MAC-IP route Mz-IPx, check for existing local
> routes for IPx with a different MAC association (Mx-IPx). If found, count
> this as an "IP move" for IPx, independent of the MAC.
>
> A MAC-IP route SHOULD be treated as duplicate if either:
>
> * The corresponding MAC route is marked as duplicate via the existing
> detection procedure.
>
> * The corresponding IP is marked as duplicate via the extended procedure
> described above.
>
> 8.3. Scenario C
>
> In a purely routed overlay scenario, as described in section 7, where only
> a host IP is advertised via EVPN RT-5 with a sequence number mobility
> attribute, duplicate MAC detection procedures specified in [RFC7432] can be
> applied intuitively to IP-only host routes for duplicate IP detection.
>
> * Upon learning a local host IP route IPx, check for existing remote or
> local routes for IPx with a different ESI association. If found, count this
> as an "IP move" for IPx.
>
> * Upon learning a remote host IP route IPx, check for existing local
> routes for IPx with a different ESI association. If found, count this as an
> "IP move" for IPx.
>
> * Using configurable parameters "N" and "M," if "N" IP moves are detected
> within "M" seconds for IPx, IPx should be treated as duplicate.
>
> 8.4. Duplicate Host Recovery
>
> Once a MAC or IP is marked as duplicate and frozen, corrective action must
> be taken to un-provision one of the duplicate MAC or IP addresses.
> Un-provisioning refers to corrective action taken on the host side.
> Following this correction, normal operation will not resume until the
> duplicate MAC or IP ages out unless additional action is taken to expedite
> recovery.
>
> Possible additional corrective actions for faster recovery include:
>
> 8.4.1. Route Unfreezing Configuration
>
> Unfreezing the duplicate or frozen MAC or IP via a CLI can be used to
> recover from the duplicate and frozen state following corrective
> un-provisioning of the duplicate MAC or IP. Unfreezing the MAC or IP should
> result in advertising it with a sequence number higher than that advertised
> from the other location.
>
> Two scenarios exist:
>
> * Scenario A: The duplicate MAC or IP is un-provisioned at the location
> where it was not marked as duplicate.
>
> * Scenario B: The duplicate MAC or IP is un-provisioned at the location
> where it was marked as duplicate.
>
> Unfreezing the duplicate and frozen MAC or IP will result in recovery to a
> steady state as follows:
>
> * Scenario A: If the duplicate MAC or IP is un-provisioned at the
> non-duplicate location, unfreezing the route at the frozen location results
> in advertising with a higher sequence number, leading to automatic clearing
> of the local route at the un-provisioned location via ARP/ND PROBE and
> DELETE procedures.
>
> * Scenario B: If the duplicate host is un-provisioned at the duplicate
> location, unfreezing the route triggers an advertisement with a higher
> sequence number to the other location, prompting re-learning and clearing
> of the local route at the original location upon receiving the remote route
> advertisement.
>
> Probes referred to in these scenarios are event-driven probes resulting
> from receiving a route with a higher sequence number. Periodic probes
> resulting from refresh timers may also occur independently.
>
> 8.4.2. Route Clearing Configuration
>
> In addition to the above, route clearing CLIs may be used to clear the
> local MAC or IP route after the duplicate host is un-provisioned:
>
> * Clear MAC CLI: Used to clear a duplicate MAC route.
>
> * Clear ARP/ND: Used to clear a duplicate IP route.
>
> The route unfreeze CLI may still need to be executed if the route was
> un-provisioned and cleared from the non-duplicate location. Given that
> unfreezing the route via the CLI would result in auto-clearing from the
> un-provisioned location, as explained earlier, using a route clearing CLI
> for recovery from the duplicate state is optional.
>
> Kind Regards,
> Gunter Van de Velde
> Routing Area Director
>
> _______________________________________________
> BESS mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
>