Re: BFD over VXLAN: Trapping BFD Control packet at VTEP

["Joel M. Halpern" <[e-mail]>]

I am going by what the draft says its purpose is.  If you (Dinesh) want 
the draft to fulfill a different purpose, then either ask the chairs to 
take this draft back to the WG, or write a separate draft.
As currently written, the behavior Greg proposed meets the needs, and 
does so in a way that is consistent with VxLAN.

Yours,
Joel

On 8/2/2019 8:30 PM, Dinesh Dutt wrote:
> What is the stated purpose of this BFD session? The VTEP reachability is 
> determined by the underlay, I don't need VXLAN-encaped packet for that. 
> Do we agree?
> 
> If I want to test the VXLAN encap/decap functionality alone, picking any 
> single VNI maybe fine. But is this all any network operator wants? Why? 
> In what situations has this been a problem? I suspect operators also 
> want to verify path continuity over a specific VNI. If you say this is 
> not defined by the document, I disagree because the current version 
> talks about controlling the number of BFD sessions between the VTEPs 
> (see section 3). More importantly, this is a real problem that operators 
> like to verify.
> 
> Dinesh
> 
> On Fri, Aug 2, 2019 at 5:08 PM Joel M. Halpern <[e-mail] 
> <mailto:[e-mail]>> wrote:
> 
>     What is special about the management VNI is precisely that it is NOT a
>     tenant VNI.  The VxLAN administration does know how it allocates VNI to
>     tenants, and which VNI it has allocated.  In contrast, it does not know
>     which IP addresses or MAC adddresses teh tenant is using or may plan
>     to use.
> 
>     Yours,
>     Joel
> 
>     On 8/2/2019 6:41 PM, Dinesh Dutt wrote:
>      > The assumption of an IP address within any VNI is suspect that way.
>      > What's special about a single VNI, the management VNI? The VTEP IP
>      > address does not belong in reality in any VNI.
>      >
>      > Dinesh
>      >
>      > On Fri, Aug 2, 2019 at 3:17 PM Joel M. Halpern
>     <[e-mail] <mailto:[e-mail]>
>      > <mailto:[e-mail] <mailto:[e-mail]>>> wrote:
>      >
>      >     Your response seems to miss two points:
>      >
>      >     First, the problem you describe is not what the document says
>     it is
>      >     solving.  To the degree it discusses it at all, the document
>     says "
>      >       In
>      >     most cases, a single BFD session is sufficient for the given
>     VTEP to
>      >     monitor the reachability of a remote VTEP, regardless of the
>     number of
>      >     VNIs in common. "
>      >
>      >     Second, you assume the existence of an IP address for a VTEP
>     within a
>      >     VNI.  As with the MAC address, the VTEP does not have an IP
>     address
>      >     within the VNI.  Some implementations may have created such a
>     thing,
>      >     but
>      >     the general construct, as defined to date, does not support such.
>      >
>      >     In short, you are requiring a behavior that violates the
>     architectural
>      >     structure of overlay / underlay separation, and common
>     usage.  And you
>      >     are doing so to support a use case that the working group has not
>      >     indicated in the document as important.
>      >
>      >     Yours,
>      >     Joel
>      >
>      >     On 8/2/2019 5:01 PM, Dinesh Dutt wrote:
>      >      > Joel,
>      >      >
>      >      > You understood correctly.
>      >      >
>      >      > The VNIs may not share fate due to misconfiguration. And I
>     strongly
>      >      > suspect someone will want to use BFD for that because its
>     about
>      >     checking
>      >      > path continuity as stated by the draft. As long as there's a
>      >     valid IP
>      >      > (because it's BFD) owned by the VTEP in that VNI, you can
>     use BFD in
>      >      > that VNI. Thats all that you need to dictate.  That IP address
>      >     has a MAC
>      >      > address and you can use that on the inner frame. That is
>     all normal
>      >      > VXLAN processing. The outer IP is always that of the VTEP.
>      >      >
>      >      > Dinesh
>      >      >
>      >      > On Fri, Aug 2, 2019 at 11:03 AM Joel M. Halpern
>      >     <[e-mail] <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]>>
>      >      > <mailto:[e-mail] <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]>>>> wrote:
>      >      >
>      >      >     If I am reading your various emails correctly Dinesh
>     (and I
>      >     may have
>      >      >     missed something) you are trying to use the MAC address
>      >     because you
>      >      >     want
>      >      >     to be able to send these BFD packets over arbitrary VNI to
>      >     monitor the
>      >      >     VNI.  That is not a requirement identified in the
>     document.
>      >     It is not
>      >      >     even a problem I understand, since all the VNI between an
>      >     ingress and
>      >      >     egress VTEP share fate.
>      >      >
>      >      >     Yours,
>      >      >     Joel
>      >      >
>      >      >     On 8/2/2019 1:44 PM, Dinesh Dutt wrote:
>      >      >      > Thanks for verifying this. On Linux and hardware
>     routers
>      >     that I'm
>      >      >     aware
>      >      >      > of (Cisco circa 2012 and Cumulus), the physical MAC
>     address is
>      >      >     reused
>      >      >      > across the VNIs on the VTEP. Did you check on a non-VMW
>      >     device?
>      >      >     This is
>      >      >      > more for my own curiosity.
>      >      >      >
>      >      >      > To address the general case, can we not define a
>      >     well-known (or
>      >      >     reserve
>      >      >      > one) unicast MAC address for use with VTEP? If the MAC
>      >     address is
>      >      >      > configurable in BFD command, this can be moot.
>      >      >      >
>      >      >      > Dinesh
>      >      >      >
>      >      >      > On Fri, Aug 2, 2019 at 10:27 AM Santosh P K
>      >      >      > <[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail]
>     <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail]
>     <mailto:[e-mail]>>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail]
>     <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail]
>     <mailto:[e-mail]>>>>> wrote:
>      >      >      >
>      >      >      >     I have cross checked point raised about MAC address
>      >     usage. It is
>      >      >      >     possible that tenant could be using physical MAC
>      >     address and
>      >      >     when a
>      >      >      >     packet comes with valid VNI with a MAC address
>     that is
>      >     being
>      >      >     used by
>      >      >      >     tenant then packet will be sent to that tenant.
>     This rules
>      >      >     out the
>      >      >      >     fact that we could use physical MAC address as
>     inner
>      >     MAC to
>      >      >     ensure
>      >      >      >     packets get terminated at VTEP itself.
>      >      >      >
>      >      >      >     Thanks
>      >      >      >     Santosh P K
>      >      >      >
>      >      >      >     On Wed, Jul 31, 2019 at 11:00 AM Santosh P K
>      >      >      >     <[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail]
>     <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail]
>     <mailto:[e-mail]>>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail]
>     <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail]
>     <mailto:[e-mail]>>>>>
>      >      >      >     wrote:
>      >      >      >
>      >      >      >         Joel,
>      >      >      >             Thanks for your inputs. I checked
>      >     implementation within
>      >      >      >         Vmware. Perhaps I should have been more clear
>      >     about MAC
>      >      >     address
>      >      >      >         space while checking internally. I will cross
>      >     check again for
>      >      >      >         the same and get back on this list.
>      >      >      >
>      >      >      >         Thanks
>      >      >      >         Santosh P K
>      >      >      >
>      >      >      >         On Wed, Jul 31, 2019 at 10:54 AM Joel M.
>     Halpern
>      >      >      >         <[e-mail]
>     <mailto:[e-mail]> <mailto:[e-mail]
>     <mailto:[e-mail]>>
>      >     <mailto:[e-mail] <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]>>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]> <mailto:[e-mail]
>     <mailto:[e-mail]>>
>      >     <mailto:[e-mail] <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]>>>>> wrote:
>      >      >      >
>      >      >      >             Sorry to ask a stupid question.  Whose
>      >     implementation?
>      >      >      >
>      >      >      >             The reason I ask is that as far as I
>     can tell,
>      >     since the
>      >      >      >             tenant does not
>      >      >      >             have any control access to the VTEP,
>     there is no
>      >      >     reason for
>      >      >      >             the VTEP to
>      >      >      >             have a MAC address in the tenant
>     space.  Yes, the
>      >      >     device has
>      >      >      >             a physical
>      >      >      >             MAC address.  But the tenant could well be
>      >     using that MAC
>      >      >      >             address.  Yes,
>      >      >      >             they would be violating the Ethernet spec.
>      >     But the whole
>      >      >      >             point of
>      >      >      >             segregation is not to care about such
>     issues.
>      >      >      >
>      >      >      >             On the other hand, if you tell me that
>     the VMWare
>      >      >      >             implementation has an
>      >      >      >             Ethernet address that is part of the tenant
>      >     space, well,
>      >      >      >             they made up
>      >      >      >             this particular game.
>      >      >      >
>      >      >      >             Yours,
>      >      >      >             Joel
>      >      >      >
>      >      >      >             On 7/31/2019 1:44 PM, Santosh P K wrote:
>      >      >      >              > I have checked with implementation
>     in data
>      >     path.
>      >      >     When we
>      >      >      >             receive a
>      >      >      >              > packet with valid VNI then lookup
>     for MAC will
>      >      >     happen and
>      >      >      >             it is VTEP own
>      >      >      >              > MAC then it will be trapped to control
>      >     plane for
>      >      >      >             processing. I think we
>      >      >      >              > can have following options
>      >      >      >              > 1. Optional managment VNI
>      >      >      >              > 2. Mandatory inner MAC set to VTEP mac
>      >      >      >              > 3. Inner IP TTL set to 1 to avoid
>      >     forwarding of packet
>      >      >      >             via inner IP
>      >      >      >              > address.
>      >      >      >              >
>      >      >      >              >
>      >      >      >              > Thoughts?
>      >      >      >              >
>      >      >      >              > Thansk
>      >      >      >              > Santosh P K
>      >      >      >              >
>      >      >      >              > On Wed, Jul 31, 2019 at 9:20 AM Greg
>     Mirsky
>      >      >      >             <[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail] <mailto:[e-mail]>>
>     <mailto:[e-mail] <mailto:[e-mail]>
>      >     <mailto:[e-mail] <mailto:[e-mail]>>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]> <mailto:[e-mail]
>     <mailto:[e-mail]>>
>      >     <mailto:[e-mail] <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]>>>>
>      >      >      >              > <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail] <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]> <mailto:[e-mail]
>     <mailto:[e-mail]>>>
>      >      >      >             <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail] <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail]
>     <mailto:[e-mail]>>>>>> wrote:
>      >      >      >              >
>      >      >      >              >     Hi Dinesh,
>      >      >      >              >     thank you for your consideration
>     of the
>      >      >     proposal and
>      >      >      >             questions. What
>      >      >      >              >     would you see as the scope of
>     testing the
>      >      >      >             connectivity for the
>      >      >      >              >     specific VNI? If it is
>      >     tenant-to-tenant, then
>      >      >     VTEPs
>      >      >      >             will treat these
>      >      >      >              >     packets as regular user frames. More
>      >     likely, these
>      >      >      >             could be Layer 2
>      >      >      >              >     OAM, e.g. CCM frames. The reason
>     to use
>      >     127/8 for
>      >      >      >             IPv4, and
>      >      >      >              >     0:0:0:0:0:FFFF:7F00:0/104 for
>     IPv6 is
>      >     to safeguard
>      >      >      >             from leaking
>      >      >      >              >     Ethernet frames with BFD Control
>     packet
>      >     to a
>      >      >     tenant.
>      >      >      >              >     You've suggested using a MAC
>     address to
>      >     trap the
>      >      >      >             control packet at
>      >      >      >              >     VTEP. What that address could be? We
>      >     had proposed
>      >      >      >             using the
>      >      >      >              >     dedicated MAC and VTEP's MAC and
>     both
>      >     raised
>      >      >     concerns
>      >      >      >             among VXLAN
>      >      >      >              >     experts. The idea of using
>     Management
>      >     VNI may
>      >      >     be more
>      >      >      >             acceptable
>      >      >      >              >     based on its similarity to the
>     practice
>      >     of using
>      >      >      >             Management VLAN.
>      >      >      >              >
>      >      >      >              >     Regards,
>      >      >      >              >     Greg
>      >      >      >              >
>      >      >      >              >     On Wed, Jul 31, 2019 at 12:03 PM
>     Dinesh
>      >     Dutt
>      >      >      >             <[e-mail]
>     <mailto:[e-mail]> <mailto:[e-mail]
>     <mailto:[e-mail]>>
>      >     <mailto:[e-mail] <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]>>>
>      >      >     <mailto:[e-mail] <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]>>
>      >     <mailto:[e-mail] <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]>>>>
>      >      >      >              >     <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail] <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail] <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]>>>
>      >     <mailto:[e-mail] <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail] <mailto:[e-mail]>
>     <mailto:[e-mail] <mailto:[e-mail]>>>>>>
>      >      >      >             wrote:
>      >      >      >              >
>      >      >      >              >         Hi Greg,
>      >      >      >              >
>      >      >      >              >         As long as the inner MAC
>     address is
>      >     such
>      >      >     that the
>      >      >      >             packet is
>      >      >      >              >         trapped to the CPU, it should be
>      >     fine for
>      >      >     use as
>      >      >      >             an inner MAC is
>      >      >      >              >         it not? Stating that is
>     better than
>      >     trying to
>      >      >      >             force a management
>      >      >      >              >         VNI. What if someone wants
>     to test
>      >      >     connectivity
>      >      >      >             on a specific
>      >      >      >              >         VNI? I would not pick a
>     loopback IP
>      >      >     address for
>      >      >      >             this since that
>      >      >      >              >         address range is host/node local
>      >     only. Is
>      >      >     there a
>      >      >      >             reason you're
>      >      >      >              >         not using the VTEP IP as the
>     inner IP
>      >      >     address ?
>      >      >      >              >
>      >      >      >              >         Dinesh
>      >      >      >              >
>      >      >      >              >         On Wed, Jul 31, 2019 at 5:48 AM
>      >     Greg Mirsky
>      >      >      >              >         <[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail] <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]> <mailto:[e-mail]
>     <mailto:[e-mail]>>>
>      >      >      >             <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail] <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail]
>     <mailto:[e-mail]>>>> <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail] <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]> <mailto:[e-mail]
>     <mailto:[e-mail]>>>
>      >      >      >             <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail] <mailto:[e-mail]>>
>      >      >     <mailto:[e-mail]
>     <mailto:[e-mail]>
>      >     <mailto:[e-mail]
>     <mailto:[e-mail]>>>>>> wrote:
>      >      >      >              >
>      >      >      >              >             Dear All,
>      >      >      >              >             thank you for your comments,
>      >      >     suggestions on
>      >      >      >             this issue,
>      >      >      >              >             probably the most
>     challenging
>      >     for this
>      >      >      >             specification. In the
>      >      >      >              >             course of our discussions,
>      >     we've agreed to
>      >      >      >             abandon the
>      >      >      >              >             request to allocate the
>      >     dedicated MAC
>      >      >     address
>      >      >      >             to be used as
>      >      >      >              >             the destination MAC
>     address in
>      >     the inner
>      >      >      >             Ethernet frame.
>      >      >      >              >             Also, earlier using VNI
>     0 was
>      >     changed from
>      >      >      >             mandatory to one
>      >      >      >              >             of the options an
>      >     implementation may
>      >      >     offer to
>      >      >      >             an operator.
>      >      >      >              >             The most recent
>     discussion was
>      >     whether
>      >      >     VTEP's
>      >      >      >             MAC address
>      >      >      >              >             might be used as the
>      >     destination MAC
>      >      >     address
>      >      >      >             in the inner
>      >      >      >              >             Ethernet frame. As I
>     recall it, the
>      >      >     comments
>      >      >      >             from VXLAN
>      >      >      >              >             experts equally split
>     with one
>      >     for it
>      >      >     and one
>      >      >      >             against. Hence
>      >      >      >              >             I would like to propose
>     a new
>      >     text to
>      >      >     resolve
>      >      >      >             the issue. The
>      >      >      >              >             idea is to let an
>     operator select
>      >      >     Management
>      >      >      >             VNI and use
>      >      >      >              >             that VNI in VXLAN
>     encapsulation
>      >     of BFD
>      >      >      >             Control packets:
>      >      >      >              >             NEW TEXT:
>      >      >      >              >
>      >      >      >              >                 An operator MUST
>     select a VNI
>      >      >     number to
>      >      >      >             be used as
>      >      >      >              >                 Management VNI. VXLAN
>      >     packet for
>      >      >      >             Management VNI MUST NOT
>      >      >      >              >                 be sent to a tenant. VNI
>      >     number 1 is
>      >      >      >             RECOMMENDED as the
>      >      >      >              >                 default for
>     Management VNI.
>      >      >      >              >
>      >      >      >              >             With that new text, what
>     can be the
>      >      >     value of
>      >      >      >             the destination
>      >      >      >              >             MAC in the inner Ethernet? I
>      >     tend to
>      >      >     believe
>      >      >      >             that it can be
>      >      >      >              >             anything and ignored by the
>      >     reciever VTEP.
>      >      >      >             Also, if the
>      >      >      >              >             trapping is based on VNI
>      >     number, the
>      >      >      >             destination IP address
>      >      >      >              >             of the inner IP packet
>     can from
>      >     the range
>      >      >      >             127/8 for IPv4,
>      >      >      >              >             and for IPv6 from the range
>      >      >      >             0:0:0:0:0:FFFF:7F00:0/104. And
>      >      >      >              >             lastly, the TTL to be
>     set to 1 (no
>      >      >     change here).
>      >      >      >              >
>      >      >      >              >             Much appreciate your
>     comments,
>      >      >     questions, and
>      >      >      >             suggestions.
>      >      >      >              >
>      >      >      >              >             Best regards,
>      >      >      >              >             Greg
>      >      >      >              >
>      >      >      >
>      >      >
>      >
>