Re: BFD over VXLAN: Trapping BFD Control packet at VTEP

[Dinesh Dutt <[e-mail]>]

The abstract reads this: "

This document describes the use of the Bidirectional Forwarding
   Detection (BFD) protocol in point-to-point Virtual eXtensible Local
   Area Network (VXLAN) tunnels forming up an overlay network."

How do you infer what you said?

Dinesh


On Fri, Aug 2, 2019 at 5:38 PM Joel M. Halpern <[e-mail]> wrote:

> I am going by what the draft says its purpose is.  If you (Dinesh) want
> the draft to fulfill a different purpose, then either ask the chairs to
> take this draft back to the WG, or write a separate draft.
> As currently written, the behavior Greg proposed meets the needs, and
> does so in a way that is consistent with VxLAN.
>
> Yours,
> Joel
>
> On 8/2/2019 8:30 PM, Dinesh Dutt wrote:
> > What is the stated purpose of this BFD session? The VTEP reachability is
> > determined by the underlay, I don't need VXLAN-encaped packet for that.
> > Do we agree?
> >
> > If I want to test the VXLAN encap/decap functionality alone, picking any
> > single VNI maybe fine. But is this all any network operator wants? Why?
> > In what situations has this been a problem? I suspect operators also
> > want to verify path continuity over a specific VNI. If you say this is
> > not defined by the document, I disagree because the current version
> > talks about controlling the number of BFD sessions between the VTEPs
> > (see section 3). More importantly, this is a real problem that operators
> > like to verify.
> >
> > Dinesh
> >
> > On Fri, Aug 2, 2019 at 5:08 PM Joel M. Halpern <[e-mail]
> > <mailto:[e-mail]>> wrote:
> >
> >     What is special about the management VNI is precisely that it is NOT
> a
> >     tenant VNI.  The VxLAN administration does know how it allocates VNI
> to
> >     tenants, and which VNI it has allocated.  In contrast, it does not
> know
> >     which IP addresses or MAC adddresses teh tenant is using or may plan
> >     to use.
> >
> >     Yours,
> >     Joel
> >
> >     On 8/2/2019 6:41 PM, Dinesh Dutt wrote:
> >      > The assumption of an IP address within any VNI is suspect that
> way.
> >      > What's special about a single VNI, the management VNI? The VTEP IP
> >      > address does not belong in reality in any VNI.
> >      >
> >      > Dinesh
> >      >
> >      > On Fri, Aug 2, 2019 at 3:17 PM Joel M. Halpern
> >     <[e-mail] <mailto:[e-mail]>
> >      > <mailto:[e-mail] <mailto:[e-mail]>>> wrote:
> >      >
> >      >     Your response seems to miss two points:
> >      >
> >      >     First, the problem you describe is not what the document says
> >     it is
> >      >     solving.  To the degree it discusses it at all, the document
> >     says "
> >      >       In
> >      >     most cases, a single BFD session is sufficient for the given
> >     VTEP to
> >      >     monitor the reachability of a remote VTEP, regardless of the
> >     number of
> >      >     VNIs in common. "
> >      >
> >      >     Second, you assume the existence of an IP address for a VTEP
> >     within a
> >      >     VNI.  As with the MAC address, the VTEP does not have an IP
> >     address
> >      >     within the VNI.  Some implementations may have created such a
> >     thing,
> >      >     but
> >      >     the general construct, as defined to date, does not support
> such.
> >      >
> >      >     In short, you are requiring a behavior that violates the
> >     architectural
> >      >     structure of overlay / underlay separation, and common
> >     usage.  And you
> >      >     are doing so to support a use case that the working group has
> not
> >      >     indicated in the document as important.
> >      >
> >      >     Yours,
> >      >     Joel
> >      >
> >      >     On 8/2/2019 5:01 PM, Dinesh Dutt wrote:
> >      >      > Joel,
> >      >      >
> >      >      > You understood correctly.
> >      >      >
> >      >      > The VNIs may not share fate due to misconfiguration. And I
> >     strongly
> >      >      > suspect someone will want to use BFD for that because its
> >     about
> >      >     checking
> >      >      > path continuity as stated by the draft. As long as there's
> a
> >      >     valid IP
> >      >      > (because it's BFD) owned by the VTEP in that VNI, you can
> >     use BFD in
> >      >      > that VNI. Thats all that you need to dictate.  That IP
> address
> >      >     has a MAC
> >      >      > address and you can use that on the inner frame. That is
> >     all normal
> >      >      > VXLAN processing. The outer IP is always that of the VTEP.
> >      >      >
> >      >      > Dinesh
> >      >      >
> >      >      > On Fri, Aug 2, 2019 at 11:03 AM Joel M. Halpern
> >      >     <[e-mail] <mailto:[e-mail]>
> >     <mailto:[e-mail] <mailto:[e-mail]>>
> >      >      > <mailto:[e-mail] <mailto:[e-mail]>
> >     <mailto:[e-mail] <mailto:[e-mail]>>>> wrote:
> >      >      >
> >      >      >     If I am reading your various emails correctly Dinesh
> >     (and I
> >      >     may have
> >      >      >     missed something) you are trying to use the MAC address
> >      >     because you
> >      >      >     want
> >      >      >     to be able to send these BFD packets over arbitrary
> VNI to
> >      >     monitor the
> >      >      >     VNI.  That is not a requirement identified in the
> >     document.
> >      >     It is not
> >      >      >     even a problem I understand, since all the VNI between
> an
> >      >     ingress and
> >      >      >     egress VTEP share fate.
> >      >      >
> >      >      >     Yours,
> >      >      >     Joel
> >      >      >
> >      >      >     On 8/2/2019 1:44 PM, Dinesh Dutt wrote:
> >      >      >      > Thanks for verifying this. On Linux and hardware
> >     routers
> >      >     that I'm
> >      >      >     aware
> >      >      >      > of (Cisco circa 2012 and Cumulus), the physical MAC
> >     address is
> >      >      >     reused
> >      >      >      > across the VNIs on the VTEP. Did you check on a
> non-VMW
> >      >     device?
> >      >      >     This is
> >      >      >      > more for my own curiosity.
> >      >      >      >
> >      >      >      > To address the general case, can we not define a
> >      >     well-known (or
> >      >      >     reserve
> >      >      >      > one) unicast MAC address for use with VTEP? If the
> MAC
> >      >     address is
> >      >      >      > configurable in BFD command, this can be moot.
> >      >      >      >
> >      >      >      > Dinesh
> >      >      >      >
> >      >      >      > On Fri, Aug 2, 2019 at 10:27 AM Santosh P K
> >      >      >      > <[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>>>> wrote:
> >      >      >      >
> >      >      >      >     I have cross checked point raised about MAC
> address
> >      >     usage. It is
> >      >      >      >     possible that tenant could be using physical MAC
> >      >     address and
> >      >      >     when a
> >      >      >      >     packet comes with valid VNI with a MAC address
> >     that is
> >      >     being
> >      >      >     used by
> >      >      >      >     tenant then packet will be sent to that tenant.
> >     This rules
> >      >      >     out the
> >      >      >      >     fact that we could use physical MAC address as
> >     inner
> >      >     MAC to
> >      >      >     ensure
> >      >      >      >     packets get terminated at VTEP itself.
> >      >      >      >
> >      >      >      >     Thanks
> >      >      >      >     Santosh P K
> >      >      >      >
> >      >      >      >     On Wed, Jul 31, 2019 at 11:00 AM Santosh P K
> >      >      >      >     <[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>>>>
> >      >      >      >     wrote:
> >      >      >      >
> >      >      >      >         Joel,
> >      >      >      >             Thanks for your inputs. I checked
> >      >     implementation within
> >      >      >      >         Vmware. Perhaps I should have been more
> clear
> >      >     about MAC
> >      >      >     address
> >      >      >      >         space while checking internally. I will
> cross
> >      >     check again for
> >      >      >      >         the same and get back on this list.
> >      >      >      >
> >      >      >      >         Thanks
> >      >      >      >         Santosh P K
> >      >      >      >
> >      >      >      >         On Wed, Jul 31, 2019 at 10:54 AM Joel M.
> >     Halpern
> >      >      >      >         <[e-mail]
> >     <mailto:[e-mail]> <mailto:[e-mail]
> >     <mailto:[e-mail]>>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>
> >     <mailto:[e-mail] <mailto:[e-mail]>>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]> <mailto:[e-mail]
> >     <mailto:[e-mail]>>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>
> >     <mailto:[e-mail] <mailto:[e-mail]>>>>> wrote:
> >      >      >      >
> >      >      >      >             Sorry to ask a stupid question.  Whose
> >      >     implementation?
> >      >      >      >
> >      >      >      >             The reason I ask is that as far as I
> >     can tell,
> >      >     since the
> >      >      >      >             tenant does not
> >      >      >      >             have any control access to the VTEP,
> >     there is no
> >      >      >     reason for
> >      >      >      >             the VTEP to
> >      >      >      >             have a MAC address in the tenant
> >     space.  Yes, the
> >      >      >     device has
> >      >      >      >             a physical
> >      >      >      >             MAC address.  But the tenant could well
> be
> >      >     using that MAC
> >      >      >      >             address.  Yes,
> >      >      >      >             they would be violating the Ethernet
> spec.
> >      >     But the whole
> >      >      >      >             point of
> >      >      >      >             segregation is not to care about such
> >     issues.
> >      >      >      >
> >      >      >      >             On the other hand, if you tell me that
> >     the VMWare
> >      >      >      >             implementation has an
> >      >      >      >             Ethernet address that is part of the
> tenant
> >      >     space, well,
> >      >      >      >             they made up
> >      >      >      >             this particular game.
> >      >      >      >
> >      >      >      >             Yours,
> >      >      >      >             Joel
> >      >      >      >
> >      >      >      >             On 7/31/2019 1:44 PM, Santosh P K wrote:
> >      >      >      >              > I have checked with implementation
> >     in data
> >      >     path.
> >      >      >     When we
> >      >      >      >             receive a
> >      >      >      >              > packet with valid VNI then lookup
> >     for MAC will
> >      >      >     happen and
> >      >      >      >             it is VTEP own
> >      >      >      >              > MAC then it will be trapped to
> control
> >      >     plane for
> >      >      >      >             processing. I think we
> >      >      >      >              > can have following options
> >      >      >      >              > 1. Optional managment VNI
> >      >      >      >              > 2. Mandatory inner MAC set to VTEP
> mac
> >      >      >      >              > 3. Inner IP TTL set to 1 to avoid
> >      >     forwarding of packet
> >      >      >      >             via inner IP
> >      >      >      >              > address.
> >      >      >      >              >
> >      >      >      >              >
> >      >      >      >              > Thoughts?
> >      >      >      >              >
> >      >      >      >              > Thansk
> >      >      >      >              > Santosh P K
> >      >      >      >              >
> >      >      >      >              > On Wed, Jul 31, 2019 at 9:20 AM Greg
> >     Mirsky
> >      >      >      >             <[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>>
> >     <mailto:[e-mail] <mailto:[e-mail]>
> >      >     <mailto:[e-mail] <mailto:[e-mail]
> >>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]> <mailto:[e-mail]
> >     <mailto:[e-mail]>>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>
> >     <mailto:[e-mail] <mailto:[e-mail]>>>>
> >      >      >      >              > <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]> <mailto:[e-mail]
> >     <mailto:[e-mail]>>>
> >      >      >      >             <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>>>>> wrote:
> >      >      >      >              >
> >      >      >      >              >     Hi Dinesh,
> >      >      >      >              >     thank you for your consideration
> >     of the
> >      >      >     proposal and
> >      >      >      >             questions. What
> >      >      >      >              >     would you see as the scope of
> >     testing the
> >      >      >      >             connectivity for the
> >      >      >      >              >     specific VNI? If it is
> >      >     tenant-to-tenant, then
> >      >      >     VTEPs
> >      >      >      >             will treat these
> >      >      >      >              >     packets as regular user frames.
> More
> >      >     likely, these
> >      >      >      >             could be Layer 2
> >      >      >      >              >     OAM, e.g. CCM frames. The reason
> >     to use
> >      >     127/8 for
> >      >      >      >             IPv4, and
> >      >      >      >              >     0:0:0:0:0:FFFF:7F00:0/104 for
> >     IPv6 is
> >      >     to safeguard
> >      >      >      >             from leaking
> >      >      >      >              >     Ethernet frames with BFD Control
> >     packet
> >      >     to a
> >      >      >     tenant.
> >      >      >      >              >     You've suggested using a MAC
> >     address to
> >      >     trap the
> >      >      >      >             control packet at
> >      >      >      >              >     VTEP. What that address could
> be? We
> >      >     had proposed
> >      >      >      >             using the
> >      >      >      >              >     dedicated MAC and VTEP's MAC and
> >     both
> >      >     raised
> >      >      >     concerns
> >      >      >      >             among VXLAN
> >      >      >      >              >     experts. The idea of using
> >     Management
> >      >     VNI may
> >      >      >     be more
> >      >      >      >             acceptable
> >      >      >      >              >     based on its similarity to the
> >     practice
> >      >     of using
> >      >      >      >             Management VLAN.
> >      >      >      >              >
> >      >      >      >              >     Regards,
> >      >      >      >              >     Greg
> >      >      >      >              >
> >      >      >      >              >     On Wed, Jul 31, 2019 at 12:03 PM
> >     Dinesh
> >      >     Dutt
> >      >      >      >             <[e-mail]
> >     <mailto:[e-mail]> <mailto:[e-mail]
> >     <mailto:[e-mail]>>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>
> >     <mailto:[e-mail] <mailto:[e-mail]>>>
> >      >      >     <mailto:[e-mail] <mailto:[e-mail]>
> >     <mailto:[e-mail] <mailto:[e-mail]>>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>
> >     <mailto:[e-mail] <mailto:[e-mail]>>>>
> >      >      >      >              >     <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail] <mailto:[e-mail]>
> >     <mailto:[e-mail] <mailto:[e-mail]>>>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>
> >     <mailto:[e-mail] <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail] <mailto:[e-mail]>
> >     <mailto:[e-mail] <mailto:[e-mail]>>>>>>
> >      >      >      >             wrote:
> >      >      >      >              >
> >      >      >      >              >         Hi Greg,
> >      >      >      >              >
> >      >      >      >              >         As long as the inner MAC
> >     address is
> >      >     such
> >      >      >     that the
> >      >      >      >             packet is
> >      >      >      >              >         trapped to the CPU, it
> should be
> >      >     fine for
> >      >      >     use as
> >      >      >      >             an inner MAC is
> >      >      >      >              >         it not? Stating that is
> >     better than
> >      >     trying to
> >      >      >      >             force a management
> >      >      >      >              >         VNI. What if someone wants
> >     to test
> >      >      >     connectivity
> >      >      >      >             on a specific
> >      >      >      >              >         VNI? I would not pick a
> >     loopback IP
> >      >      >     address for
> >      >      >      >             this since that
> >      >      >      >              >         address range is host/node
> local
> >      >     only. Is
> >      >      >     there a
> >      >      >      >             reason you're
> >      >      >      >              >         not using the VTEP IP as the
> >     inner IP
> >      >      >     address ?
> >      >      >      >              >
> >      >      >      >              >         Dinesh
> >      >      >      >              >
> >      >      >      >              >         On Wed, Jul 31, 2019 at 5:48
> AM
> >      >     Greg Mirsky
> >      >      >      >              >         <[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]> <mailto:[e-mail]
> >     <mailto:[e-mail]>>>
> >      >      >      >             <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>>> <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]> <mailto:[e-mail]
> >     <mailto:[e-mail]>>>
> >      >      >      >             <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail] <mailto:[e-mail]>>
> >      >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>
> >      >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>>>>> wrote:
> >      >      >      >              >
> >      >      >      >              >             Dear All,
> >      >      >      >              >             thank you for your
> comments,
> >      >      >     suggestions on
> >      >      >      >             this issue,
> >      >      >      >              >             probably the most
> >     challenging
> >      >     for this
> >      >      >      >             specification. In the
> >      >      >      >              >             course of our
> discussions,
> >      >     we've agreed to
> >      >      >      >             abandon the
> >      >      >      >              >             request to allocate the
> >      >     dedicated MAC
> >      >      >     address
> >      >      >      >             to be used as
> >      >      >      >              >             the destination MAC
> >     address in
> >      >     the inner
> >      >      >      >             Ethernet frame.
> >      >      >      >              >             Also, earlier using VNI
> >     0 was
> >      >     changed from
> >      >      >      >             mandatory to one
> >      >      >      >              >             of the options an
> >      >     implementation may
> >      >      >     offer to
> >      >      >      >             an operator.
> >      >      >      >              >             The most recent
> >     discussion was
> >      >     whether
> >      >      >     VTEP's
> >      >      >      >             MAC address
> >      >      >      >              >             might be used as the
> >      >     destination MAC
> >      >      >     address
> >      >      >      >             in the inner
> >      >      >      >              >             Ethernet frame. As I
> >     recall it, the
> >      >      >     comments
> >      >      >      >             from VXLAN
> >      >      >      >              >             experts equally split
> >     with one
> >      >     for it
> >      >      >     and one
> >      >      >      >             against. Hence
> >      >      >      >              >             I would like to propose
> >     a new
> >      >     text to
> >      >      >     resolve
> >      >      >      >             the issue. The
> >      >      >      >              >             idea is to let an
> >     operator select
> >      >      >     Management
> >      >      >      >             VNI and use
> >      >      >      >              >             that VNI in VXLAN
> >     encapsulation
> >      >     of BFD
> >      >      >      >             Control packets:
> >      >      >      >              >             NEW TEXT:
> >      >      >      >              >
> >      >      >      >              >                 An operator MUST
> >     select a VNI
> >      >      >     number to
> >      >      >      >             be used as
> >      >      >      >              >                 Management VNI. VXLAN
> >      >     packet for
> >      >      >      >             Management VNI MUST NOT
> >      >      >      >              >                 be sent to a tenant.
> VNI
> >      >     number 1 is
> >      >      >      >             RECOMMENDED as the
> >      >      >      >              >                 default for
> >     Management VNI.
> >      >      >      >              >
> >      >      >      >              >             With that new text, what
> >     can be the
> >      >      >     value of
> >      >      >      >             the destination
> >      >      >      >              >             MAC in the inner
> Ethernet? I
> >      >     tend to
> >      >      >     believe
> >      >      >      >             that it can be
> >      >      >      >              >             anything and ignored by
> the
> >      >     reciever VTEP.
> >      >      >      >             Also, if the
> >      >      >      >              >             trapping is based on VNI
> >      >     number, the
> >      >      >      >             destination IP address
> >      >      >      >              >             of the inner IP packet
> >     can from
> >      >     the range
> >      >      >      >             127/8 for IPv4,
> >      >      >      >              >             and for IPv6 from the
> range
> >      >      >      >             0:0:0:0:0:FFFF:7F00:0/104. And
> >      >      >      >              >             lastly, the TTL to be
> >     set to 1 (no
> >      >      >     change here).
> >      >      >      >              >
> >      >      >      >              >             Much appreciate your
> >     comments,
> >      >      >     questions, and
> >      >      >      >             suggestions.
> >      >      >      >              >
> >      >      >      >              >             Best regards,
> >      >      >      >              >             Greg
> >      >      >      >              >
> >      >      >      >
> >      >      >
> >      >
> >
>