Re: [nvo3] BFD over VXLAN: Trapping BFD Control packet at VTEP

[Dinesh Dutt <[e-mail]>]

Oops, sorry for misspelling your name Joel,

Dinesh

On Wed, Oct 23, 2019 at 1:47 AM, Dinesh Dutt <[e-mail]> wrote:
> oel,
> 
> I'm a tad frustrated that we're rehashing this discussions all over 
> again. I specifically explained all the questions that were raised at 
> that time. Let me try one last time.
> 
> 1. BFD for VTEP is only useful for testing VXLAN plumbing, not the 
> underlay itself.
> 2. So, the question is what do we use for VNI and the inner header?
> 3. The inner header is an IP packet because it is BFD. The IP address 
> and the corresponding MAC address used MUST be one that is owned by 
> the VTEP in the VNI that is used in the packet
> 
> This is sufficient to come up with an implementation that only ever 
> tests one VNI or multiple VNIs between thr same pair of VTEPs. It is 
> upto the users to decide what VNI, inner MAC and IP to use. The only 
> restriction is that the VTEP must own those addresses to (i) prevent 
> the packet from leaking to tenants and (ii) allow the tenants 
> themselves to be running BFD.
> 
> If implementations want to use VNI 1 as the recommended default VNI 
> to use, that is fine. But if implementations want to pick more than 1 
> because they have a need to do so (I've seen operators do this 
> because of their specific use case), then they can as long as they 
> satisfy point 3, the draft is done. Why does there need to be any 
> more discussion? The draft does need to spell out that using more 
> than VNI has scaling issues that the user needs to be aware of, and 
> it does.
> 
> Dinesh
> 
> On Wed, Oct 23, 2019 at 1:28 AM, Joel Halpern Direct 
> <[e-mail]> wrote:
>> That is input to the calculation at the VTEP.  It is NOT information 
>> sued by the network between the VTEPs.
>> 
>> As such, the VTEPs can emulate that by adjusting the source ports 
>> that it uses for the VFD packets.  The network does not need the VNI 
>> to actually be varied to achieve this purpose.
>> 
>> Yours,
>> Joel
>> 
>> On 10/22/2019 3:55 PM, Greg Mirsky wrote:
>>> Hi Joel,
>>> RFC 7348 suggests using information from the inner packet to 
>>> calculate the value to be used in the Source UDP port number:
>>>        -  Source Port:  It is recommended that the UDP source port 
>>> number
>>>           be calculated using a hash of fields from the inner 
>>> packet --
>>>           one example being a hash of the inner Ethernet frame's 
>>> headers.
>>>           This is to enable a level of entropy for the ECMP/load-
>>>           balancing of the VM-to-VM traffic across the VXLAN 
>>> overlay.
>>>  From that text, I assume that VNI may be used as input for hashing 
>>> function. If BFD over VXLAN doesn't support per VNI BFD session, 
>>> then it cannot monitor multiple paths in underlay used to balance 
>>> VM-to-VM traffic between the same pair of VTEPs. In my opinion, 
>>> this is perfectly fine if that is WG's agreement. I'm glad we are 
>>> discussing this and will have a conclusion.
>>> 
>>> Regards,
>>> Greg
>>> 
>>> On Tue, Oct 22, 2019 at 3:30 PM Joel M. Halpern 
>>> <[e-mail] <mailto:[e-mail]>> wrote:
>>> 
>>>     As I recall, the VNI is not in the same place nor the same size 
>>> as the
>>>     TCP / UDP ports.  So it seems very unlikely that it would be 
>>> used in
>>>     ECMP.  In fact, avoiding that is why VXLAN does interesting 
>>> things with
>>>     the source UDP port.  Which the BFD can do.  And presumably 
>>> MUST do if
>>>     it was path matching.
>>> 
>>>     Yours,
>>>     Joel
>>> 
>>>     On 10/22/2019 3:16 PM, Greg Mirsky wrote:
>>>      > Hi Joel,
>>>      > if the underlay may balance VXLAN between two VTEPs using 
>>> VNI in
>>>      > addition to other fields, then Option 2 has a certain value 
>>> in my
>>>     opinion.
>>>      >
>>>      > Regards,
>>>      > Greg
>>>      >
>>>      > On Tue, Oct 22, 2019 at 3:06 PM Joel M. Halpern
>>>     <[e-mail] <mailto:[e-mail]>
>>>      > <mailto:[e-mail] <mailto:[e-mail]>>> 
>>> wrote:
>>>      >
>>>      >     I do not understand the value of option 2.
>>>      >     Which is why I asked in my initial review to move to 
>>> option 1.
>>>      >
>>>      >     And option 2 requires stealing MAC addresses from the 
>>> users,
>>>     which
>>>      >     seems
>>>      >     to me to be a very bad thing that option 1 avoids.
>>>      >
>>>      >     Yours,
>>>      >     Joel
>>>      >
>>>      >     On 10/22/2019 2:17 PM, Greg Mirsky wrote:
>>>      >      > Hi Anoop, et al.,
>>>      >      > I agree with your understanding of what is being 
>>> defined
>>>     in the
>>>      >     current
>>>      >      > version of the BFD over VxLAN specification. But, as I
>>>      >     understand, the
>>>      >      > WG is discussing the scope before the WGLC is closed. 
>>> I
>>>     believe
>>>      >     there
>>>      >      > are three options:
>>>      >      >
>>>      >      >  1. single BFD session between two VTEPs
>>>      >      >  2. single BFD session per VNI between two VTEPs
>>>      >      >  3. multiple BFD sessions per VNI between two VTEPs
>>>      >      >
>>>      >      > The current text reflects #2. Is WG accepts this 
>>> scope? If
>>>     not,
>>>      >     which
>>>      >      > option WG would accept?
>>>      >      >
>>>      >      > Regards,
>>>      >      > Greg
>>>      >      >
>>>      >      > On Tue, Oct 22, 2019 at 2:09 PM Anoop Ghanwani
>>>      >     <[e-mail] <mailto:[e-mail]>
>>>     <mailto:[e-mail] <mailto:[e-mail]>>
>>>      >      > <mailto:[e-mail]
>>>     <mailto:[e-mail]> <mailto:[e-mail]
>>>     <mailto:[e-mail]>>>> wrote:
>>>      >      >
>>>      >      >     I concur with Joel's assessment with the following
>>>      >     clarifications.
>>>      >      >
>>>      >      >     The current document is already capable of 
>>> monitoring
>>>      >     multiple VNIs
>>>      >      >     between VTEPs.
>>>      >      >
>>>      >      >     The issue under discussion was how do we use BFD 
>>> to
>>>     monitor
>>>      >     multiple
>>>      >      >     VAPs that use the same VNI between a pair of 
>>> VTEPs.     The use case
>>>      >      >     for this is not clear to me, as from my 
>>> understanding,
>>>     we cannot
>>>      >      >     have a situation with multiple VAPs using the same
>>>     VNI--there
>>>      >     is 1:1
>>>      >      >     mapping between VAP and VNI.
>>>      >      >
>>>      >      >     Anoop
>>>      >      >
>>>      >      >     On Tue, Oct 22, 2019 at 6:06 AM Joel M. Halpern
>>>      >     <[e-mail] <mailto:[e-mail]>
>>>     <mailto:[e-mail] <mailto:[e-mail]>>
>>>      >      >     <mailto:[e-mail]
>>>     <mailto:[e-mail]> <mailto:[e-mail]
>>>     <mailto:[e-mail]>>>> wrote:
>>>      >      >
>>>      >      >           From what I can tell, there are two separate
>>>     problems.
>>>      >      >         The document we have is a VTEP-VTEP monitoring
>>>     document.
>>>      >     There
>>>      >      >         is no
>>>      >      >         need for that document to handle the multiple 
>>> VNI
>>>     case.
>>>      >      >         If folks want a protocol for doing BFD 
>>> monitoring
>>>     of things
>>>      >      >         behind the
>>>      >      >         VTEPs (multiple VNIs), then do that as a 
>>> separate
>>>      >     document.   The
>>>      >      >         encoding will be a tenant encoding, and thus 
>>> sesparate
>>>      >     from what is
>>>      >      >         defined in this document.
>>>      >      >
>>>      >      >         Yours,
>>>      >      >         Joel
>>>      >      >
>>>      >      >         On 10/21/2019 5:07 PM, Jeffrey Haas wrote:
>>>      >      >          > Santosh and others,
>>>      >      >          >
>>>      >      >          > On Thu, Oct 03, 2019 at 07:50:20PM +0530,
>>>     Santosh P K
>>>      >     wrote:
>>>      >      >          >>     Thanks for your explanation. This 
>>> helps a
>>>     lot. I
>>>      >     would
>>>      >      >         wait for more
>>>      >      >          >> comments from others to see if this what 
>>> we
>>>     need in this
>>>      >      >         draft to be
>>>      >      >          >> supported based on that we can provide 
>>> appropriate
>>>      >     sections
>>>      >      >         in the draft.
>>>      >      >          >
>>>      >      >          > The threads on the list have spidered to 
>>> the point
>>>      >     where it
>>>      >      >         is challenging
>>>      >      >          > to follow what the current status of the 
>>> draft
>>>     is, or
>>>      >     should
>>>      >      >         be.  :-)
>>>      >      >          >
>>>      >      >          > However, if I've followed things properly, 
>>> the
>>>      >     question below
>>>      >      >         is really the
>>>      >      >          > hinge point on what our encapsulation for 
>>> BFD
>>>     over vxlan
>>>      >      >         should look like.
>>>      >      >          > Correct?
>>>      >      >          >
>>>      >      >          > Essentially, do we or do we not require the
>>>     ability to
>>>      >     permit
>>>      >      >         multiple BFD
>>>      >      >          > sessions between distinct VAPs?
>>>      >      >          >
>>>      >      >          > If this is so, do we have a sense as to 
>>> how we
>>>     should
>>>      >     proceed?
>>>      >      >          >
>>>      >      >          > -- Jeff
>>>      >      >          >
>>>      >      >          > [context preserved below...]
>>>      >      >          >
>>>      >      >          >> Santosh P K
>>>      >      >          >>
>>>      >      >          >> On Wed, Sep 25, 2019 at 8:10 AM
>>>     <[e-mail] <mailto:[e-mail]>
>>>      >     <mailto:[e-mail] 
>>> <mailto:[e-mail]>>
>>>      >      >         <mailto:[e-mail]
>>>     <mailto:[e-mail]>
>>>      >     <mailto:[e-mail] 
>>> <mailto:[e-mail]>>>>
>>>     wrote:
>>>      >      >          >>
>>>      >      >          >>> Hi Santosh,
>>>      >      >          >>>
>>>      >      >          >>>
>>>      >      >          >>> With regard to the question whether we 
>>> should
>>>     allow
>>>      >      >         multiple BFD sessions
>>>      >      >          >>> for the same VNI or not, IMHO we should 
>>> allow
>>>     it, more
>>>      >      >         explanation as
>>>      >      >          >>> follows.
>>>      >      >          >>>
>>>      >      >          >>> Below is a figure derived from figure 2 
>>> of
>>>     RFC8014 (An
>>>      >      >         Architecture for
>>>      >      >          >>> Data-Center Network Virtualization over 
>>> Layer
>>>     3 (NVO3)).
>>>      >      >          >>>
>>>      >      >          >>>                      |         Data 
>>> Center
>>>     Network (IP)
>>>      >      >              |
>>>      >      >          >>>                      |
>>>      >      >             |
>>>      >      >          >>>
>>>      >      >         +-----------------------------------------+
>>>      >      >          >>>                           |              
>>>                  |
>>>      >      >          >>>                           |       Tunnel
>>>     Overlay      |
>>>      >      >          >>>              +------------+---------+
>>>      >      >           +---------+------------+
>>>      >      >          >>>              | +----------+-------+ |    
>>>    |
>>>      >      >         +-------+----------+ |
>>>      >      >          >>>              | |  Overlay Module  | |    
>>>    |
>>>     |  Overlay
>>>      >      >         Module  | |
>>>      >      >          >>>              | +---------+--------+ |    
>>>    |
>>>      >      >         +---------+--------+ |
>>>      >      >          >>>              |           |          |    
>>>    |
>>>      >         |
>>>      >      >                  |
>>>      >      >          >>>       NVE1   |           |          |    
>>>    |
>>>      >         |
>>>      >      >                  | NVE2
>>>      >      >          >>>              |  +--------+-------+  |    
>>>    |
>>>      >      >         +--------+-------+  |
>>>      >      >          >>>              |  |VNI1 VNI2  VNI1 |  |    
>>>    |  |
>>>      >     VNI1 VNI2
>>>      >      >         VNI1 |  |
>>>      >      >          >>>              |  +-+-----+----+---+  |    
>>>    |
>>>      >      >         +-+-----+-----+--+  |
>>>      >      >          >>>              |VAP1| VAP2|    | VAP3 |    
>>>        |VAP1|
>>>      >     VAP2|
>>>      >      >           | VAP3|
>>>      >      >          >>>              +----+-----+----+------+
>>>      >      >           +----+-----+-----+-----+
>>>      >      >          >>>                   |     |    |           
>>>             |
>>>      >         |     |
>>>      >      >          >>>                   |     |    |           
>>>             |
>>>      >         |     |
>>>      >      >          >>>                   |     |    |           
>>>             |
>>>      >         |     |
>>>      >      >          >>>
>>>      >      >             
>>> -------+-----+----+-------------------+-----+-----+-------
>>>      >      >          >>>                   |     |    |     
>>> Tenant            |
>>>      >         |     |
>>>      >      >          >>>              TSI1 | TSI2|    | TSI3      
>>>        TSI1|
>>>      >     TSI2|
>>>      >      >           |TSI3
>>>      >      >          >>>                  +---+ +---+ +---+       
>>>           +---+
>>>      >     +---+
>>>      >      >           +---+
>>>      >      >          >>>                  |TS1| |TS2| |TS3|       
>>>           |TS4|
>>>      >     |TS5|
>>>      >      >           |TS6|
>>>      >      >          >>>                  +---+ +---+ +---+       
>>>           +---+
>>>      >     +---+
>>>      >      >           +---+
>>>      >      >          >>>
>>>      >      >          >>> To my understanding, the BFD sessions 
>>> between
>>>     NVE1
>>>      >     and NVE2
>>>      >      >         are actually
>>>      >      >          >>> initiated and terminated at VAP of NVE.
>>>      >      >          >>>
>>>      >      >          >>> If the network operator want to set up 
>>> one
>>>     BFD session
>>>      >      >         between VAP1 of
>>>      >      >          >>> NVE1 and VAP1of NVE2, at the same time
>>>     another BFD
>>>      >     session
>>>      >      >         between VAP3 of
>>>      >      >          >>> NVE1 and VAP3 of NVE2, although the two 
>>> BFD
>>>     sessions are
>>>      >      >         for the same
>>>      >      >          >>> VNI1, I believe it's reasonable, so 
>>> that's why I
>>>      >     think we
>>>      >      >         should allow it
>>>      >      >
>>>      >      >         
>>> _______________________________________________
>>>      >      >         nvo3 mailing list
>>>      >      > [e-mail] <mailto:[e-mail]> 
>>> <mailto:[e-mail]
>>>     <mailto:[e-mail]>> <mailto:[e-mail] 
>>> <mailto:[e-mail]>
>>>      >     <mailto:[e-mail] <mailto:[e-mail]>>>
>>>      >      > https://www.ietf.org/mailman/listinfo/nvo3
>>>      >      >
>>>      >
>>>