Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[John E Drake <[e-mail]>]

Lucy,

That presupposes that the group likes either of the two proposed solutions in your draft.  

Yours Irrespectively,

John

> -----Original Message-----
> From: BESS [mailto:[e-mail]] On Behalf Of Lucy yong
> Sent: Friday, November 20, 2015 11:49 AM
> To: EXT - [e-mail]; Henderickx, Wim (Wim);
> [e-mail]
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> 
> Share my 2 cent.
> 
> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
> Option C is a way to realize it. Both solutions summarized by Thomas have no
> change on WAN VPN side and seamlessly work with WAN VPN option C.
> However, to support either solution, DC has to do some enhancement on DC
> BR or ToR switch, etc, which dictates to different implementations within a
> DC. Option C pro and con remains regardless what implementation is used in
> a DC.
> 
> Two solutions should not coexist in one DC (does not make a sense), but it
> does not matter if one DC operator prefers to use one solution and another
> DC prefers to use another solution. Since there are many cloud providers
> today, it is worth for the WG to document both solutions and point out the
> implementation requirements on impacted components. Then, up to
> vendors and operators to select a solution for their DC.
> 
> It does not make a sense for us to vote which solution is better here because
> a better solution for a DC depends on many factors.
> 
> 
> Lucy
> 
> 
> 
> 
> 
> -----Original Message-----
> From: BESS [mailto:[e-mail]] On Behalf Of
> [e-mail]
> Sent: Friday, November 20, 2015 3:02 AM
> To: Henderickx, Wim (Wim); [e-mail]
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> 
> 2015-11-19, Henderickx, Wim (Wim):
> > WH> I vote for a an evolution of switches/TORs that have proper
> > support for this. I hope some HW vendors of TOR chips shime in, but I
> > am told the MPLS solution is possible in the next generation chips
> > they are working on.
> 
> Well, it looks like the key questions are:
> - when would ToR chips support MPLS/MPLS/UDP ?  (the generation that has
> been released recently but not present in most shipping ToRs yet, the next
> one ?)
> - do we want an interim VXLAN-based solution ? (that will involve at best a
> performance penalty with existing chips, and at worse impossible to
> implement -- we haven't seen clear information in this thread)
> 
> -Thomas
> 
> 
> >> Zhuangshunwan  :
> >>>
> >>> Hi Diego,
> >>>
> >>> Thanks for your comments. Pls see inline with [Vincent].
> >>>
> >>> Vincent
> >>>
> >>> *发件人:*BESS [mailto:[e-mail]] *代表 *Diego Garcia
> del Rio
> >>> *发送时间:*2015年11月18日14:25 *收件人:*[e-mail] *主题
> :*Re: [bess]
> >>> draft-hao-bess-inter-nvo3-vpn-optionc
> >>>
> >>> Some comments from my side, I think the draft makes quite a few
> >>> assumptions on specific implementation details that are way too
> >>> general to be considered widely available. Most of the TOR devices
> >>> today already pay a double-pass penalty when doing routing of
> >>> traffic in/out of vxlan-type tunnels. Only the newest generation can
> >>> route into tunnels without additional passes. And are definitively
> >>> limited in being able to arbitrary select UDP ports on a per tunnel
> >>> basis. In fact, most are even limited at using more than one VNID
> >>> per "service" of sorts. [Vincent]: Yes, the new generation BCM
> >>> chipset has already supported VXLAN routing without additional
> >>> passes. For OVS/TOR, VXLAN encapsulation is more popular than
> >>> MPLSoGRE/UDP, and has better performance. The IP-addressed based
> >>> implementation which would, I assume, imply assigning one or more
> >>> CIDRs to a loopback interface on the ASBR-d is also quite arbitrary
> >>> and does not seem like a technically "clean" solution. (besides
> >>> burning tons of IPs). As a side-note, most PE-grade routers i've
> >>> worked with were quite limited in terms of IP addresses used for
> >>> tunnel termination and it wasn't that just a simple pool can be
> >>> used. [Vincent]: I think the larger VTEP IP address range on ASBR-d
> >>> has no limitations.
> >>> For the hardware on ASBR-d, it has capability to terminate multiple
> >>> VXLAN tunnels with arbitrary destination VTEP IP addresses. Wim's
> >>> mentions on cases where the Application itself, hosted in a
> >>> datacenter, would be part of the option-C interconnect, was
> >>> dismissed without much discussion so far, while, if we look in
> >>> detail at the type of users which will even consider a complex
> >>> topology like model-C its most likely users and operators very
> >>> familiar with MPLS VPNs in the WAN. Those type of operators will
> >>> most likely be very interested in deploying MPLS or WAN-grade
> >>> applications (i.e., virtual-routers or other
> >>> VNFs) in the DC and thus its highly likely that the interconnect
> >>> would not terminate at the NVE itself but rather the TS (the virtual
> >>> machine). Also, the use of UDP ports at random would imply quite
> >>> complex logic on the ASBR-d IMHO. Im not saying its impossible, but
> >>> since the received packet now not only has to be received on a
> >>> random (though locally chosen) UDP port and this information
> >>> preserved in the pipeline to be able to do the
> >>> double-tunnel-stitching, it sounds quite complex. I am sure someone
> >>> in the list will mention this has already been implemented
> >>> somewhere, and I won't argue with that. And let's not even bring
> >>> into account what this would do to any DC middlebox that now has to
> >>> look at vxlan over almost any random port. We have to go back to the
> >>> "is it a 4 or is it a 6 in byte x" heuristics to try to guess
> >>> whether the packet is vxlan or just something entirely different
> >>> running over IP. [Vincent]: Using NP or multi-core CPU hardware
> >>> technology, it can be implemented although deeper packet inspection
> >>> is needed to perform UDP port and MPLS stitching. In general I feel
> >>> the proposed solution seems to be fitting of a specific use-case
> >>> which is not really detailed
> >>> in the draft and does not describe   a solution that would
> >>> "elegantly" solve the issues at hand. It just feels like we're using
> >>> any available bit-space to stuff data into places that do not
> >>> necesarily belong. Yes, MPLS encapsulations on virtual switches are
> >>> not yet fully available, and there can be some performance penalty
> >>> on the TORs, but the solutions are much cleaner from a control and
> >>> data plane point of view. Maybe I'm too utopic. [Vincent]: I think
> >>> pure VXLAN solution has its scenario, it's general rather than
> >>> specific. We can't require all OVS/NVEs support VXLAN + MPLSoGRE at
> >>> the same time. Best regards, Diego
> >>> --------------------------------------------------------------------
> >>> -------------
> >>>
> >>>
> Hi,
> >>> The problem we are trying to solve is to reduce data center
> >>> GW/ASBR-d's forwarding table size, the motivation is same as
> >>> traditional MPLS VPN option-C. Currently, the most common practise
> >>> on ASBR-d is to terminate VXLAN encapsulation, look up local routing
> >>> table, and then perform MPLS encapsulation to the WAN network.
> >>> ASBR-d needs to maintain all VM's MAC/IP. In Option-C method, only
> >>> transport layer information needed to be maintained at GW/ASBR-d,
> >>> the scalability will be greatly enhanced. Traditonal Option-C is
> >>> only for MPLS VPN network interworking, because VXLAN is becoming
> >>> pervasive in data center, the solution in this draft was proposed
> >>> for the heterogeneous network interworking. The advantage of this
> >>> solution is that only VXLAN encapsulation is required for OVS/TOR.
> >>> Unlike Wim's solution, east-west bound traffic uses VXLAN encap,
> >>> while north-south bound traffic uses MPLSoGRE/UDP encap. There are
> >>> two solutions in this draft: 1. Using VXLAN tunnel destination IP
> >>> for stitching at ASBR-d. No data plane modification requirements on
> >>> OVS or TOR switches, only hardware changes on ASBR-d. ASBR-d
> >>> normally is router, it has capability to realize the hardware
> >>> changes. It will consume many IP addresses and the IP pool for
> >>> allocation needs to be configured on ASBR-d beforehand. 2. Using
> >>> VXLAN destination UDP port for stitching at ASBR-d. Compared with
> >>> solution 1, less IP address will be consumed for allocation. If UDP
> >>> port range is too large, we can combine with solution 1 and 2. In
> >>> this solution, both data plane modification changes are needed at
> >>> OVS/TOR and ASBR-d. ASBR-d also has capability to realize the
> >>> hardware changes. For OVS, it also can realize the data plane
> >>> changes. For TOR switch, it normally can't realize this function.
> >>> This solution mainly focuses on pure software based overlay network,
> >>> it has more scalability. In public cloud data center, software based
> >>> overlay network is the majority case. Whether using solution 1 or 2
> >>> depends on the operators real envionment. So I think our solution
> >>> has no flaws, it works fine.
> >>> Thanks, weiguo ________________________________ From: BESS
> >>> [[e-mail] <mailto:[e-mail]>] on behalf of
> >>> John E Drake [[e-mail] <mailto:[e-mail]>]
> >>> Sent: Wednesday, November 18, 2015 2:49 To: Henderickx, Wim (Wim);
> >>> EXT - [e-mail] <mailto:[e-mail]>;
> BESS
> >>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc Hi, I
> >>> think Wim has conclusively demonstrated that this draft has fatal
> >>> flaws and I don’t support it.  I also agree with his suggestion that
> >>> we first figure out what problem we are trying to solve before
> >>> solving it. Yours Irrespectively, John From: BESS
> >>> [mailto:[e-mail] <mailto:[e-mail]>] On
> >>> Behalf Of Henderickx, Wim (Wim) Sent: Tuesday, November 17, 2015
> >>> 12:49 PM To: EXT - [e-mail]
> >>> <mailto:[e-mail]>; BESS Subject: Re: [bess]
> >>> draft-hao-bess-inter-nvo3-vpn-optionc — Snip — No, the spec as it is
> >>> can be implemented in its VXLAN variant with existing vswitches
> >>> (e.g. OVS allows to choose the VXLAN destination port, ditto for the
> >>> linux kernel stack). (ToR is certainly another story, most of them
> >>> not having a flexible enough VXLAN dataplane nor support for any
> >>> MPLS-over-IP.) WH> and how many ports simultaneously would they
> >>> support? For this to work every tenant needs a different VXLAN UDP
> >>> destination port/receive port. There might be SW elements that could
> >>> do some of this, but IETF defines solutions which should be
> >>> implemented across the board HW/SW/etc.
> >>> Even if some SW switches can do this, the proposal will impose so
> >>> many issues in HW/data-plane engines that I cannot be behind this
> >>> solution. To make this work generically we will have to make changes
> >>> anyhow. Given this, we better do it in the right way and guide the
> >>> industry to a solution which does not imply those complexities.
> >>> Otherwise we will stick with these specials forever with all
> >>> consequences (bugs, etc). - snip - From:
> >>> "[e-mail]
> >>>
> <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>" <[e-mail]
> >>>
> <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>> Organization: Orange Date:
> >>> Tuesday 17 November 2015 at 01:37 To: Wim Henderickx
> >>> <[e-mail]
> >>> <mailto:wim.henderickx@alcatel-
> lucent.com><mailto:wim.henderickx@alc
> >>> atel-lucent.com
> >>>
> >>>
> <mailto:[e-mail]>>>, BESS <[e-mail]
> >>> <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>> Subject: Re: [bess]
> >>> draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, WG, 2015-11-16,
> >>> Henderickx, Wim (Wim): 2015-11-13, Henderickx, Wim (Wim): Thomas,
> we
> >>> can discuss forever and someone need to describe requirements, but
> >>> the current proposal I cannot agree to for the reasons explained.
> >>> TM> Well, although discussing forever is certainly not the goal, the
> >>> reasons for rejecting a proposal need to be thoroughly understood.
> >>> WH> my point is what is the real driver for supporting a plain VXLAN
> >>> data-plane here, the use cases I have seen in this txt is always
> >>> where an application behind a NVE/TOR is demanding option c, but
> >>> none of the NVE/TOR elements.
> >>> My understanding is that the applications  are contexts where
> >>> overlays are present is when workloads (VMs or baremetal) need to be
> >>> interconnected with VPNs. In these contexts, there can be reasons to
> >>> want Option C to reduce the state on ASBRs. In these context, its
> >>> not the workload (VM or baremetal) that would typically handle VRFs,
> >>> but really the vswitch or ToR. WH2> can it not be all cases:
> >>> TOR/vswitch/Application. I would make the solution flexible to
> >>> support all of these not? 2015-11-13, Henderickx, Wim (Wim): TM> The
> >>> right trade-off to make may in fact depend on whether you prefer:
> >>> (a) a new dataplane stitching behavior on DC ASBRs (the behavior
> >>> specified in this draft) or
> >>> (b) an evolution of the encaps on the vswitches and ToRs to support
> >>> MPLS/MPLS/(UDP or GRE) WH> b depends on the use case I don't get
> >>> what you mean by "b depends on the use case". WH> see my above
> >>> comment. If the real use case is an application behind NVE/TOR
> >>> requiring model C, than all the discussion on impact on NVE/TOR is
> >>> void. As such I want to have a discussion on the real
> >>> driver/requirement for option c interworking with an IP based
> >>> Fabric. Although I can agree than detailing requirements can always
> >>> help, I don't think one can assume a certain application to dismiss
> >>> the proposal. WH> for me the proposal is not acceptable for the
> >>> reasons explained: too much impact on the data-planes I wrote the
> >>> above based on the idea that the encap used in MPLS/MPLS/(UDP or
> >>> GRE), which hence has to be supported on the ToRs and vswitches.
> >>> Another possibility would be service-label/middle-label/Ethernet
> >>> assuming an L2 adjacency between vswitches/ToRs and ASBRs, but this
> >>> certainly does not match your typical DC architecture. Or perhaps
> >>> had you something else in mind ? WH> see above. The draft right now
> >>> also requires changes in existing TOR/NVE so for me all this
> >>> discussion/debate is void. No, the spec as it is can be implemented
> >>> in its VXLAN variant with existing vswitches (e.g. OVS allows to
> >>> choose the VXLAN destination port, ditto for the linux kernel
> >>> stack). (ToR is certainly another story, most of them not having a
> >>> flexible enough VXLAN dataplane nor support for any MPLS-over-IP.)
> >>> WH> and how many ports simultaneously would they support? WH> and
> >>> depending on implementation you don’t need to change any of the
> >>> TOR/vswitches. Does this mean that for some implementations you may
> >>> not need to change any of the TOR/vswitches, but that for some
> >>> others you may ? WH> any proposal on the table requires changes, so
> >>> for me this is not a valid discussion See above, the proposal in the
> >>> draft does not necessarily need changes in vswitches. Let me take a
> >>> practical example : while I can quite easily see how to implement
> >>> the procedures in draft-hao-bess-inter-nvo3-vpn-optionc based on
> >>> current vswitch implementations of VXLAN, the lack of
> >>> MPLS/MPLS/(UDP, GRE) support in commonplace vswitches seems to
> me as
> >>> making that alternate solution you suggest harder to implement. WH>
> >>> I would disagree to this. Tell me which switch/TOR handles multiple
> >>> UDP ports for VXLAN ? I mentioned _v_switches, and many do support a
> >>> variable destination port for VXLAN, which is sufficient to
> >>> implement what the draft proposes. -Thomas From: Thomas Morin
> >>> <[e-mail]
> >>>
> <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>> Organization: Orange Date:
> >>> Friday 13 November 2015 at 09:57 To: Wim Henderickx
> >>> <[e-mail]
> >>> <mailto:wim.henderickx@alcatel-
> lucent.com><mailto:wim.henderickx@alc
> >>> atel-lucent.com
> >>>
> >>>
> <mailto:[e-mail]>>>
> >>> Cc: "[e-mail] <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>" <[e-mail]
> >>> <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>> Subject: Re: [bess]
> >>> draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, I agree on the
> >>> analysis that this proposal is restricted to implementations that
> >>> supports the chosen encap with non-IANA ports (which may be hard to
> >>> achieve for instance on hardware implementations, as you suggest),
> >>> or to context where managing multiple IPs would be operationally
> >>> viable. However, it does not seem obvious to me how the alternative
> >>> you propose [relying on 3-label option C with an
> >>> MPLS/MPLS/(UDP|GRE) encap] addresses the issue of whether the
> encap
> >>> behavior is supported or not (e.g. your typical ToR chipset possibly
> >>> may not support this kind of encap,  and even software-based
> >>> switches may not be ready to support that today).
> >>> My take is that having different options to adapt to various
> >>> implementations constraints we may have would have value. (+ one
> >>> question below on VXLAN...) -Thomas 2015-11-12, Henderickx, Wim
> >>> (Wim): On VXLAN/NVGRE, do you challenge the fact that they would be
> >>> used with a dummy MAC address that would be replaced by the right
> >>> MAC by a sender based on an ARP request when needed ? Is the above
> >>> the issue you had in mind about VXLAN and NVGRE ? WH> yes I you
> >>> don't mind me asking : why do you challenge that ?
> >>>
> 
> __________________________________________________________
> __________________________________________________________
> _____
> 
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites
> ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez
> le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les
> messages electroniques etant susceptibles d'alteration, France Telecom -
> Orange decline toute responsabilite si ce message a ete altere, deforme ou
> falsifie. Merci
> 
> This message and its attachments may contain confidential or privileged
> information that may be protected by law; they should not be distributed,
> used or copied without authorization.
> If you have received this email in error, please notify the sender and delete
> this message and its attachments.
> As emails may be altered, France Telecom - Orange shall not be liable if this
> message was modified, changed or falsified.
> Thank you.
> 
> _______________________________________________
> BESS mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/bess
> _______________________________________________
> BESS mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/bess