Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong <[e-mail]>]

That is my 2 cents. Anyone can share his/her opinion. :)
Lucy

-----Original Message-----
From: John E Drake [mailto:[e-mail]] 
Sent: Friday, November 20, 2015 10:54 AM
To: Lucy yong; EXT - [e-mail]; Henderickx, Wim (Wim); [e-mail]
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Lucy,

That presupposes that the group likes either of the two proposed solutions in your draft.  

Yours Irrespectively,

John

> -----Original Message-----
> From: BESS [mailto:[e-mail]] On Behalf Of Lucy yong
> Sent: Friday, November 20, 2015 11:49 AM
> To: EXT - [e-mail]; Henderickx, Wim (Wim); 
> [e-mail]
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> 
> Share my 2 cent.
> 
> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
> Option C is a way to realize it. Both solutions summarized by Thomas 
> have no change on WAN VPN side and seamlessly work with WAN VPN option C.
> However, to support either solution, DC has to do some enhancement on 
> DC BR or ToR switch, etc, which dictates to different implementations 
> within a DC. Option C pro and con remains regardless what 
> implementation is used in a DC.
> 
> Two solutions should not coexist in one DC (does not make a sense), 
> but it does not matter if one DC operator prefers to use one solution 
> and another DC prefers to use another solution. Since there are many 
> cloud providers today, it is worth for the WG to document both 
> solutions and point out the implementation requirements on impacted 
> components. Then, up to vendors and operators to select a solution for their DC.
> 
> It does not make a sense for us to vote which solution is better here 
> because a better solution for a DC depends on many factors.
> 
> 
> Lucy
> 
> 
> 
> 
> 
> -----Original Message-----
> From: BESS [mailto:[e-mail]] On Behalf Of 
> [e-mail]
> Sent: Friday, November 20, 2015 3:02 AM
> To: Henderickx, Wim (Wim); [e-mail]
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> 
> 2015-11-19, Henderickx, Wim (Wim):
> > WH> I vote for a an evolution of switches/TORs that have proper
> > support for this. I hope some HW vendors of TOR chips shime in, but 
> > I am told the MPLS solution is possible in the next generation chips 
> > they are working on.
> 
> Well, it looks like the key questions are:
> - when would ToR chips support MPLS/MPLS/UDP ?  (the generation that 
> has been released recently but not present in most shipping ToRs yet, 
> the next one ?)
> - do we want an interim VXLAN-based solution ? (that will involve at 
> best a performance penalty with existing chips, and at worse 
> impossible to implement -- we haven't seen clear information in this 
> thread)
> 
> -Thomas
> 
> 
> >> Zhuangshunwan  :
> >>>
> >>> Hi Diego,
> >>>
> >>> Thanks for your comments. Pls see inline with [Vincent].
> >>>
> >>> Vincent
> >>>
> >>> *发件人:*BESS [mailto:[e-mail]] *代表 *Diego Garcia
> del Rio
> >>> *发送时间:*2015年11月18日14:25 *收件人:*[e-mail] *主题
> :*Re: [bess]
> >>> draft-hao-bess-inter-nvo3-vpn-optionc
> >>>
> >>> Some comments from my side, I think the draft makes quite a few 
> >>> assumptions on specific implementation details that are way too 
> >>> general to be considered widely available. Most of the TOR devices 
> >>> today already pay a double-pass penalty when doing routing of 
> >>> traffic in/out of vxlan-type tunnels. Only the newest generation 
> >>> can route into tunnels without additional passes. And are 
> >>> definitively limited in being able to arbitrary select UDP ports 
> >>> on a per tunnel basis. In fact, most are even limited at using 
> >>> more than one VNID per "service" of sorts. [Vincent]: Yes, the new 
> >>> generation BCM chipset has already supported VXLAN routing without 
> >>> additional passes. For OVS/TOR, VXLAN encapsulation is more 
> >>> popular than MPLSoGRE/UDP, and has better performance. The 
> >>> IP-addressed based implementation which would, I assume, imply 
> >>> assigning one or more CIDRs to a loopback interface on the ASBR-d 
> >>> is also quite arbitrary and does not seem like a technically 
> >>> "clean" solution. (besides burning tons of IPs). As a side-note, 
> >>> most PE-grade routers i've worked with were quite limited in terms 
> >>> of IP addresses used for tunnel termination and it wasn't that 
> >>> just a simple pool can be used. [Vincent]: I think the larger VTEP 
> >>> IP address range on ASBR-d has no limitations.
> >>> For the hardware on ASBR-d, it has capability to terminate 
> >>> multiple VXLAN tunnels with arbitrary destination VTEP IP 
> >>> addresses. Wim's mentions on cases where the Application itself, 
> >>> hosted in a datacenter, would be part of the option-C 
> >>> interconnect, was dismissed without much discussion so far, while, 
> >>> if we look in detail at the type of users which will even consider 
> >>> a complex topology like model-C its most likely users and 
> >>> operators very familiar with MPLS VPNs in the WAN. Those type of 
> >>> operators will most likely be very interested in deploying MPLS or 
> >>> WAN-grade applications (i.e., virtual-routers or other
> >>> VNFs) in the DC and thus its highly likely that the interconnect 
> >>> would not terminate at the NVE itself but rather the TS (the 
> >>> virtual machine). Also, the use of UDP ports at random would imply 
> >>> quite complex logic on the ASBR-d IMHO. Im not saying its 
> >>> impossible, but since the received packet now not only has to be 
> >>> received on a random (though locally chosen) UDP port and this 
> >>> information preserved in the pipeline to be able to do the 
> >>> double-tunnel-stitching, it sounds quite complex. I am sure 
> >>> someone in the list will mention this has already been implemented 
> >>> somewhere, and I won't argue with that. And let's not even bring 
> >>> into account what this would do to any DC middlebox that now has 
> >>> to look at vxlan over almost any random port. We have to go back 
> >>> to the "is it a 4 or is it a 6 in byte x" heuristics to try to 
> >>> guess whether the packet is vxlan or just something entirely 
> >>> different running over IP. [Vincent]: Using NP or multi-core CPU 
> >>> hardware technology, it can be implemented although deeper packet 
> >>> inspection is needed to perform UDP port and MPLS stitching. In 
> >>> general I feel the proposed solution seems to be fitting of a 
> >>> specific use-case which is not really detailed
> >>> in the draft and does not describe   a solution that would
> >>> "elegantly" solve the issues at hand. It just feels like we're 
> >>> using any available bit-space to stuff data into places that do 
> >>> not necesarily belong. Yes, MPLS encapsulations on virtual 
> >>> switches are not yet fully available, and there can be some 
> >>> performance penalty on the TORs, but the solutions are much 
> >>> cleaner from a control and data plane point of view. Maybe I'm too 
> >>> utopic. [Vincent]: I think pure VXLAN solution has its scenario, 
> >>> it's general rather than specific. We can't require all OVS/NVEs 
> >>> support VXLAN + MPLSoGRE at the same time. Best regards, Diego
> >>> ------------------------------------------------------------------
> >>> --
> >>> -------------
> >>>
> >>>
> Hi,
> >>> The problem we are trying to solve is to reduce data center 
> >>> GW/ASBR-d's forwarding table size, the motivation is same as 
> >>> traditional MPLS VPN option-C. Currently, the most common practise 
> >>> on ASBR-d is to terminate VXLAN encapsulation, look up local 
> >>> routing table, and then perform MPLS encapsulation to the WAN network.
> >>> ASBR-d needs to maintain all VM's MAC/IP. In Option-C method, only 
> >>> transport layer information needed to be maintained at GW/ASBR-d, 
> >>> the scalability will be greatly enhanced. Traditonal Option-C is 
> >>> only for MPLS VPN network interworking, because VXLAN is becoming 
> >>> pervasive in data center, the solution in this draft was proposed 
> >>> for the heterogeneous network interworking. The advantage of this 
> >>> solution is that only VXLAN encapsulation is required for OVS/TOR.
> >>> Unlike Wim's solution, east-west bound traffic uses VXLAN encap, 
> >>> while north-south bound traffic uses MPLSoGRE/UDP encap. There are 
> >>> two solutions in this draft: 1. Using VXLAN tunnel destination IP 
> >>> for stitching at ASBR-d. No data plane modification requirements 
> >>> on OVS or TOR switches, only hardware changes on ASBR-d. ASBR-d 
> >>> normally is router, it has capability to realize the hardware 
> >>> changes. It will consume many IP addresses and the IP pool for 
> >>> allocation needs to be configured on ASBR-d beforehand. 2. Using 
> >>> VXLAN destination UDP port for stitching at ASBR-d. Compared with 
> >>> solution 1, less IP address will be consumed for allocation. If 
> >>> UDP port range is too large, we can combine with solution 1 and 2. 
> >>> In this solution, both data plane modification changes are needed 
> >>> at OVS/TOR and ASBR-d. ASBR-d also has capability to realize the 
> >>> hardware changes. For OVS, it also can realize the data plane 
> >>> changes. For TOR switch, it normally can't realize this function.
> >>> This solution mainly focuses on pure software based overlay 
> >>> network, it has more scalability. In public cloud data center, 
> >>> software based overlay network is the majority case. Whether using 
> >>> solution 1 or 2 depends on the operators real envionment. So I 
> >>> think our solution has no flaws, it works fine.
> >>> Thanks, weiguo ________________________________ From: BESS 
> >>> [[e-mail] <mailto:[e-mail]>] on behalf 
> >>> of John E Drake [[e-mail] <mailto:[e-mail]>]
> >>> Sent: Wednesday, November 18, 2015 2:49 To: Henderickx, Wim (Wim); 
> >>> EXT - [e-mail] <mailto:[e-mail]>;
> BESS
> >>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc Hi, I 
> >>> think Wim has conclusively demonstrated that this draft has fatal 
> >>> flaws and I don’t support it.  I also agree with his suggestion 
> >>> that we first figure out what problem we are trying to solve 
> >>> before solving it. Yours Irrespectively, John From: BESS 
> >>> [mailto:[e-mail] <mailto:[e-mail]>] On 
> >>> Behalf Of Henderickx, Wim (Wim) Sent: Tuesday, November 17, 2015
> >>> 12:49 PM To: EXT - [e-mail] 
> >>> <mailto:[e-mail]>; BESS Subject: Re: [bess] 
> >>> draft-hao-bess-inter-nvo3-vpn-optionc — Snip — No, the spec as it 
> >>> is can be implemented in its VXLAN variant with existing vswitches 
> >>> (e.g. OVS allows to choose the VXLAN destination port, ditto for 
> >>> the linux kernel stack). (ToR is certainly another story, most of 
> >>> them not having a flexible enough VXLAN dataplane nor support for 
> >>> any
> >>> MPLS-over-IP.) WH> and how many ports simultaneously would they 
> >>> support? For this to work every tenant needs a different VXLAN UDP 
> >>> destination port/receive port. There might be SW elements that 
> >>> could do some of this, but IETF defines solutions which should be 
> >>> implemented across the board HW/SW/etc.
> >>> Even if some SW switches can do this, the proposal will impose so 
> >>> many issues in HW/data-plane engines that I cannot be behind this 
> >>> solution. To make this work generically we will have to make 
> >>> changes anyhow. Given this, we better do it in the right way and 
> >>> guide the industry to a solution which does not imply those complexities.
> >>> Otherwise we will stick with these specials forever with all 
> >>> consequences (bugs, etc). - snip - From:
> >>> "[e-mail]
> >>>
> <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>" <[e-mail]
> >>>
> <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>> Organization: Orange Date:
> >>> Tuesday 17 November 2015 at 01:37 To: Wim Henderickx 
> >>> <[e-mail]
> >>> <mailto:wim.henderickx@alcatel-
> lucent.com><mailto:wim.henderickx@alc
> >>> atel-lucent.com
> >>>
> >>>
> <mailto:[e-mail]>>>, BESS <[e-mail]
> >>> <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>> Subject: Re: [bess] 
> >>> draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, WG, 2015-11-16, 
> >>> Henderickx, Wim (Wim): 2015-11-13, Henderickx, Wim (Wim): Thomas,
> we
> >>> can discuss forever and someone need to describe requirements, but 
> >>> the current proposal I cannot agree to for the reasons explained.
> >>> TM> Well, although discussing forever is certainly not the goal, 
> >>> TM> the
> >>> reasons for rejecting a proposal need to be thoroughly understood.
> >>> WH> my point is what is the real driver for supporting a plain 
> >>> WH> VXLAN
> >>> data-plane here, the use cases I have seen in this txt is always 
> >>> where an application behind a NVE/TOR is demanding option c, but 
> >>> none of the NVE/TOR elements.
> >>> My understanding is that the applications  are contexts where 
> >>> overlays are present is when workloads (VMs or baremetal) need to 
> >>> be interconnected with VPNs. In these contexts, there can be 
> >>> reasons to want Option C to reduce the state on ASBRs. In these 
> >>> context, its not the workload (VM or baremetal) that would 
> >>> typically handle VRFs, but really the vswitch or ToR. WH2> can it not be all cases:
> >>> TOR/vswitch/Application. I would make the solution flexible to 
> >>> support all of these not? 2015-11-13, Henderickx, Wim (Wim): TM> 
> >>> The right trade-off to make may in fact depend on whether you prefer:
> >>> (a) a new dataplane stitching behavior on DC ASBRs (the behavior 
> >>> specified in this draft) or
> >>> (b) an evolution of the encaps on the vswitches and ToRs to 
> >>> support MPLS/MPLS/(UDP or GRE) WH> b depends on the use case I 
> >>> don't get what you mean by "b depends on the use case". WH> see my 
> >>> above comment. If the real use case is an application behind 
> >>> NVE/TOR requiring model C, than all the discussion on impact on 
> >>> NVE/TOR is void. As such I want to have a discussion on the real 
> >>> driver/requirement for option c interworking with an IP based 
> >>> Fabric. Although I can agree than detailing requirements can 
> >>> always help, I don't think one can assume a certain application to 
> >>> dismiss the proposal. WH> for me the proposal is not acceptable 
> >>> for the reasons explained: too much impact on the data-planes I 
> >>> wrote the above based on the idea that the encap used in 
> >>> MPLS/MPLS/(UDP or GRE), which hence has to be supported on the ToRs and vswitches.
> >>> Another possibility would be service-label/middle-label/Ethernet
> >>> assuming an L2 adjacency between vswitches/ToRs and ASBRs, but 
> >>> this certainly does not match your typical DC architecture. Or 
> >>> perhaps had you something else in mind ? WH> see above. The draft 
> >>> right now also requires changes in existing TOR/NVE so for me all 
> >>> this discussion/debate is void. No, the spec as it is can be 
> >>> implemented in its VXLAN variant with existing vswitches (e.g. OVS 
> >>> allows to choose the VXLAN destination port, ditto for the linux 
> >>> kernel stack). (ToR is certainly another story, most of them not 
> >>> having a flexible enough VXLAN dataplane nor support for any 
> >>> MPLS-over-IP.)
> >>> WH> and how many ports simultaneously would they support? WH> and
> >>> depending on implementation you don’t need to change any of the 
> >>> TOR/vswitches. Does this mean that for some implementations you 
> >>> may not need to change any of the TOR/vswitches, but that for some 
> >>> others you may ? WH> any proposal on the table requires changes, 
> >>> so for me this is not a valid discussion See above, the proposal 
> >>> in the draft does not necessarily need changes in vswitches. Let 
> >>> me take a practical example : while I can quite easily see how to 
> >>> implement the procedures in draft-hao-bess-inter-nvo3-vpn-optionc 
> >>> based on current vswitch implementations of VXLAN, the lack of 
> >>> MPLS/MPLS/(UDP, GRE) support in commonplace vswitches seems to
> me as
> >>> making that alternate solution you suggest harder to implement. 
> >>> WH> I would disagree to this. Tell me which switch/TOR handles 
> >>> multiple UDP ports for VXLAN ? I mentioned _v_switches, and many 
> >>> do support a variable destination port for VXLAN, which is 
> >>> sufficient to implement what the draft proposes. -Thomas From: 
> >>> Thomas Morin <[e-mail]
> >>>
> <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>> Organization: Orange Date:
> >>> Friday 13 November 2015 at 09:57 To: Wim Henderickx 
> >>> <[e-mail]
> >>> <mailto:wim.henderickx@alcatel-
> lucent.com><mailto:wim.henderickx@alc
> >>> atel-lucent.com
> >>>
> >>>
> <mailto:[e-mail]>>>
> >>> Cc: "[e-mail] <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>" <[e-mail] 
> >>> <mailto:[e-mail]><mailto:[e-mail]
> >>> <mailto:[e-mail]>>> Subject: Re: [bess] 
> >>> draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, I agree on the 
> >>> analysis that this proposal is restricted to implementations that 
> >>> supports the chosen encap with non-IANA ports (which may be hard 
> >>> to achieve for instance on hardware implementations, as you 
> >>> suggest), or to context where managing multiple IPs would be 
> >>> operationally viable. However, it does not seem obvious to me how 
> >>> the alternative you propose [relying on 3-label option C with an
> >>> MPLS/MPLS/(UDP|GRE) encap] addresses the issue of whether the
> encap
> >>> behavior is supported or not (e.g. your typical ToR chipset 
> >>> possibly may not support this kind of encap,  and even 
> >>> software-based switches may not be ready to support that today).
> >>> My take is that having different options to adapt to various 
> >>> implementations constraints we may have would have value. (+ one 
> >>> question below on VXLAN...) -Thomas 2015-11-12, Henderickx, Wim
> >>> (Wim): On VXLAN/NVGRE, do you challenge the fact that they would 
> >>> be used with a dummy MAC address that would be replaced by the 
> >>> right MAC by a sender based on an ARP request when needed ? Is the 
> >>> above the issue you had in mind about VXLAN and NVGRE ? WH> yes I 
> >>> you don't mind me asking : why do you challenge that ?
> >>>
> 
> __________________________________________________________
> __________________________________________________________
> _____
> 
> Ce message et ses pieces jointes peuvent contenir des informations 
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, 
> exploites ou copies sans autorisation. Si vous avez recu ce message 
> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi 
> que les pieces jointes. Les messages electroniques etant susceptibles 
> d'alteration, France Telecom - Orange decline toute responsabilite si 
> ce message a ete altere, deforme ou falsifie. Merci
> 
> This message and its attachments may contain confidential or 
> privileged information that may be protected by law; they should not 
> be distributed, used or copied without authorization.
> If you have received this email in error, please notify the sender and 
> delete this message and its attachments.
> As emails may be altered, France Telecom - Orange shall not be liable 
> if this message was modified, changed or falsified.
> Thank you.
> 
> _______________________________________________
> BESS mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/bess
> _______________________________________________
> BESS mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/bess