Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

["Rabadan, Jorge (Jorge)" <[e-mail]>]

IMHO if TOR chip vendors can confirm they are seriously looking at MPLS/MPLS/UDP, Wim’s suggestion makes all the sense since we know it works and scales.
My 2 cents.

Jorge



On 11/20/15, 9:51 AM, "BESS on behalf of UTTARO, JAMES" <[e-mail] on behalf of [e-mail]> wrote:

>+1
>
>-----Original Message-----
>From: BESS [mailto:[e-mail]] On Behalf Of John E Drake
>Sent: Friday, November 20, 2015 12:19 PM
>To: EXT - [e-mail]; Lucy yong; Henderickx, Wim (Wim); [e-mail]
>Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>
>Lucy,
>
>My apologies, I misunderstood.
>
>I think we have to accept the fact that we will have to deal with a multiplicity of different encapsulations in the data plane along a packet's e2e path and that we should take a more measured approach to deciding how to deal with this in a general and extensible way before accepting any solutions. 
>  
>Yours Irrespectively,
>
>John
>
>> -----Original Message-----
>> From: [e-mail] [mailto:[e-mail]]
>> Sent: Friday, November 20, 2015 12:04 PM
>> To: John E Drake; Lucy yong; Henderickx, Wim (Wim); [e-mail]
>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>> 
>> 2015-11-20, John E Drake:
>> > That presupposes that the group likes either of the two proposed solutions
>> in your draft.
>> 
>> John, I think Lucy's "two solutions" was referring to draft-hao-bess-inter-
>> nvo3-vpn-optionc solution and the 3-label Optionc MPLS/MPLS/UDP solution
>> described by Wim.
>> 
>> -Thomas
>> 
>> 
>> 
>> >
>> >> -----Original Message-----
>> >> From: BESS [mailto:[e-mail]] On Behalf Of Lucy yong
>> >> Sent: Friday, November 20, 2015 11:49 AM
>> >> To: EXT - [e-mail]; Henderickx, Wim (Wim);
>> >> [e-mail]
>> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>> >>
>> >> Share my 2 cent.
>> >>
>> >> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
>> >> Option C is a way to realize it. Both solutions summarized by Thomas
>> >> have no change on WAN VPN side and seamlessly work with WAN VPN
>> option C.
>> >> However, to support either solution, DC has to do some enhancement on
>> >> DC BR or ToR switch, etc, which dictates to different implementations
>> >> within a DC. Option C pro and con remains regardless what
>> >> implementation is used in a DC.
>> >>
>> >> Two solutions should not coexist in one DC (does not make a sense),
>> >> but it does not matter if one DC operator prefers to use one solution
>> >> and another DC prefers to use another solution. Since there are many
>> >> cloud providers today, it is worth for the WG to document both
>> >> solutions and point out the implementation requirements on impacted
>> >> components. Then, up to vendors and operators to select a solution for
>> their DC.
>> >>
>> >> It does not make a sense for us to vote which solution is better here
>> >> because a better solution for a DC depends on many factors.
>> >>
>> >>
>> >> Lucy
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: BESS [mailto:[e-mail]] On Behalf Of
>> >> [e-mail]
>> >> Sent: Friday, November 20, 2015 3:02 AM
>> >> To: Henderickx, Wim (Wim); [e-mail]
>> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>> >>
>> >> 2015-11-19, Henderickx, Wim (Wim):
>> >>> WH> I vote for a an evolution of switches/TORs that have proper
>> >>> support for this. I hope some HW vendors of TOR chips shime in, but
>> >>> I am told the MPLS solution is possible in the next generation chips
>> >>> they are working on.
>> >>
>> >> Well, it looks like the key questions are:
>> >> - when would ToR chips support MPLS/MPLS/UDP ?  (the generation that
>> >> has been released recently but not present in most shipping ToRs yet,
>> >> the next one ?)
>> >> - do we want an interim VXLAN-based solution ? (that will involve at
>> >> best a performance penalty with existing chips, and at worse
>> >> impossible to implement -- we haven't seen clear information in this
>> >> thread)
>> >>
>> >> -Thomas
>> >>
>> >>
>> >>>> Zhuangshunwan  :
>> >>>>>
>> >>>>> Hi Diego,
>> >>>>>
>> >>>>> Thanks for your comments. Pls see inline with [Vincent].
>> >>>>>
>> >>>>> Vincent
>> >>>>>
>> >>>>> *发件人:*BESS [mailto:[e-mail]] *代表 *Diego Garcia
>> >> del Rio
>> >>>>> *发送时间:*2015年11月18日14:25 *收件人:*[e-mail] *主题
>> >> :*Re: [bess]
>> >>>>> draft-hao-bess-inter-nvo3-vpn-optionc
>> >>>>>
>> >>>>> Some comments from my side, I think the draft makes quite a few
>> >>>>> assumptions on specific implementation details that are way too
>> >>>>> general to be considered widely available. Most of the TOR devices
>> >>>>> today already pay a double-pass penalty when doing routing of
>> >>>>> traffic in/out of vxlan-type tunnels. Only the newest generation
>> >>>>> can route into tunnels without additional passes. And are
>> >>>>> definitively limited in being able to arbitrary select UDP ports
>> >>>>> on a per tunnel basis. In fact, most are even limited at using
>> >>>>> more than one VNID per "service" of sorts. [Vincent]: Yes, the new
>> >>>>> generation BCM chipset has already supported VXLAN routing
>> without
>> >>>>> additional passes. For OVS/TOR, VXLAN encapsulation is more
>> >>>>> popular than MPLSoGRE/UDP, and has better performance. The
>> >>>>> IP-addressed based implementation which would, I assume, imply
>> >>>>> assigning one or more CIDRs to a loopback interface on the ASBR-d
>> >>>>> is also quite arbitrary and does not seem like a technically
>> >>>>> "clean" solution. (besides burning tons of IPs). As a side-note,
>> >>>>> most PE-grade routers i've worked with were quite limited in terms
>> >>>>> of IP addresses used for tunnel termination and it wasn't that
>> >>>>> just a simple pool can be used. [Vincent]: I think the larger VTEP
>> >>>>> IP address range on ASBR-d has no limitations.
>> >>>>> For the hardware on ASBR-d, it has capability to terminate
>> >>>>> multiple VXLAN tunnels with arbitrary destination VTEP IP
>> >>>>> addresses. Wim's mentions on cases where the Application itself,
>> >>>>> hosted in a datacenter, would be part of the option-C
>> >>>>> interconnect, was dismissed without much discussion so far, while,
>> >>>>> if we look in detail at the type of users which will even consider
>> >>>>> a complex topology like model-C its most likely users and
>> >>>>> operators very familiar with MPLS VPNs in the WAN. Those type of
>> >>>>> operators will most likely be very interested in deploying MPLS or
>> >>>>> WAN-grade applications (i.e., virtual-routers or other
>> >>>>> VNFs) in the DC and thus its highly likely that the interconnect
>> >>>>> would not terminate at the NVE itself but rather the TS (the
>> >>>>> virtual machine). Also, the use of UDP ports at random would imply
>> >>>>> quite complex logic on the ASBR-d IMHO. Im not saying its
>> >>>>> impossible, but since the received packet now not only has to be
>> >>>>> received on a random (though locally chosen) UDP port and this
>> >>>>> information preserved in the pipeline to be able to do the
>> >>>>> double-tunnel-stitching, it sounds quite complex. I am sure
>> >>>>> someone in the list will mention this has already been implemented
>> >>>>> somewhere, and I won't argue with that. And let's not even bring
>> >>>>> into account what this would do to any DC middlebox that now has
>> >>>>> to look at vxlan over almost any random port. We have to go back
>> >>>>> to the "is it a 4 or is it a 6 in byte x" heuristics to try to
>> >>>>> guess whether the packet is vxlan or just something entirely
>> >>>>> different running over IP. [Vincent]: Using NP or multi-core CPU
>> >>>>> hardware technology, it can be implemented although deeper packet
>> >>>>> inspection is needed to perform UDP port and MPLS stitching. In
>> >>>>> general I feel the proposed solution seems to be fitting of a
>> >>>>> specific use-case which is not really detailed
>> >>>>> in the draft and does not describe   a solution that would
>> >>>>> "elegantly" solve the issues at hand. It just feels like we're
>> >>>>> using any available bit-space to stuff data into places that do
>> >>>>> not necesarily belong. Yes, MPLS encapsulations on virtual
>> >>>>> switches are not yet fully available, and there can be some
>> >>>>> performance penalty on the TORs, but the solutions are much
>> >>>>> cleaner from a control and data plane point of view. Maybe I'm too
>> >>>>> utopic. [Vincent]: I think pure VXLAN solution has its scenario,
>> >>>>> it's general rather than specific. We can't require all OVS/NVEs
>> >>>>> support VXLAN + MPLSoGRE at the same time. Best regards, Diego
>> >>>>> ------------------------------------------------------------------
>> >>>>> --
>> >>>>> -------------
>> >>>>>
>> >>>>>
>> >> Hi,
>> >>>>> The problem we are trying to solve is to reduce data center
>> >>>>> GW/ASBR-d's forwarding table size, the motivation is same as
>> >>>>> traditional MPLS VPN option-C. Currently, the most common practise
>> >>>>> on ASBR-d is to terminate VXLAN encapsulation, look up local
>> >>>>> routing table, and then perform MPLS encapsulation to the WAN
>> network.
>> >>>>> ASBR-d needs to maintain all VM's MAC/IP. In Option-C method, only
>> >>>>> transport layer information needed to be maintained at GW/ASBR-d,
>> >>>>> the scalability will be greatly enhanced. Traditonal Option-C is
>> >>>>> only for MPLS VPN network interworking, because VXLAN is
>> becoming
>> >>>>> pervasive in data center, the solution in this draft was proposed
>> >>>>> for the heterogeneous network interworking. The advantage of this
>> >>>>> solution is that only VXLAN encapsulation is required for OVS/TOR.
>> >>>>> Unlike Wim's solution, east-west bound traffic uses VXLAN encap,
>> >>>>> while north-south bound traffic uses MPLSoGRE/UDP encap. There
>> are
>> >>>>> two solutions in this draft: 1. Using VXLAN tunnel destination IP
>> >>>>> for stitching at ASBR-d. No data plane modification requirements
>> >>>>> on OVS or TOR switches, only hardware changes on ASBR-d. ASBR-d
>> >>>>> normally is router, it has capability to realize the hardware
>> >>>>> changes. It will consume many IP addresses and the IP pool for
>> >>>>> allocation needs to be configured on ASBR-d beforehand. 2. Using
>> >>>>> VXLAN destination UDP port for stitching at ASBR-d. Compared with
>> >>>>> solution 1, less IP address will be consumed for allocation. If
>> >>>>> UDP port range is too large, we can combine with solution 1 and 2.
>> >>>>> In this solution, both data plane modification changes are needed
>> >>>>> at OVS/TOR and ASBR-d. ASBR-d also has capability to realize the
>> >>>>> hardware changes. For OVS, it also can realize the data plane
>> >>>>> changes. For TOR switch, it normally can't realize this function.
>> >>>>> This solution mainly focuses on pure software based overlay
>> >>>>> network, it has more scalability. In public cloud data center,
>> >>>>> software based overlay network is the majority case. Whether using
>> >>>>> solution 1 or 2 depends on the operators real envionment. So I
>> >>>>> think our solution has no flaws, it works fine.
>> >>>>> Thanks, weiguo ________________________________ From: BESS
>> >>>>> [[e-mail] <mailto:[e-mail]>] on behalf
>> >>>>> of John E Drake [[e-mail] <mailto:[e-mail]>]
>> >>>>> Sent: Wednesday, November 18, 2015 2:49 To: Henderickx, Wim
>> (Wim);
>> >>>>> EXT - [e-mail]
>> <mailto:[e-mail]>;
>> >> BESS
>> >>>>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc Hi, I
>> >>>>> think Wim has conclusively demonstrated that this draft has fatal
>> >>>>> flaws and I don’t support it.  I also agree with his suggestion
>> >>>>> that we first figure out what problem we are trying to solve
>> >>>>> before solving it. Yours Irrespectively, John From: BESS
>> >>>>> [mailto:[e-mail] <mailto:[e-mail]>] On
>> >>>>> Behalf Of Henderickx, Wim (Wim) Sent: Tuesday, November 17, 2015
>> >>>>> 12:49 PM To: EXT - [e-mail]
>> >>>>> <mailto:[e-mail]>; BESS Subject: Re: [bess]
>> >>>>> draft-hao-bess-inter-nvo3-vpn-optionc — Snip — No, the spec as it
>> >>>>> is can be implemented in its VXLAN variant with existing vswitches
>> >>>>> (e.g. OVS allows to choose the VXLAN destination port, ditto for
>> >>>>> the linux kernel stack). (ToR is certainly another story, most of
>> >>>>> them not having a flexible enough VXLAN dataplane nor support for
>> >>>>> any
>> >>>>> MPLS-over-IP.) WH> and how many ports simultaneously would they
>> >>>>> support? For this to work every tenant needs a different VXLAN UDP
>> >>>>> destination port/receive port. There might be SW elements that
>> >>>>> could do some of this, but IETF defines solutions which should be
>> >>>>> implemented across the board HW/SW/etc.
>> >>>>> Even if some SW switches can do this, the proposal will impose so
>> >>>>> many issues in HW/data-plane engines that I cannot be behind this
>> >>>>> solution. To make this work generically we will have to make
>> >>>>> changes anyhow. Given this, we better do it in the right way and
>> >>>>> guide the industry to a solution which does not imply those
>> complexities.
>> >>>>> Otherwise we will stick with these specials forever with all
>> >>>>> consequences (bugs, etc). - snip - From:
>> >>>>> "[e-mail]
>> >>>>>
>> >>
>> <mailto:[e-mail]><mailto:[e-mail]
>> >>>>> <mailto:[e-mail]>>" <[e-mail]
>> >>>>>
>> >>
>> <mailto:[e-mail]><mailto:[e-mail]
>> >>>>> <mailto:[e-mail]>>> Organization: Orange Date:
>> >>>>> Tuesday 17 November 2015 at 01:37 To: Wim Henderickx
>> >>>>> <[e-mail]
>> >>>>> <mailto:wim.henderickx@alcatel-
>> >> lucent.com><mailto:wim.henderickx@alc
>> >>>>> atel-lucent.com
>> >>>>>
>> >>>>>
>> >> <mailto:[e-mail]>>>, BESS <[e-mail]
>> >>>>> <mailto:[e-mail]><mailto:[e-mail]
>> >>>>> <mailto:[e-mail]>>> Subject: Re: [bess]
>> >>>>> draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, WG, 2015-11-16,
>> >>>>> Henderickx, Wim (Wim): 2015-11-13, Henderickx, Wim (Wim):
>> Thomas,
>> >> we
>> >>>>> can discuss forever and someone need to describe requirements, but
>> >>>>> the current proposal I cannot agree to for the reasons explained.
>> >>>>> TM> Well, although discussing forever is certainly not the goal,
>> >>>>> TM> the
>> >>>>> reasons for rejecting a proposal need to be thoroughly understood.
>> >>>>> WH> my point is what is the real driver for supporting a plain
>> >>>>> WH> VXLAN
>> >>>>> data-plane here, the use cases I have seen in this txt is always
>> >>>>> where an application behind a NVE/TOR is demanding option c, but
>> >>>>> none of the NVE/TOR elements.
>> >>>>> My understanding is that the applications  are contexts where
>> >>>>> overlays are present is when workloads (VMs or baremetal) need to
>> >>>>> be interconnected with VPNs. In these contexts, there can be
>> >>>>> reasons to want Option C to reduce the state on ASBRs. In these
>> >>>>> context, its not the workload (VM or baremetal) that would
>> >>>>> typically handle VRFs, but really the vswitch or ToR. WH2> can it not
>> be all cases:
>> >>>>> TOR/vswitch/Application. I would make the solution flexible to
>> >>>>> support all of these not? 2015-11-13, Henderickx, Wim (Wim): TM>
>> >>>>> The right trade-off to make may in fact depend on whether you
>> prefer:
>> >>>>> (a) a new dataplane stitching behavior on DC ASBRs (the behavior
>> >>>>> specified in this draft) or
>> >>>>> (b) an evolution of the encaps on the vswitches and ToRs to
>> >>>>> support MPLS/MPLS/(UDP or GRE) WH> b depends on the use case I
>> >>>>> don't get what you mean by "b depends on the use case". WH> see
>> my
>> >>>>> above comment. If the real use case is an application behind
>> >>>>> NVE/TOR requiring model C, than all the discussion on impact on
>> >>>>> NVE/TOR is void. As such I want to have a discussion on the real
>> >>>>> driver/requirement for option c interworking with an IP based
>> >>>>> Fabric. Although I can agree than detailing requirements can
>> >>>>> always help, I don't think one can assume a certain application to
>> >>>>> dismiss the proposal. WH> for me the proposal is not acceptable
>> >>>>> for the reasons explained: too much impact on the data-planes I
>> >>>>> wrote the above based on the idea that the encap used in
>> >>>>> MPLS/MPLS/(UDP or GRE), which hence has to be supported on the
>> ToRs and vswitches.
>> >>>>> Another possibility would be service-label/middle-label/Ethernet
>> >>>>> assuming an L2 adjacency between vswitches/ToRs and ASBRs, but
>> >>>>> this certainly does not match your typical DC architecture. Or
>> >>>>> perhaps had you something else in mind ? WH> see above. The draft
>> >>>>> right now also requires changes in existing TOR/NVE so for me all
>> >>>>> this discussion/debate is void. No, the spec as it is can be
>> >>>>> implemented in its VXLAN variant with existing vswitches (e.g. OVS
>> >>>>> allows to choose the VXLAN destination port, ditto for the linux
>> >>>>> kernel stack). (ToR is certainly another story, most of them not
>> >>>>> having a flexible enough VXLAN dataplane nor support for any
>> >>>>> MPLS-over-IP.)
>> >>>>> WH> and how many ports simultaneously would they support? WH>
>> and
>> >>>>> depending on implementation you don’t need to change any of the
>> >>>>> TOR/vswitches. Does this mean that for some implementations you
>> >>>>> may not need to change any of the TOR/vswitches, but that for some
>> >>>>> others you may ? WH> any proposal on the table requires changes,
>> >>>>> so for me this is not a valid discussion See above, the proposal
>> >>>>> in the draft does not necessarily need changes in vswitches. Let
>> >>>>> me take a practical example : while I can quite easily see how to
>> >>>>> implement the procedures in draft-hao-bess-inter-nvo3-vpn-optionc
>> >>>>> based on current vswitch implementations of VXLAN, the lack of
>> >>>>> MPLS/MPLS/(UDP, GRE) support in commonplace vswitches seems to
>> >> me as
>> >>>>> making that alternate solution you suggest harder to implement.
>> >>>>> WH> I would disagree to this. Tell me which switch/TOR handles
>> >>>>> multiple UDP ports for VXLAN ? I mentioned _v_switches, and many
>> >>>>> do support a variable destination port for VXLAN, which is
>> >>>>> sufficient to implement what the draft proposes. -Thomas From:
>> >>>>> Thomas Morin <[e-mail]
>> >>>>>
>> >>
>> <mailto:[e-mail]><mailto:[e-mail]
>> >>>>> <mailto:[e-mail]>>> Organization: Orange Date:
>> >>>>> Friday 13 November 2015 at 09:57 To: Wim Henderickx
>> >>>>> <[e-mail]
>> >>>>> <mailto:wim.henderickx@alcatel-
>> >> lucent.com><mailto:wim.henderickx@alc
>> >>>>> atel-lucent.com
>> >>>>>
>> >>>>>
>> >> <mailto:[e-mail]>>>
>> >>>>> Cc: "[e-mail] <mailto:[e-mail]><mailto:[e-mail]
>> >>>>> <mailto:[e-mail]>>" <[e-mail]
>> >>>>> <mailto:[e-mail]><mailto:[e-mail]
>> >>>>> <mailto:[e-mail]>>> Subject: Re: [bess]
>> >>>>> draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, I agree on the
>> >>>>> analysis that this proposal is restricted to implementations that
>> >>>>> supports the chosen encap with non-IANA ports (which may be hard
>> >>>>> to achieve for instance on hardware implementations, as you
>> >>>>> suggest), or to context where managing multiple IPs would be
>> >>>>> operationally viable. However, it does not seem obvious to me how
>> >>>>> the alternative you propose [relying on 3-label option C with an
>> >>>>> MPLS/MPLS/(UDP|GRE) encap] addresses the issue of whether the
>> >> encap
>> >>>>> behavior is supported or not (e.g. your typical ToR chipset
>> >>>>> possibly may not support this kind of encap,  and even
>> >>>>> software-based switches may not be ready to support that today).
>> >>>>> My take is that having different options to adapt to various
>> >>>>> implementations constraints we may have would have value. (+ one
>> >>>>> question below on VXLAN...) -Thomas 2015-11-12, Henderickx, Wim
>> >>>>> (Wim): On VXLAN/NVGRE, do you challenge the fact that they would
>> >>>>> be used with a dummy MAC address that would be replaced by the
>> >>>>> right MAC by a sender based on an ARP request when needed ? Is the
>> >>>>> above the issue you had in mind about VXLAN and NVGRE ? WH> yes I
>> >>>>> you don't mind me asking : why do you challenge that ?
>> >>>>>
>> >>
>> >>
>> __________________________________________________________
>> >>
>> __________________________________________________________
>> >> _____
>> >>
>> >> Ce message et ses pieces jointes peuvent contenir des informations
>> >> confidentielles ou privilegiees et ne doivent donc pas etre diffuses,
>> >> exploites ou copies sans autorisation. Si vous avez recu ce message
>> >> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi
>> >> que les pieces jointes. Les messages electroniques etant susceptibles
>> >> d'alteration, France Telecom - Orange decline toute responsabilite si
>> >> ce message a ete altere, deforme ou falsifie. Merci
>> >>
>> >> This message and its attachments may contain confidential or
>> >> privileged information that may be protected by law; they should not
>> >> be distributed, used or copied without authorization.
>> >> If you have received this email in error, please notify the sender
>> >> and delete this message and its attachments.
>> >> As emails may be altered, France Telecom - Orange shall not be liable
>> >> if this message was modified, changed or falsified.
>> >> Thank you.
>> >>
>> >> _______________________________________________
>> >> BESS mailing list
>> >> [e-mail]
>> >> https://www.ietf.org/mailman/listinfo/bess
>> >> _______________________________________________
>> >> BESS mailing list
>> >> [e-mail]
>> >> https://www.ietf.org/mailman/listinfo/bess
>> 
>> 
>> __________________________________________________________
>> __________________________________________________________
>> _____
>> 
>> Ce message et ses pieces jointes peuvent contenir des informations
>> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites
>> ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez
>> le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les
>> messages electroniques etant susceptibles d'alteration, France Telecom -
>> Orange decline toute responsabilite si ce message a ete altere, deforme ou
>> falsifie. Merci
>> 
>> This message and its attachments may contain confidential or privileged
>> information that may be protected by law; they should not be distributed,
>> used or copied without authorization.
>> If you have received this email in error, please notify the sender and delete
>> this message and its attachments.
>> As emails may be altered, France Telecom - Orange shall not be liable if this
>> message was modified, changed or falsified.
>> Thank you.
>
>_______________________________________________
>BESS mailing list
>[e-mail]
>https://www.ietf.org/mailman/listinfo/bess
>_______________________________________________
>BESS mailing list
>[e-mail]
>https://www.ietf.org/mailman/listinfo/bess