Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

["UTTARO, JAMES" <[e-mail]>]

+1

-----Original Message-----
From: BESS [mailto:[e-mail]] On Behalf Of John E Drake
Sent: Friday, November 20, 2015 12:19 PM
To: EXT - [e-mail]; Lucy yong; Henderickx, Wim (Wim); [e-mail]
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Lucy,

My apologies, I misunderstood.

I think we have to accept the fact that we will have to deal with a multiplicity of different encapsulations in the data plane along a packet's e2e path and that we should take a more measured approach to deciding how to deal with this in a general and extensible way before accepting any solutions. 
  
Yours Irrespectively,

John

> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]]
> Sent: Friday, November 20, 2015 12:04 PM
> To: John E Drake; Lucy yong; Henderickx, Wim (Wim); [e-mail]
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> 
> 2015-11-20, John E Drake:
> > That presupposes that the group likes either of the two proposed solutions
> in your draft.
> 
> John, I think Lucy's "two solutions" was referring to draft-hao-bess-inter-
> nvo3-vpn-optionc solution and the 3-label Optionc MPLS/MPLS/UDP solution
> described by Wim.
> 
> -Thomas
> 
> 
> 
> >
> >> -----Original Message-----
> >> From: BESS [mailto:[e-mail]] On Behalf Of Lucy yong
> >> Sent: Friday, November 20, 2015 11:49 AM
> >> To: EXT - [e-mail]; Henderickx, Wim (Wim);
> >> [e-mail]
> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> >>
> >> Share my 2 cent.
> >>
> >> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
> >> Option C is a way to realize it. Both solutions summarized by Thomas
> >> have no change on WAN VPN side and seamlessly work with WAN VPN
> option C.
> >> However, to support either solution, DC has to do some enhancement on
> >> DC BR or ToR switch, etc, which dictates to different implementations
> >> within a DC. Option C pro and con remains regardless what
> >> implementation is used in a DC.
> >>
> >> Two solutions should not coexist in one DC (does not make a sense),
> >> but it does not matter if one DC operator prefers to use one solution
> >> and another DC prefers to use another solution. Since there are many
> >> cloud providers today, it is worth for the WG to document both
> >> solutions and point out the implementation requirements on impacted
> >> components. Then, up to vendors and operators to select a solution for
> their DC.
> >>
> >> It does not make a sense for us to vote which solution is better here
> >> because a better solution for a DC depends on many factors.
> >>
> >>
> >> Lucy
> >>
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: BESS [mailto:[e-mail]] On Behalf Of
> >> [e-mail]
> >> Sent: Friday, November 20, 2015 3:02 AM
> >> To: Henderickx, Wim (Wim); [e-mail]
> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> >>
> >> 2015-11-19, Henderickx, Wim (Wim):
> >>> WH> I vote for a an evolution of switches/TORs that have proper
> >>> support for this. I hope some HW vendors of TOR chips shime in, but
> >>> I am told the MPLS solution is possible in the next generation chips
> >>> they are working on.
> >>
> >> Well, it looks like the key questions are:
> >> - when would ToR chips support MPLS/MPLS/UDP ?  (the generation that
> >> has been released recently but not present in most shipping ToRs yet,
> >> the next one ?)
> >> - do we want an interim VXLAN-based solution ? (that will involve at
> >> best a performance penalty with existing chips, and at worse
> >> impossible to implement -- we haven't seen clear information in this
> >> thread)
> >>
> >> -Thomas
> >>
> >>
> >>>> Zhuangshunwan  :
> >>>>>
> >>>>> Hi Diego,
> >>>>>
> >>>>> Thanks for your comments. Pls see inline with [Vincent].
> >>>>>
> >>>>> Vincent
> >>>>>
> >>>>> *发件人:*BESS [mailto:[e-mail]] *代表 *Diego Garcia
> >> del Rio
> >>>>> *发送时间:*2015年11月18日14:25 *收件人:*[e-mail] *主题
> >> :*Re: [bess]
> >>>>> draft-hao-bess-inter-nvo3-vpn-optionc
> >>>>>
> >>>>> Some comments from my side, I think the draft makes quite a few
> >>>>> assumptions on specific implementation details that are way too
> >>>>> general to be considered widely available. Most of the TOR devices
> >>>>> today already pay a double-pass penalty when doing routing of
> >>>>> traffic in/out of vxlan-type tunnels. Only the newest generation
> >>>>> can route into tunnels without additional passes. And are
> >>>>> definitively limited in being able to arbitrary select UDP ports
> >>>>> on a per tunnel basis. In fact, most are even limited at using
> >>>>> more than one VNID per "service" of sorts. [Vincent]: Yes, the new
> >>>>> generation BCM chipset has already supported VXLAN routing
> without
> >>>>> additional passes. For OVS/TOR, VXLAN encapsulation is more
> >>>>> popular than MPLSoGRE/UDP, and has better performance. The
> >>>>> IP-addressed based implementation which would, I assume, imply
> >>>>> assigning one or more CIDRs to a loopback interface on the ASBR-d
> >>>>> is also quite arbitrary and does not seem like a technically
> >>>>> "clean" solution. (besides burning tons of IPs). As a side-note,
> >>>>> most PE-grade routers i've worked with were quite limited in terms
> >>>>> of IP addresses used for tunnel termination and it wasn't that
> >>>>> just a simple pool can be used. [Vincent]: I think the larger VTEP
> >>>>> IP address range on ASBR-d has no limitations.
> >>>>> For the hardware on ASBR-d, it has capability to terminate
> >>>>> multiple VXLAN tunnels with arbitrary destination VTEP IP
> >>>>> addresses. Wim's mentions on cases where the Application itself,
> >>>>> hosted in a datacenter, would be part of the option-C
> >>>>> interconnect, was dismissed without much discussion so far, while,
> >>>>> if we look in detail at the type of users which will even consider
> >>>>> a complex topology like model-C its most likely users and
> >>>>> operators very familiar with MPLS VPNs in the WAN. Those type of
> >>>>> operators will most likely be very interested in deploying MPLS or
> >>>>> WAN-grade applications (i.e., virtual-routers or other
> >>>>> VNFs) in the DC and thus its highly likely that the interconnect
> >>>>> would not terminate at the NVE itself but rather the TS (the
> >>>>> virtual machine). Also, the use of UDP ports at random would imply
> >>>>> quite complex logic on the ASBR-d IMHO. Im not saying its
> >>>>> impossible, but since the received packet now not only has to be
> >>>>> received on a random (though locally chosen) UDP port and this
> >>>>> information preserved in the pipeline to be able to do the
> >>>>> double-tunnel-stitching, it sounds quite complex. I am sure
> >>>>> someone in the list will mention this has already been implemented
> >>>>> somewhere, and I won't argue with that. And let's not even bring
> >>>>> into account what this would do to any DC middlebox that now has
> >>>>> to look at vxlan over almost any random port. We have to go back
> >>>>> to the "is it a 4 or is it a 6 in byte x" heuristics to try to
> >>>>> guess whether the packet is vxlan or just something entirely
> >>>>> different running over IP. [Vincent]: Using NP or multi-core CPU
> >>>>> hardware technology, it can be implemented although deeper packet
> >>>>> inspection is needed to perform UDP port and MPLS stitching. In
> >>>>> general I feel the proposed solution seems to be fitting of a
> >>>>> specific use-case which is not really detailed
> >>>>> in the draft and does not describe   a solution that would
> >>>>> "elegantly" solve the issues at hand. It just feels like we're
> >>>>> using any available bit-space to stuff data into places that do
> >>>>> not necesarily belong. Yes, MPLS encapsulations on virtual
> >>>>> switches are not yet fully available, and there can be some
> >>>>> performance penalty on the TORs, but the solutions are much
> >>>>> cleaner from a control and data plane point of view. Maybe I'm too
> >>>>> utopic. [Vincent]: I think pure VXLAN solution has its scenario,
> >>>>> it's general rather than specific. We can't require all OVS/NVEs
> >>>>> support VXLAN + MPLSoGRE at the same time. Best regards, Diego
> >>>>> ------------------------------------------------------------------
> >>>>> --
> >>>>> -------------
> >>>>>
> >>>>>
> >> Hi,
> >>>>> The problem we are trying to solve is to reduce data center
> >>>>> GW/ASBR-d's forwarding table size, the motivation is same as
> >>>>> traditional MPLS VPN option-C. Currently, the most common practise
> >>>>> on ASBR-d is to terminate VXLAN encapsulation, look up local
> >>>>> routing table, and then perform MPLS encapsulation to the WAN
> network.
> >>>>> ASBR-d needs to maintain all VM's MAC/IP. In Option-C method, only
> >>>>> transport layer information needed to be maintained at GW/ASBR-d,
> >>>>> the scalability will be greatly enhanced. Traditonal Option-C is
> >>>>> only for MPLS VPN network interworking, because VXLAN is
> becoming
> >>>>> pervasive in data center, the solution in this draft was proposed
> >>>>> for the heterogeneous network interworking. The advantage of this
> >>>>> solution is that only VXLAN encapsulation is required for OVS/TOR.
> >>>>> Unlike Wim's solution, east-west bound traffic uses VXLAN encap,
> >>>>> while north-south bound traffic uses MPLSoGRE/UDP encap. There
> are
> >>>>> two solutions in this draft: 1. Using VXLAN tunnel destination IP
> >>>>> for stitching at ASBR-d. No data plane modification requirements
> >>>>> on OVS or TOR switches, only hardware changes on ASBR-d. ASBR-d
> >>>>> normally is router, it has capability to realize the hardware
> >>>>> changes. It will consume many IP addresses and the IP pool for
> >>>>> allocation needs to be configured on ASBR-d beforehand. 2. Using
> >>>>> VXLAN destination UDP port for stitching at ASBR-d. Compared with
> >>>>> solution 1, less IP address will be consumed for allocation. If
> >>>>> UDP port range is too large, we can combine with solution 1 and 2.
> >>>>> In this solution, both data plane modification changes are needed
> >>>>> at OVS/TOR and ASBR-d. ASBR-d also has capability to realize the
> >>>>> hardware changes. For OVS, it also can realize the data plane
> >>>>> changes. For TOR switch, it normally can't realize this function.
> >>>>> This solution mainly focuses on pure software based overlay
> >>>>> network, it has more scalability. In public cloud data center,
> >>>>> software based overlay network is the majority case. Whether using
> >>>>> solution 1 or 2 depends on the operators real envionment. So I
> >>>>> think our solution has no flaws, it works fine.
> >>>>> Thanks, weiguo ________________________________ From: BESS
> >>>>> [[e-mail] <mailto:[e-mail]>] on behalf
> >>>>> of John E Drake [[e-mail] <mailto:[e-mail]>]
> >>>>> Sent: Wednesday, November 18, 2015 2:49 To: Henderickx, Wim
> (Wim);
> >>>>> EXT - [e-mail]
> <mailto:[e-mail]>;
> >> BESS
> >>>>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc Hi, I
> >>>>> think Wim has conclusively demonstrated that this draft has fatal
> >>>>> flaws and I don’t support it.  I also agree with his suggestion
> >>>>> that we first figure out what problem we are trying to solve
> >>>>> before solving it. Yours Irrespectively, John From: BESS
> >>>>> [mailto:[e-mail] <mailto:[e-mail]>] On
> >>>>> Behalf Of Henderickx, Wim (Wim) Sent: Tuesday, November 17, 2015
> >>>>> 12:49 PM To: EXT - [e-mail]
> >>>>> <mailto:[e-mail]>; BESS Subject: Re: [bess]
> >>>>> draft-hao-bess-inter-nvo3-vpn-optionc — Snip — No, the spec as it
> >>>>> is can be implemented in its VXLAN variant with existing vswitches
> >>>>> (e.g. OVS allows to choose the VXLAN destination port, ditto for
> >>>>> the linux kernel stack). (ToR is certainly another story, most of
> >>>>> them not having a flexible enough VXLAN dataplane nor support for
> >>>>> any
> >>>>> MPLS-over-IP.) WH> and how many ports simultaneously would they
> >>>>> support? For this to work every tenant needs a different VXLAN UDP
> >>>>> destination port/receive port. There might be SW elements that
> >>>>> could do some of this, but IETF defines solutions which should be
> >>>>> implemented across the board HW/SW/etc.
> >>>>> Even if some SW switches can do this, the proposal will impose so
> >>>>> many issues in HW/data-plane engines that I cannot be behind this
> >>>>> solution. To make this work generically we will have to make
> >>>>> changes anyhow. Given this, we better do it in the right way and
> >>>>> guide the industry to a solution which does not imply those
> complexities.
> >>>>> Otherwise we will stick with these specials forever with all
> >>>>> consequences (bugs, etc). - snip - From:
> >>>>> "[e-mail]
> >>>>>
> >>
> <mailto:[e-mail]><mailto:[e-mail]
> >>>>> <mailto:[e-mail]>>" <[e-mail]
> >>>>>
> >>
> <mailto:[e-mail]><mailto:[e-mail]
> >>>>> <mailto:[e-mail]>>> Organization: Orange Date:
> >>>>> Tuesday 17 November 2015 at 01:37 To: Wim Henderickx
> >>>>> <[e-mail]
> >>>>> <mailto:wim.henderickx@alcatel-
> >> lucent.com><mailto:wim.henderickx@alc
> >>>>> atel-lucent.com
> >>>>>
> >>>>>
> >> <mailto:[e-mail]>>>, BESS <[e-mail]
> >>>>> <mailto:[e-mail]><mailto:[e-mail]
> >>>>> <mailto:[e-mail]>>> Subject: Re: [bess]
> >>>>> draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, WG, 2015-11-16,
> >>>>> Henderickx, Wim (Wim): 2015-11-13, Henderickx, Wim (Wim):
> Thomas,
> >> we
> >>>>> can discuss forever and someone need to describe requirements, but
> >>>>> the current proposal I cannot agree to for the reasons explained.
> >>>>> TM> Well, although discussing forever is certainly not the goal,
> >>>>> TM> the
> >>>>> reasons for rejecting a proposal need to be thoroughly understood.
> >>>>> WH> my point is what is the real driver for supporting a plain
> >>>>> WH> VXLAN
> >>>>> data-plane here, the use cases I have seen in this txt is always
> >>>>> where an application behind a NVE/TOR is demanding option c, but
> >>>>> none of the NVE/TOR elements.
> >>>>> My understanding is that the applications  are contexts where
> >>>>> overlays are present is when workloads (VMs or baremetal) need to
> >>>>> be interconnected with VPNs. In these contexts, there can be
> >>>>> reasons to want Option C to reduce the state on ASBRs. In these
> >>>>> context, its not the workload (VM or baremetal) that would
> >>>>> typically handle VRFs, but really the vswitch or ToR. WH2> can it not
> be all cases:
> >>>>> TOR/vswitch/Application. I would make the solution flexible to
> >>>>> support all of these not? 2015-11-13, Henderickx, Wim (Wim): TM>
> >>>>> The right trade-off to make may in fact depend on whether you
> prefer:
> >>>>> (a) a new dataplane stitching behavior on DC ASBRs (the behavior
> >>>>> specified in this draft) or
> >>>>> (b) an evolution of the encaps on the vswitches and ToRs to
> >>>>> support MPLS/MPLS/(UDP or GRE) WH> b depends on the use case I
> >>>>> don't get what you mean by "b depends on the use case". WH> see
> my
> >>>>> above comment. If the real use case is an application behind
> >>>>> NVE/TOR requiring model C, than all the discussion on impact on
> >>>>> NVE/TOR is void. As such I want to have a discussion on the real
> >>>>> driver/requirement for option c interworking with an IP based
> >>>>> Fabric. Although I can agree than detailing requirements can
> >>>>> always help, I don't think one can assume a certain application to
> >>>>> dismiss the proposal. WH> for me the proposal is not acceptable
> >>>>> for the reasons explained: too much impact on the data-planes I
> >>>>> wrote the above based on the idea that the encap used in
> >>>>> MPLS/MPLS/(UDP or GRE), which hence has to be supported on the
> ToRs and vswitches.
> >>>>> Another possibility would be service-label/middle-label/Ethernet
> >>>>> assuming an L2 adjacency between vswitches/ToRs and ASBRs, but
> >>>>> this certainly does not match your typical DC architecture. Or
> >>>>> perhaps had you something else in mind ? WH> see above. The draft
> >>>>> right now also requires changes in existing TOR/NVE so for me all
> >>>>> this discussion/debate is void. No, the spec as it is can be
> >>>>> implemented in its VXLAN variant with existing vswitches (e.g. OVS
> >>>>> allows to choose the VXLAN destination port, ditto for the linux
> >>>>> kernel stack). (ToR is certainly another story, most of them not
> >>>>> having a flexible enough VXLAN dataplane nor support for any
> >>>>> MPLS-over-IP.)
> >>>>> WH> and how many ports simultaneously would they support? WH>
> and
> >>>>> depending on implementation you don’t need to change any of the
> >>>>> TOR/vswitches. Does this mean that for some implementations you
> >>>>> may not need to change any of the TOR/vswitches, but that for some
> >>>>> others you may ? WH> any proposal on the table requires changes,
> >>>>> so for me this is not a valid discussion See above, the proposal
> >>>>> in the draft does not necessarily need changes in vswitches. Let
> >>>>> me take a practical example : while I can quite easily see how to
> >>>>> implement the procedures in draft-hao-bess-inter-nvo3-vpn-optionc
> >>>>> based on current vswitch implementations of VXLAN, the lack of
> >>>>> MPLS/MPLS/(UDP, GRE) support in commonplace vswitches seems to
> >> me as
> >>>>> making that alternate solution you suggest harder to implement.
> >>>>> WH> I would disagree to this. Tell me which switch/TOR handles
> >>>>> multiple UDP ports for VXLAN ? I mentioned _v_switches, and many
> >>>>> do support a variable destination port for VXLAN, which is
> >>>>> sufficient to implement what the draft proposes. -Thomas From:
> >>>>> Thomas Morin <[e-mail]
> >>>>>
> >>
> <mailto:[e-mail]><mailto:[e-mail]
> >>>>> <mailto:[e-mail]>>> Organization: Orange Date:
> >>>>> Friday 13 November 2015 at 09:57 To: Wim Henderickx
> >>>>> <[e-mail]
> >>>>> <mailto:wim.henderickx@alcatel-
> >> lucent.com><mailto:wim.henderickx@alc
> >>>>> atel-lucent.com
> >>>>>
> >>>>>
> >> <mailto:[e-mail]>>>
> >>>>> Cc: "[e-mail] <mailto:[e-mail]><mailto:[e-mail]
> >>>>> <mailto:[e-mail]>>" <[e-mail]
> >>>>> <mailto:[e-mail]><mailto:[e-mail]
> >>>>> <mailto:[e-mail]>>> Subject: Re: [bess]
> >>>>> draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, I agree on the
> >>>>> analysis that this proposal is restricted to implementations that
> >>>>> supports the chosen encap with non-IANA ports (which may be hard
> >>>>> to achieve for instance on hardware implementations, as you
> >>>>> suggest), or to context where managing multiple IPs would be
> >>>>> operationally viable. However, it does not seem obvious to me how
> >>>>> the alternative you propose [relying on 3-label option C with an
> >>>>> MPLS/MPLS/(UDP|GRE) encap] addresses the issue of whether the
> >> encap
> >>>>> behavior is supported or not (e.g. your typical ToR chipset
> >>>>> possibly may not support this kind of encap,  and even
> >>>>> software-based switches may not be ready to support that today).
> >>>>> My take is that having different options to adapt to various
> >>>>> implementations constraints we may have would have value. (+ one
> >>>>> question below on VXLAN...) -Thomas 2015-11-12, Henderickx, Wim
> >>>>> (Wim): On VXLAN/NVGRE, do you challenge the fact that they would
> >>>>> be used with a dummy MAC address that would be replaced by the
> >>>>> right MAC by a sender based on an ARP request when needed ? Is the
> >>>>> above the issue you had in mind about VXLAN and NVGRE ? WH> yes I
> >>>>> you don't mind me asking : why do you challenge that ?
> >>>>>
> >>
> >>
> __________________________________________________________
> >>
> __________________________________________________________
> >> _____
> >>
> >> Ce message et ses pieces jointes peuvent contenir des informations
> >> confidentielles ou privilegiees et ne doivent donc pas etre diffuses,
> >> exploites ou copies sans autorisation. Si vous avez recu ce message
> >> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi
> >> que les pieces jointes. Les messages electroniques etant susceptibles
> >> d'alteration, France Telecom - Orange decline toute responsabilite si
> >> ce message a ete altere, deforme ou falsifie. Merci
> >>
> >> This message and its attachments may contain confidential or
> >> privileged information that may be protected by law; they should not
> >> be distributed, used or copied without authorization.
> >> If you have received this email in error, please notify the sender
> >> and delete this message and its attachments.
> >> As emails may be altered, France Telecom - Orange shall not be liable
> >> if this message was modified, changed or falsified.
> >> Thank you.
> >>
> >> _______________________________________________
> >> BESS mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/bess
> >> _______________________________________________
> >> BESS mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/bess
> 
> 
> __________________________________________________________
> __________________________________________________________
> _____
> 
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites
> ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez
> le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les
> messages electroniques etant susceptibles d'alteration, France Telecom -
> Orange decline toute responsabilite si ce message a ete altere, deforme ou
> falsifie. Merci
> 
> This message and its attachments may contain confidential or privileged
> information that may be protected by law; they should not be distributed,
> used or copied without authorization.
> If you have received this email in error, please notify the sender and delete
> this message and its attachments.
> As emails may be altered, France Telecom - Orange shall not be liable if this
> message was modified, changed or falsified.
> Thank you.

_______________________________________________
BESS mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/bess