Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

["UTTARO, JAMES" <[e-mail]>]

IMO an Option C model not CSC between to different administrative authorities where internal reachability state is exchanged is a non-starter.  But coming back to this draft it seems that tying the encap and the network architecture is a requirement or am I getting that wrong. If not, how should I interpret this draft? Should it be if I do Option C as I must have different AS domains between my internal DCs and WAN and my encap models are different in the WAN and DC, and I am using specific encaps in different places etcc... it would be in my solution space??

Jim Uttaro

From: Osama Zia [mailto:[e-mail]]
Sent: Thursday, November 19, 2015 2:38 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - [e-mail]; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

CIL

From: UTTARO, JAMES [mailto:[e-mail]]
Sent: Thursday, November 19, 2015 11:16 AM
To: Osama Zia <[e-mail]<mailto:[e-mail]>>; UTTARO, JAMES <[e-mail]<mailto:[e-mail]>>; Haoweiguo <[e-mail]<mailto:[e-mail]>>; John E Drake <[e-mail]<mailto:[e-mail]>>; Henderickx, Wim (Wim) <[e-mail]<mailto:[e-mail]>>; EXT - [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>; BESS <[e-mail]<mailto:[e-mail]>>
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Comments In-Line..

Jim Uttaro

From: Osama Zia [mailto:[e-mail]]
Sent: Thursday, November 19, 2015 2:06 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - [e-mail]<mailto:[e-mail]>; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

CIL

From: UTTARO, JAMES [mailto:[e-mail]]
Sent: Thursday, November 19, 2015 10:56 AM
To: Osama Zia <[e-mail]<mailto:[e-mail]>>; UTTARO, JAMES <[e-mail]<mailto:[e-mail]>>; Haoweiguo <[e-mail]<mailto:[e-mail]>>; John E Drake <[e-mail]<mailto:[e-mail]>>; Henderickx, Wim (Wim) <[e-mail]<mailto:[e-mail]>>; EXT - [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>; BESS <[e-mail]<mailto:[e-mail]>>
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The assumption being based on one's view of what a DC is ;) Yes when an SP and a Cloud Provider connect an inter-AS model of some form is needed.. It is doubtful that it would ever be Option C as the exposes a tremendous amount of internal information.
[OZ] I would beg to disagree here. Customer traffic can be tunneled to a customer router in the cloud that should only expose customer routes. I had argued previously in IETF that current practice of option A is not scalable.
[Jim U>] So, do you mean a CSC model where the NHs are the customer CEs, if so, then that is not Classical Option C where PE LBs are exchanged as the NHs for the customers path/routes.
[OZ] Sorry for the confusion. My comment was not specific to the draft. I was responding to your comment  "Option C as the exposes a tremendous amount of internal information"

At the end of the day network architecture/design whether internal/external to a provider is not mandated by the IETF or any standards body. This draft seems to mix encap, interconnect etc.... If the encap "requires" a specific connectivity model than I am not sure how that exactly works.

Jim Uttaro

From: Osama Zia [mailto:[e-mail]]
Sent: Thursday, November 19, 2015 1:51 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - [e-mail]<mailto:[e-mail]>; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Today many large enterprises connect to cloud providers using their existing MPLS VPN networks. This would mean different AS Numbers at WAN and DC sites.

Regards,
Osama

From: BESS [mailto:[e-mail]] On Behalf Of UTTARO, JAMES
Sent: Thursday, November 19, 2015 9:46 AM
To: Haoweiguo <[e-mail]<mailto:[e-mail]>>; John E Drake <[e-mail]<mailto:[e-mail]>>; Henderickx, Wim (Wim) <[e-mail]<mailto:[e-mail]>>; EXT - [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>; BESS <[e-mail]<mailto:[e-mail]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

A concern I have with this draft is that it pre-supposes the network architecture in terms of how DCs and WAN are connected. It assumes that DCs and WAN networks are in different AS domains. Why is that??

Jim Uttaro

From: BESS [mailto:[e-mail]] On Behalf Of Haoweiguo
Sent: Tuesday, November 17, 2015 11:24 PM
To: John E Drake; Henderickx, Wim (Wim); EXT - [e-mail]<mailto:[e-mail]>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc


Hi,

The problem we are trying to solve is to reduce data center GW/ASBR-d's forwarding table size, the motivation is same as traditional MPLS VPN option-C. Currently, the most common practise on ASBR-d is to terminate VXLAN encapsulation, look up local routing table, and then perform MPLS encapsulation to the WAN network. ASBR-d needs to maintain all VM's MAC/IP. In Option-C method, only transport layer information needed to be maintained at GW/ASBR-d, the scalability will be greatly enhanced. Traditonal Option-C is only for MPLS VPN network interworking, because VXLAN is becoming pervasive in data center,  the solution in this draft was proposed for the heterogeneous network interworking.

The advantage of this solution is that only VXLAN encapsulation is required for OVS/TOR. Unlike Wim's solution, east-west bound traffic uses VXLAN encap, while north-south bound traffic uses MPLSoGRE/UDP encap.

There are two solutions in this draft:

1. Using VXLAN tunnel destination IP for stitching at ASBR-d.

No data plane modification requirements on OVS or TOR switches, only hardware changes on ASBR-d. ASBR-d normally is router, it has capability to realize the hardware changes. It will consume many IP addresses and the IP pool for allocation needs to be configured on ASBR-d beforehand.

2. Using VXLAN destination UDP port for stitching at ASBR-d.

Compared with solution 1, less IP address will be consumed for allocation. If UDP port range is too large, we can combine with solution 1 and 2.

In this solution, both data plane modification changes are needed at OVS/TOR and ASBR-d. ASBR-d also has capability to realize the hardware changes. For OVS, it also can realize the data plane changes. For TOR switch, it normally can't realize this function.  This solution mainly focuses on pure software based overlay network, it has more scalability. In public cloud data center, software based overlay network is the majority case.



Whether using solution 1 or 2 depends on the operators real envionment.



So I think our solution has no flaws, it works fine.

Thanks,

weiguo



________________________________
From: BESS [[e-mail]] on behalf of John E Drake [[e-mail]]
Sent: Wednesday, November 18, 2015 2:49
To: Henderickx, Wim (Wim); EXT - [e-mail]<mailto:[e-mail]>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
Hi,

I think Wim has conclusively demonstrated that this draft has fatal flaws and I don't support it.  I also agree with his suggestion that we first figure out what problem we are trying to solve before solving it.

Yours Irrespectively,

John

From: BESS [mailto:[e-mail]] On Behalf Of Henderickx, Wim (Wim)
Sent: Tuesday, November 17, 2015 12:49 PM
To: EXT - [e-mail]<mailto:[e-mail]>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

- Snip -

No, the spec as it is can be implemented in its VXLAN variant with existing vswitches (e.g. OVS allows to choose the VXLAN destination port, ditto for the linux kernel stack).

(ToR is certainly another story, most of them not having a flexible enough VXLAN dataplane nor support for any MPLS-over-IP.)

WH> and how many ports simultaneously would they support? For this to work every tenant needs a different VXLAN UDP destination port/receive port.
There might be SW elements that could do some of this, but IETF defines solutions which should be implemented across the board HW/SW/etc. Even if some SW switches can do this, the proposal will impose so many issues in HW/data-plane engines that I cannot be behind this solution.

To make this work generically we will have to make changes anyhow. Given this, we better do it in the right way and guide the industry to a solution which does not imply those complexities. Otherwise we will stick with these specials forever with all consequences (bugs, etc).

- snip -

From: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Organization: Orange
Date: Tuesday 17 November 2015 at 01:37
To: Wim Henderickx <[e-mail]<mailto:[e-mail]>>, BESS <[e-mail]<mailto:[e-mail]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim, WG,

2015-11-16, Henderickx, Wim (Wim):

2015-11-13, Henderickx, Wim (Wim):
Thomas, we can discuss forever and someone need to describe requirements, but the current proposal I cannot agree to for the reasons explained.

TM> Well, although discussing forever is certainly not the goal, the reasons for rejecting a proposal need to be thoroughly understood.
WH> my point is what is the real driver for supporting a plain VXLAN data-plane here, the use cases I have seen in this txt is always where an application behind a NVE/TOR is demanding option c, but none of the NVE/TOR elements.


My understanding is that the applications  are contexts where overlays are present is when workloads (VMs or baremetal) need to be interconnected with VPNs. In these contexts, there can be reasons to want Option C to reduce the state on ASBRs.

In these context, its not the workload (VM or baremetal) that would typically handle VRFs, but really the vswitch or ToR.

WH2> can it not be all cases: TOR/vswitch/Application. I would make the solution flexible to support all of these not?

2015-11-13, Henderickx, Wim (Wim):

TM> The right trade-off to make may in fact depend on whether you prefer:
(a) a new dataplane stitching behavior on DC ASBRs (the behavior specified in this draft)
or (b) an evolution of the encaps on the vswitches and ToRs to support MPLS/MPLS/(UDP or GRE)

WH> b depends on the use case

I don't get what you mean by "b depends on the use case".
WH> see my above comment. If the real use case is an application behind NVE/TOR requiring model C, than all the discussion on impact on NVE/TOR is void. As such I want to have a discussion on the real driver/requirement for option c interworking with an IP based Fabric.

Although I can agree than detailing requirements can always help, I don't think one can assume a certain application to dismiss the proposal.

WH> for me the proposal is not acceptable for the reasons explained: too much impact on the data-planes

I wrote the above based on the idea that the encap used in MPLS/MPLS/(UDP or GRE), which hence has to be supported on the ToRs and vswitches.
Another possibility would be service-label/middle-label/Ethernet assuming an L2 adjacency between vswitches/ToRs and ASBRs, but this certainly does not match your typical DC architecture. Or perhaps had you something else in mind ?

WH> see above. The draft right now also requires changes in existing TOR/NVE so for me all this discussion/debate is void.

No, the spec as it is can be implemented in its VXLAN variant with existing vswitches (e.g. OVS allows to choose the VXLAN destination port, ditto for the linux kernel stack).

(ToR is certainly another story, most of them not having a flexible enough VXLAN dataplane nor support for any MPLS-over-IP.)

WH> and how many ports simultaneously would they support?

WH> and depending on implementation you don't need to change any of the TOR/vswitches.

Does this mean that for some implementations you may not need to change any of the TOR/vswitches, but that for some others you may ?

WH> any proposal on the table requires changes, so for me this is not a valid discussion

See above, the proposal in the draft does not necessarily need changes in vswitches.

Let me take a practical example : while I can quite easily see how to implement the procedures in draft-hao-bess-inter-nvo3-vpn-optionc based on current vswitch implementations of VXLAN, the lack of MPLS/MPLS/(UDP, GRE) support in commonplace vswitches seems to me as making that alternate solution you suggest harder to implement.

WH> I would disagree to this. Tell me which switch/TOR handles multiple UDP ports for VXLAN ?

I mentioned _v_switches, and many do support a variable destination port for VXLAN, which is sufficient to implement what the draft proposes.

-Thomas



From: Thomas Morin <[e-mail]<mailto:[e-mail]>>
Organization: Orange
Date: Friday 13 November 2015 at 09:57
To: Wim Henderickx <[e-mail]<mailto:[e-mail]>>
Cc: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

I agree on the analysis that this proposal is restricted to implementations that supports the chosen encap with non-IANA ports (which may be hard to achieve for instance on hardware implementations, as you suggest), or to context where managing multiple IPs would be operationally viable.

However, it does not seem obvious to me how the alternative you propose [relying on 3-label option C with an MPLS/MPLS/(UDP|GRE) encap] addresses the issue of whether the encap behavior is supported or not (e.g. your typical ToR chipset possibly may not support this kind of encap,  and even software-based switches may not be ready to support that today).

My take is that having different options to adapt to various implementations constraints we may have would have value.

(+ one question below on VXLAN...)

-Thomas


2015-11-12, Henderickx, Wim (Wim):
On VXLAN/NVGRE, do you challenge the fact that they would be used with a dummy MAC address that would be replaced by the right MAC by a sender based on an ARP request when needed ?

Is the above the issue you had in mind about VXLAN and NVGRE ?

WH> yes

I you don't mind me asking : why do you challenge that ?




_________________________________________________________________________________________________________________________



Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc

pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler

a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,

France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci



This message and its attachments may contain confidential or privileged information that may be protected by law;

they should not be distributed, used or copied without authorization.

If you have received this email in error, please notify the sender and delete this message and its attachments.

As emails may be altered, France Telecom - Orange shall not be liable if this message was modified, changed or falsified.

Thank you.