Re: [nvo3] Draft NVO3 WG Charter

[<[e-mail]>]

Yaakov,

> What are the specific *technical* reason(s) why MPLS over GRE is
> a "non-starter" for (a) ToR switches, (b) datacenter access switches,
> and (c) hypervisor softswitches ?

Sure, that was on my list of things to do, so thanks for asking, but
my original assertion was:

> > > > BGP and MPLS are non-starters for a lot of datacenter-internal
> > > > networks.

Let's start with one of the more important problems that has motivated
interest in overlays for virtual networking.  From the proposed charter:

   Support for multi-tenancy has become a core requirement of data
   centers, especially in the context of data centers which include
   virtualized servers known as virtual machines (VMs).  

In these datacenters, there is a sizeable population of virtual machines
running using VLANs.  In a bit more detail, that means:
	- Data Plane: TCP/IP, Ethernet VLANs
	- Control Plane: IP Routing based on IGPs (e.g., OSPF), VLAN
		configuration, LLDP, etc.
Beyond that, management, operational practices and network admin skills
are matched to the environment.

Again from the proposed charter:

   Tenant isolation is primarily achieved today within data centers using
   Ethernet VLANs. But the 12-bit VLAN tag field isn't large enough to
   support existing and future needs.

This is an incremental growth problem - the datacenter is running fine
with VLANs, but VLAN address space is being exhausted.  The solution
should be incremental in impact and incrementally deployable.

Taking a look at MPLS and BGP, and assuming that the gaps previously
pointed out in the marques-l3vpn-end-system draft are addressed, I see
the following:
	- Introduce new data plane: MPLS
	- Introduce new control plane: BGP
	- Significant changes to management and operational practices
	- New network admin skills required
That's not the best incremental impact story.  The last one is particularly
important.

Incremental deployment also leaves a bit to be desired, as the
approach described in the marques-l3vpn-end-system does not work with
existing VM live migration implementations, because the VM's IP addressing
("VM route table" in the draft) has to be modified.  This has some
consequences:

	- Existing live VM migration implementations won't be able to move
		a VM between VLAN and MPLS-BGP environments because they
		don't reconfigure the VM route table.
	- New live VM migration implementations that want to support this
		sort of cross-environment migration will need additional
		functionality (e.g., may need to coax the VM to go renew
		its DHCP lease to discover what happened, and hope it copes).
	- Cold migration of a VM between VLAN and MPLS-BGP environments
		requires reconfiguration to change the VM route table. 
		Similarly, use of a common VM template across both environments
		requires an additional reconfiguration step.

If you'll pardon the double-negative, this isn't to say that an MPLS-BGP
approach can't be made to work, and I'd definitely like to see a common
protocol between end systems and the network for new live migration 
implementations that can be used by any relevant technology.  Rather, the
deployment and operational pain of the above makes it a non-starter
courtesy of its impacts (data plane, control plane, management, operational
practices, required netadmin skills), and operational drawbacks wrt
existing live VM migration deployments.

I believe that with enough engineering and design work, something workable
will emerge here, but I'm concerned that for a significant portion of the
problem space, this is akin to trying to turn a hammer into a powered
screwdriver.

Thanks,
--David

> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf Of Yakov Rekhter
> Sent: Monday, February 20, 2012 9:31 AM
> To: Black, David
> Cc: [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail];
> [e-mail]
> Subject: Re: [nvo3] Draft NVO3 WG Charter
> 
> David,
> 
> > Hi John,
> >
> > > > BGP and MPLS are non-starters for a lot of datacenter-internal
> > > > networks.
> > >
> > > [JD]  This is an assertion.  It is also the misses the fact that MPLS
> > > is only required to mux/demux packets at the edges of the VPN network.
> >
> > Indeed it is, but I stand by it.  The interesting "edges of the VPN
> > network" for NVO include datacenter ToR switches, datacenter access
> > switches and hypervisor softswitches - there are plenty of examples of
> > these for which MPLS and BGP are non-starters.
> 
> What are the specific *technical* reason(s) why MPLS over GRE is
> a "non-starter" for (a) ToR switches, (b) datacenter access switches,
> and (c) hypervisor softswitches ?
> 
> Yakov.
> 
> > I suggest reading the NVGRE and VXLAN drafts for more context:
> >
> >    http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-00
> >    http://tools.ietf.org/html/draft-mahalingam-dutt-dcops-vxlan-00
> >
> >
> >
> > Thanks,
> > --David
> > ----------------------------------------------------
> > David L. Black, Distinguished Engineer
> > EMC Corporation, 176 South St., Hopkinton, MA  01748
> > [phone]             FAX: [phone]
> > [e-mail]        Mobile: [phone]
> > ----------------------------------------------------