Re: [nvo3] Draft NVO3 WG Charter

[Ping Pan <[e-mail]>]

On Feb 18, 2012, at 12:22 AM, Larry Kreeger <[e-mail]> wrote:

> Yakov,
> 
> On 2/17/12 1:39 PM, "Yakov Rekhter" <[e-mail]> wrote:
>> Dave,
>> 
>>> John,
>>> 
>>> Thank you - beyond the operational concerns about extending MPLS
>>> and BGP into data centers, here are a few gaps in the
>>> marques-l3vpn-end-system draft:
>>> 
>>> (1) No multicast support.  Among the important examples for VMs is
>>> that the network load balancer in at least Windows 2000 and 2003
>>> is preferably run using multicast (otherwise it wants to use
>>> duplicate unicast MACs on the same LAN/VLAN).  There are other
>>> examples.
>>> 
>>> (2) A "Host OS" is required in the hypervisor to solve some
>>> routing problems.  At least one important hypervisor, ESXi,
>>> doesn't have a "Host OS" - the closest location for this
>>> functionality is the softswitch in the hypervisor.  I suspect that
>>> the routing functionality proposed for the Host OS can be moved
>>> into the softswitch, so this may be a very minor gap.
>>> 
>>> (3) The use of a single link-local address may cause problems
>>> with the usual technique of issuing a gratuitous ARP (or RARP)
>>> after a VM move - the design intent of that is to get the L2
>>> bridges to learn that the VM's network interfaces have moved.
>>> In particular, the draft asserts the following without
>>> explaining how it is done:
>>> 
>>>   When a particular VM "moves" from one physical end-system
>>>   to another, its respective VPN address will be advertised
>>>   by the new system and that notification propagated to all
>>>   attachment points of that VPN.
>> 
>> Please read section 5 of the draft.
>> 
>>> This is why NVO(3) is potentially complementary.  Defining
>>> a protocol that can help by communicating the VM "move" to a
>>> VPN end point that is in a softswitch or an external switch is
>>> among the things that the NVO effort proposes to undertake.  It
>>> would be desirable for this change communication protocol to
>>> be common across all (or at least a good number of) the involved
>>> networking technologies, as opposed to restricted to some L3VPNs,
>>> so that the protocol only has to be implemented once for each
>>> hypervisor.
>> 
>> Wrt the claim that "NVO(3) is potentially complementary", let me
>> point out that the currently proposed charter has plenty of other
>> stuff, besides defining the protocol to communicate the VM "move".
>> 
>> Wrt the protocol that would allow to communicate the VM "move" to
>> the network, I agree with you that it is indeed desirable. Having
>> said this, should we consider the work going on now in IEEE on VDP ?
> 
> I'm not an expert on VDP, but my understanding is that (since it is IEEE),
> it would only work for a layer 2 "inner" address.  Since we want the
> protocol to be layer agnostic (i.e. also support L3 VM addresses), it would
> need to be extended in some way...and it is not an IETF protocol to extend.
> 
>> Should we consider XMPP (along the lines proposed by
>> marques-l3vpn-end-system draft) ?
> 
> Consider away.  One reason we put a long list of protocol characteristic
> requirements within draft-kreeger-nvo3-overlay-cp-00.txt was so that all
> proposed candidate protocols could be evaluated against them so we can
> choose the most appropriate one.  I don't claim to know much about XMPP
> either, but my first reaction is...what does an protocol designed for chat
> have to do with this problem space, and isn't it a bit heavy weight for the
> purposes?
> 

Not at all. While many seem to be re-inventing the wheels in labels and L2, we see it as the need to have the applications talking to the existing network gears to forward the IP flows. XMPP is just one of the mature application-friendly protocols.

Regards,

Ping



>> 
>> Yakov.
>> 
>>> ________________________________
>>> From: John E Drake [[e-mail]]
>>> Sent: Friday, February 17, 2012 1:44 PM
>> To: Black, David; [e-mail]; [e-mail]
>>> Cc: Ronald Bonica; Nitin Bahadur; Adrian Farrel
>>> Subject: RE: [nvo3] Draft NVO3 WG Charter
>>> 
>>> David,
>>> 
>>> Please re-read my original email.
>>> 
>>> What I said was that the charter looked very much like the L3/L2 VPN charters
>> and that it would be useful to perform a gap analysis of these technologies
>> ag
>> ainst the DC specific requirements before starting to work on something new.
>>> 
>>> I didn't say there was no new work to be done.
>>> 
>>> Thanks,
>>> 
>>> John
>>> 
>>>> -----Original Message-----
>>>> From: [e-mail] [mailto:[e-mail]]
>>>> Sent: Friday, February 17, 2012 10:16 AM
>>>> To: John E Drake; [e-mail]; [e-mail]
>>>> Cc: Ronald Bonica; Nitin Bahadur; Adrian Farrel
>>>> Subject: RE: [nvo3] Draft NVO3 WG Charter
>>>> 
>>>> Hi John,
>>>> 
>>>> And I'm likewise familiar with the marques-l3vpn-end-system draft.
>>>> 
>>>> Perhaps we can tone this down a bit.  I read your initial message as
>>>> effectively asserting that there is nothing to do because VPN
>>>> technology
>>>> already solves all the problems in this entire space.  If you'll agree
>>>> that this wasn't what you meant, that ought to be a basis for a more
>>>> reasonable discussion.
>>>> 
>>>> In turn, I'll offer to lead off with some of the gaps in the -marques-
>>>> draft (yes, there has been some gap analysis of that draft), plus some
>>>> more explanation of Robert's point about how the proposed nvo (or nvo3)
>>>> work is complementary to that draft.
>>>> 
>>>> Can we do that?
>>>> 
>>>> Thanks,
>>>> --David
>>>> ________________________________
>>>> From: [e-mail] [[e-mail]] On Behalf Of John E
>>>> Drake [[e-mail]]
>>>> Sent: Friday, February 17, 2012 12:08 PM
>>>> To: Black, David; [e-mail]; [e-mail]
>>>> Cc: Ronald Bonica; Nitin Bahadur; Adrian Farrel
>>>> Subject: Re: [nvo3] Draft NVO3 WG Charter
>>>> 
>>>> David,
>>>> 
>>>> I have read all of the NV and DC drafts that have been published to
>>>> this point and the acting assumption in virtually all of them is that
>>>> something new is needed.  I know it is much more fun to work on
>>>> something new, but a gap analysis of existing technologies should be
>>>> undertaken before blithely proceeding.
>>>> 
>>>> Thanks,
>>>> 
>>>> John
>>>> 
>>>>> -----Original Message-----
>>>>> From: [e-mail] [mailto:[e-mail]]
>>>>> Sent: Friday, February 17, 2012 8:31 AM
>>>>> To: John E Drake; [e-mail]; [e-mail]
>>>>> Cc: Ronald Bonica; Nitin Bahadur; Adrian Farrel
>>>>> Subject: RE: [nvo3] Draft NVO3 WG Charter
>>>>> 
>>>>> Hi John,
>>>>> 
>>>>>>> BGP and MPLS are non-starters for a lot of datacenter-internal
>>>>>>> networks.
>>>>>> 
>>>>>> [JD]  This is an assertion.  It is also the misses the fact that
>>>> MPLS
>>>>>> is only required to mux/demux packets at the edges of the VPN
>>>>> network.
>>>>> 
>>>>> 
>>>>> 
>>>>> Indeed it is, but I stand by it.  The interesting "edges of the VPN
>>>>> 
>>>>> network" for NVO include datacenter ToR switches, datacenter access
>>>>> 
>>>>> switches and hypervisor softswitches - there are plenty of examples
>>>> of
>>>>> 
>>>>> these for which MPLS and BGP are non-starters.
>>>>> 
>>>>> 
>>>>> 
>>>>> I suggest reading the NVGRE and VXLAN drafts for more context:
>>>>> 
>>>>>   http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-00
>>>>> 
>>>>>   http://tools.ietf.org/html/draft-mahalingam-dutt-dcops-vxlan-00
>>>>> 
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> --David
>>>>> ----------------------------------------------------
>>>>> David L. Black, Distinguished Engineer
>>>>> EMC Corporation, 176 South St., Hopkinton, MA  01748
>>>>> [phone]             FAX: [phone]
>>>>> [e-mail]<mailto:[e-mail]>        Mobile: +1
>>>> (978)
>>>>> 394-7754
>>>>> ----------------------------------------------------
>>>>> ________________________________
>>>>> From: John E Drake [[e-mail]]
>>>>> Sent: Friday, February 17, 2012 11:13 AM
>>>>> To: Black, David; [e-mail]; [e-mail]
>>>>> Cc: Ronald Bonica; Nitin Bahadur; Adrian Farrel
>>>>> Subject: RE: [nvo3] Draft NVO3 WG Charter
>>>>> 
>>>>> Comments inline
>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: [e-mail] [mailto:[e-mail]]
>>>>>> Sent: Friday, February 17, 2012 8:04 AM
>>>>>> To: John E Drake; [e-mail]; [e-mail]
>>>>>> Cc: Ronald Bonica; Nitin Bahadur; Adrian Farrel
>>>>>> Subject: RE: [nvo3] Draft NVO3 WG Charter
>>>>>> 
>>>>>> John,
>>>>>> 
>>>>>>> This basically is a re-statement of what is done by L3/L2 VPNs.
>>>>> It'
>>>>>>> might be useful to do a gap analysis of these existing
>>>>> technologies,
>>>>>>> in particular E-VPNs (http://tools.ietf.org/html/draft-raggarwa-
>>>>>> sajassi-l2vpn-evpn-04),
>>>>>>> before asserting that something new is required.
>>>>>> BGP and MPLS are non-starters for a lot of datacenter-internal
>>>>>> networks.
>>>>> 
>>>>> [JD]  This is an assertion.  It is also the misses the fact that MPLS
>>>>> is only required to mux/demux packets at the edges of the VPN
>>>> network.
>>>>> 
>>>>>> Some of the more important NVO deployment scenarios involve map-
>>>> and-
>>>>>> encap in a hypervisor software network switch.
>>>>> 
>>>>> [JD]  Your point eludes me.
>>>>> 
>>>>>> 
>>>>>> Thanks,
>>>>>> --David
>>>>>> ----------------------------------------------------
>>>>>> David L. Black, Distinguished Engineer
>>>>>> EMC Corporation, 176 South St., Hopkinton, MA  01748
>>>>>> [phone]             FAX: [phone]
>>>>>> [e-mail]<mailto:[e-mail]>        Mobile: +1
>>>>> (978)
>>>>>> 394-7754
>>>>>> ----------------------------------------------------
>>>>>> ________________________________
>>>>>> From: [e-mail] [[e-mail]] On Behalf Of
>>>> John
>>>>> E
>>>>>> Drake [[e-mail]]
>>>>>> Sent: Friday, February 17, 2012 10:00 AM
>>>>>> To: Thomas Narten; [e-mail]
>>>>>> Cc: Ronald Bonica; Nitin Bahadur; Adrian Farrel
>>>>>> Subject: Re: [nvo3] Draft NVO3 WG Charter
>>>>>> 
>>>>>> Thomas,
>>>>>> 
>>>>>> This basically is a re-statement of what is done by L3/L2 VPNs.  It
>>>>>> might be useful to do a gap analysis of these existing
>>>> technologies,
>>>>> in
>>>>>> particular E-VPNs (http://tools.ietf.org/html/draft-raggarwa-
>>>> sajassi-
>>>>>> l2vpn-evpn-04), before asserting that something new is required.
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> John
>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: [e-mail] [mailto:[e-mail]] On
>>>>> Behalf
>>>>>> Of
>>>>>>> Thomas Narten
>>>>>>> Sent: Friday, February 17, 2012 6:52 AM
>>>>>>> To: [e-mail]
>>>>>>> Subject: [nvo3] Draft NVO3 WG Charter
>>>>>>> 
>>>>>>> Below is a draft charter for this effort. One detail is that we
>>>>>>> started out calling this effort NVO3 (Network Virtualization Over
>>>>>> L3),
>>>>>>> but have subsequently realized that we should not focus on just
>>>>> "over
>>>>>>> L3". One goal of this effort is to develop an overlay standard
>>>> that
>>>>>>> works over L3, but we do not want to restrict ourselves only to
>>>>> "over
>>>>>>> L3". The framework and architecture that we are proposing to work
>>>>> on
>>>>>>> should be applicable to other overlays as well (e.g., L2 over
>>>>>>> L2). This is (hopefully) captured in the proposed charter.
>>>>>>> 
>>>>>>> Comments?
>>>>>>> 
>>>>>>> Thomas
>>>>>>> 
>>>>>>> NVO: Network Virtualization Overlays
>>>>>>> 
>>>>>>> Support for multi-tenancy has become a core requirement of data
>>>>>>> centers, especially in the context of data centers which include
>>>>>>> virtualized servers known as virtual machines (VMs).  With
>>>>>>> multi-tenancy, a data center can support the needs of many
>>>>> thousands
>>>>>>> of individual tenants, ranging from individual groups or
>>>>> departments
>>>>>>> within a single organization all the way up to supporting
>>>> thousands
>>>>>> of
>>>>>>> individual customers.  A key multi-tenancy requirement is traffic
>>>>>>> isolation, so that a tenant's traffic (and internal address
>>>> usage)
>>>>> is
>>>>>>> not visible to any other tenant and does not collide with
>>>> addresses
>>>>>>> used within the data center itself.  Such isolation can be
>>>> achieved
>>>>>> by
>>>>>>> creating and assigning one or more virtual networks to each
>>>> tenant
>>>>>>> such that traffic within a virtual network is isolated from
>>>> traffic
>>>>>> in
>>>>>>> other virtual networks.
>>>>>>> 
>>>>>>> Tenant isolation is primarily achieved today within data centers
>>>>>> using
>>>>>>> Ethernet VLANs. But the 12-bit VLAN tag field isn't large enough
>>>> to
>>>>>>> support existing and future needs. A number of approaches to
>>>>>> extending
>>>>>>> VLANs and scaling L2s have been proposed or developed, including
>>>>> IEEE
>>>>>>> 802.1ah Shortest Path Bridging (SPB) and TRILL (with the proposed
>>>>>>> fine-grained labeling extension).  At the L3 (IP) level, VXLAN
>>>> and
>>>>>>> NVGRE have also been proposed. As outlined in
>>>>>>> draft-narten-nvo3-overlay-problem-statement-01.txt, however,
>>>>> existing
>>>>>>> L2 approaches are not satisfactory for all data center operators,
>>>>>>> e.g., larger data centers that desire to keep L2 domains small or
>>>>>> push
>>>>>>> L3 further into the data center (e.g., all the way to top-of-rack
>>>>>>> switches). Furthermore, there is a desire to decouple the
>>>>>>> configuration of the data center network from the configuration
>>>>>>> associated with individual tenant applications and to seamlessly
>>>>> and
>>>>>>> rapidly update the network state to handle live VM migrations or
>>>>> fast
>>>>>>> spin-up and spin-down of new tenant VMs (or servers). Such tasks
>>>>> are
>>>>>>> complicated by the need to simultaneously reconfigure and update
>>>>> data
>>>>>>> center network state (e.g., VLAN settings on individual
>>>> switches).
>>>>>>> 
>>>>>>> This WG will develop an approach to multi-tenancy that does not
>>>>> rely
>>>>>>> on any underlying L2 mechanisms to support multi-tenancy. In
>>>>>>> particular, the WG will develop an approach where multitenancy is
>>>>>>> provided at the IP layer using an encapsulation header that
>>>> resides
>>>>>>> above IP. This effort is explicitly intended to leverage the
>>>>> interest
>>>>>>> in L3 overlay approaches as exemplified by VXLAN
>>>>>>> (draft-mahalingam-dutt-dcops-vxlan-00.txt) and NVGRE
>>>>>>> (draft-sridharan-virtualization-nvgre-00.txt).
>>>>>>> 
>>>>>>> Overlays are a form of "map and encap", where an ingress node
>>>> maps
>>>>>> the
>>>>>>> destination address of an arriving packet (e.g., from a source
>>>>> tenant
>>>>>>> VM) into the address of an egress node to which the packet can be
>>>>>>> tunneled to. The ingress node then encapsulates the packet in an
>>>>>> outer
>>>>>>> header and tunnels it to the egress node, which decapsulates the
>>>>>>> packet and forwards the original (unmodified) packet to its
>>>>> ultimate
>>>>>>> destination (e.g., a destination tenant VM). All map-and-encap
>>>>>>> approaches must address two issues: the encapsulation format
>>>> (i.e.,
>>>>>>> the contents of the outer header) and how to distribute and
>>>> manage
>>>>>> the
>>>>>>> mapping tables used by the tunnel end points.
>>>>>>> 
>>>>>>> The first area of work concerns encapsulation formats. This WG
>>>> will
>>>>>>> develop requirements and desirable properties for any
>>>> encapsulation
>>>>>>> format. Given the number of already existing encapsulation
>>>> formats,
>>>>>>> it is not an explicit goal of this effort to choose exactly one
>>>>>> format
>>>>>>> or to develop yet another new one.
>>>>>>> 
>>>>>>> A second work area is in the control plane, which allows an
>>>> ingress
>>>>>>> node to map the "inner" (tenant VM) address into an "outer"
>>>>>>> (underlying transport network) address in order to tunnel a
>>>> packet
>>>>>>> across the data center. We propose to develop two control planes.
>>>>> One
>>>>>>> control plane will use a learning mechanism similar to IEEE
>>>> 802.1D
>>>>>>> learning, and could be appropriate for smaller data centers. A
>>>>>> second,
>>>>>>> more scalable control plane would be aimed at large sites,
>>>> capable
>>>>> of
>>>>>>> scaling to hundreds of thousands of nodes. Both control planes
>>>> will
>>>>>>> need to handle the case of VMs moving around the network in a
>>>>> dynamic
>>>>>>> fashion, meaning that they will need to support tunnel endpoints
>>>>>>> registering and deregistering mappings as VMs change location and
>>>>>>> ensuring that out-of-date mapping tables are only used for short
>>>>>>> periods of time. Finally, the second control plane must also be
>>>>>>> applicable to geographically dispersed data centers.
>>>>>>> 
>>>>>>> Although a key objective of this WG is to produce a solution that
>>>>>>> supports an L2 over L3 overlay, an important goal is to develop a
>>>>>>> "layer agnostic" framework and architecture, so that any specific
>>>>>>> overlay approach can reuse the output of this working group. For
>>>>>>> example, there is no inherent reason why the same framework could
>>>>> not
>>>>>>> be used to provide for L2 over L2 or L3 over L3. The main
>>>>> difference
>>>>>>> would be in the address formats of the inner and outer headers
>>>> and
>>>>>> the
>>>>>>> encapsulation header itself.
>>>>>>> 
>>>>>>> Finally, some work may be needed in connecting an overlay network
>>>>>> with
>>>>>>> traditional L2 or L3 VPNs (e.g., VPLS). One approach appears
>>>>> straight
>>>>>>> forward, in that there is a clear boundary between a VPN device
>>>> and
>>>>>>> the edge of an overlay network. Packets forwarded across the
>>>>> boundary
>>>>>>> would simply need to have the tenant identifier on the overlay
>>>> side
>>>>>>> mapped into a corresponding VPN identifier on the VPN
>>>>>>> side. Conceptually, this would appear to be analogous to what is
>>>>> done
>>>>>>> already today when interfacing between L2 VLANs and VPNs.
>>>>>>> 
>>>>>>> The specific deliverables for this group include:
>>>>>>> 
>>>>>>> 1) Finalize and publish the overall problem statement as an
>>>>>>> Informational RFC (basis:
>>>>>>> draft-narten-nvo3-overlay-problem-statement-01.txt)
>>>>>>> 
>>>>>>> 2) Develop requirements and desirable properties for any
>>>>>> encapsulation
>>>>>>> format, and identify suitable encapsulations. Given the number of
>>>>>>> already existing encapsulation formats, it is not an explicit
>>>> goal
>>>>> of
>>>>>>> this effort to choose exactly one format or to develop a new one.
>>>>>>> 
>>>>>>> 3) Produce a Standards Track control plane document that
>>>> specifies
>>>>>> how
>>>>>>> to build mapping tables using a "learning" approach. This
>>>> document
>>>>> is
>>>>>>> expected to be short, as the algorithm itself will use a
>>>> mechanism
>>>>>>> similar to IEEE 802.1D learning.
>>>>>>> 
>>>>>>> 4) Develop requirements (and later a Standards Track protocol)
>>>> for
>>>>> a
>>>>>>> more scalable control plane for managing and distributing the
>>>>>> mappings
>>>>>>> of "inner" to "outer" addresses. We will develop a reusable
>>>>> framework
>>>>>>> suitable for use by any mapping function in which there is a need
>>>>> to
>>>>>>> map "inner" to outer addresses. Starting point:
>>>>>>> draft-kreeger-nvo3-overlay-cp-00.txt
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> nvo3 mailing list
>>>>>>> [e-mail]
>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3
>>>>>> _______________________________________________
>>>>>> nvo3 mailing list
>>>>>> [e-mail]
>>>>>> https://www.ietf.org/mailman/listinfo/nvo3
>>>>> 
>>>> 
>>>> _______________________________________________
>>>> nvo3 mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/nvo3
>>> 
>>> 
>>> _______________________________________________
>>> nvo3 mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/nvo3
>> _______________________________________________
>> nvo3 mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/nvo3
> 
> _______________________________________________
> nvo3 mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/nvo3