Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Narasimhan Venkataramaiah <narave@microsoft.com> Tue, 20 September 2011 15:13 UTC

Return-Path: <narave@microsoft.com>
X-Original-To: armd@ietfa.amsl.com
Delivered-To: armd@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CCA5B21F8C2B for <armd@ietfa.amsl.com>; Tue, 20 Sep 2011 08:13:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.599
X-Spam-Level:
X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ypwtjvQhW7kr for <armd@ietfa.amsl.com>; Tue, 20 Sep 2011 08:13:12 -0700 (PDT)
Received: from smtp.microsoft.com (mailb.microsoft.com [131.107.115.215]) by ietfa.amsl.com (Postfix) with ESMTP id 9BB0621F8C1E for <armd@ietf.org>; Tue, 20 Sep 2011 08:13:12 -0700 (PDT)
Received: from TK5EX14HUBC105.redmond.corp.microsoft.com (157.54.80.48) by TK5-EXGWY-E802.partners.extranet.microsoft.com (10.251.56.168) with Microsoft SMTP Server (TLS) id 8.2.176.0; Tue, 20 Sep 2011 08:15:38 -0700
Received: from TK5EX14MLTW651.wingroup.windeploy.ntdev.microsoft.com (157.54.71.39) by TK5EX14HUBC105.redmond.corp.microsoft.com (157.54.80.48) with Microsoft SMTP Server (TLS) id 14.1.339.2; Tue, 20 Sep 2011 08:15:38 -0700
Received: from TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com ([169.254.1.117]) by TK5EX14MLTW651.wingroup.windeploy.ntdev.microsoft.com ([157.54.71.39]) with mapi id 14.01.0339.002; Tue, 20 Sep 2011 08:15:38 -0700
From: Narasimhan Venkataramaiah <narave@microsoft.com>
To: Linda Dunbar <linda.dunbar@huawei.com>, Murari Sridharan <muraris@microsoft.com>, "david.black@emc.com" <david.black@emc.com>, "armd@ietf.org" <armd@ietf.org>
Thread-Topic: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?
Thread-Index: AQHMdgwxzKQpmaafWkqNIE83xQq6KJVTbCHGgANomAD//41iwA==
Date: Tue, 20 Sep 2011 15:15:37 +0000
Message-ID: <65755BEBE02F7C41BD4F137AED91DA5E2FD31BBA@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com>
References: <CAP_bo1b_2D=fbJJ8uGb8LPWb-6+sTQn1Gsh9YAp8pFs3JY_rrw@mail.gmail.com> <CAOyVPHTLYv=-GbjimpDr5NsxMUeWKtVKzStY9yxQO7s4YD2Ywg@mail.gmail.com> <CAP_bo1Ya7p+OS7fS40jE4+UZuhmeO+MAroC=CZK5sMEE625z8Q@mail.gmail.com> <CAOyVPHTcFr7F4ymQyXyECtS6f8z1XyZn40a_5WcpcjF9y0hZvQ@mail.gmail.com> <CA+-tSzx6DGPptGdtx5awzhnPPJgRHow2SWfuwRP4rwjdN1MXmw@mail.gmail.com> <CAOyVPHRUFrm2xqwrd4OVQbRotae+3+E8xhOF4n1dmWERVdLPEg@mail.gmail.com> <CA+-tSzzvj=eUYT4ZOKiy9yGssmrx71eby2f1xkKKh4NkXL5-Vg@mail.gmail.com> <CAOyVPHS-OF8+GRpmcAxbCj5_HEvgVSOvRMA2hC66v1pxs526Nw@mail.gmail.com> <35BAFA1F-25E8-442E-8FE6-2D5691DCBEAC@kumari.net> <7C4DFCE962635144B8FAE8CA11D0BF1E058CCE4D4C@MX14A.corp.emc.com> <EF5EF2B13ED09B4F871D9A0DBCA463C216C1E72D@TK5EX14MBXC300.redmond.corp.microsoft.com> <4A95BA014132FF49AE685FAB4B9F17F610CA2E67@dfweml503-mbx.china.huawei.com> <65755BEBE02F7C41BD4F137AED91DA5E2FD2AF86@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> <4A95BA014132FF49AE685FAB4B9F17F610CA8508@dfweml503-mbx.china.huawei.com>
In-Reply-To: <4A95BA014132FF49AE685FAB4B9F17F610CA8508@dfweml503-mbx.china.huawei.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [157.54.51.42]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?
X-BeenThere: armd@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." <armd.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/armd>, <mailto:armd-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/armd>
List-Post: <mailto:armd@ietf.org>
List-Help: <mailto:armd-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/armd>, <mailto:armd-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Sep 2011 15:13:14 -0000

I am trying to understand the implication to network virtualization. Say an enterprise has 2 data centers in different geographical regions and they decided to move a subnet from one DC to another - whatever is done to route the subnet to the new data center applies here too. What is different for a virtualized network unless the subnet spans 2 data centers?

Simha

-----Original Message-----
From: Linda Dunbar [mailto:linda.dunbar@huawei.com] 
Sent: Tuesday, September 20, 2011 7:58 AM
To: Narasimhan Venkataramaiah; Linda Dunbar; Murari Sridharan; david.black@emc.com; armd@ietf.org
Subject: RE: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Simha, 

What if entire subnet is moved to "cloud data center"? For applications which communicate with external peers, do they go through the enterprise data center? What if the enterprise data center is removed after moving all the applications to "cloud"? 

Linda

> -----Original Message-----
> From: Narasimhan Venkataramaiah [mailto:narave@microsoft.com]
> Sent: Sunday, September 18, 2011 1:01 PM
> To: Linda Dunbar; Murari Sridharan; david.black@emc.com; armd@ietf.org
> Subject: RE: [armd] how does "draft-sridharan-virtualization-nvgre-00"
> advertise its external facing hosts' IP addresses to external world?
> 
> The easiest from the point of view of configuration would be to route 
> everything back through the enterprise - not necessarily the optimal 
> from the enterprise point of view. Are you referring to a scenario 
> where the VMs subnet is split between the cloud and the enterprise?
> Otherwise I don't see the implication on virtualization as its no 
> different than getting the traffic routed to the enterprise in the 
> first case.
> 
> Simha
> 
> ________________________________________
> From: armd-bounces@ietf.org [armd-bounces@ietf.org] on behalf of Linda 
> Dunbar [linda.dunbar@huawei.com]
> Sent: Sunday, September 18, 2011 7:06 AM
> To: Murari Sridharan; david.black@emc.com; armd@ietf.org
> Subject: [armd] how does "draft-sridharan-virtualization-nvgre-00"
> advertise its external facing hosts' IP addresses to external world?
> 
> Hi Murari,
> 
> Thank you very much for sharing the presentation.
> 
> One question:
> 
> For a host within an Enterprise site which needs to communicate with 
> external peers, the host either uses public IP address which is 
> visible to external peers or uses private IP address which is 
> translated to public address at the Enterprise site's gateway.
> 
> When this host is moved to "Cloud data center", will the "Cloud Data 
> center" advertise this host address to external peers? Or will all 
> external peers go through enterprise's gateway to reach this host 
> which is no longer residing in the enterprise site?
> 
> Thanks, Linda
> 
> > -----Original Message-----
> > From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf
> Of
> > Murari Sridharan
> > Sent: Saturday, September 17, 2011 3:02 PM
> > To: david.black@emc.com; armd@ietf.org
> > Subject: Re: [armd] soliciting typical network designs for ARMD
> >
> > FYI, here is a talk that I gave last week in relation to the nvgre 
> > draft below.
> > http://channel9.msdn.com/Events/BUILD/BUILD2011/SAC-442T
> >
> > Thanks
> > Murari
> >
> > -----Original Message-----
> > From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf
> Of
> > david.black@emc.com
> > Sent: Friday, September 16, 2011 6:14 AM
> > To: armd@ietf.org
> > Subject: Re: [armd] soliciting typical network designs for ARMD
> >
> > And two more drafts on this topic:
> >
> > http://www.ietf.org/id/draft-mahalingam-dutt-dcops-vxlan-00.txt
> > http://www.ietf.org/id/draft-sridharan-virtualization-nvgre-00.txt
> >
> > The edge switches could be the software switches in hypervisors.
> >
> > Thanks,
> > --David
> >
> >
> > > -----Original Message-----
> > > From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On
> Behalf
> > > Of Warren Kumari
> > > Sent: Wednesday, August 31, 2011 3:16 PM
> > > To: Vishwas Manral
> > > Cc: armd@ietf.org
> > > Subject: Re: [armd] soliciting typical network designs for ARMD
> > >
> > >
> > > On Aug 11, 2011, at 11:40 PM, Vishwas Manral wrote:
> > >
> > > > Hi Linda/ Anoop,
> > > >
> > > > Here is the example of the design I was talking about, as 
> > > > defined
> > by google.
> > >
> > > Just a clarification -- s/as defined by google/as described by
> > someone
> > > who happens to work for google/
> > >
> > > W
> > >
> > > > http://www.ietf.org/id/draft-wkumari-dcops-l3-vmmobility-00.txt
> > > >
> > > > Thanks,
> > > > Vishwas
> > > > On Tue, Aug 9, 2011 at 2:50 PM, Anoop Ghanwani
> > <anoop@alumni.duke.edu> wrote:
> > > >
> > > > >>>>
> > > > (though I think if there was a standard way to map Multicast MAC
> to
> > > > Multicast IP, they could
> > > probably use such a standard mechanisms).
> > > > >>>>
> > > >
> > > > They can do that, but then this imposes requirements on the 
> > > > equipment to be able to do multicast forwarding, and even if 
> > > > does, because of pruning requirements the number of groups would 
> > > > be
> very
> > > > large.  The average data center switch probably won't handle 
> > > > that many groups.
> > > >
> > > > On Tue, Aug 9, 2011 at 2:41 PM, Vishwas Manral
> > <vishwas.ietf@gmail.com> wrote:
> > > > Hi Anoop,
> > > >
> > > > From what I know they do not use Multicast GRE (I hear the extra
> 4
> > > > bytes in the GRE header is a
> > > proprietery extension).
> > > >
> > > > I think a directory based mechanism is what is used (though I
> think
> > > > if there was a standard way to
> > > map Multicast MAC to Multicast IP, they could probably use such a
> > standard mechanisms).
> > > >
> > > > Thanks,
> > > > Vishwas
> > > > On Tue, Aug 9, 2011 at 2:03 PM, Anoop Ghanwani
> > <anoop@alumni.duke.edu> wrote:
> > > > Hi Vishwas,
> > > >
> > > > How do they get multicast through the network in that case?
> > > > Are they planning to use multicast GRE, or just use directory
> based
> > > > lookups and not worry about multicast applications for now?
> > > >
> > > > Anoop
> > > >
> > > > On Tue, Aug 9, 2011 at 1:27 PM, Vishwas Manral
> > <vishwas.ietf@gmail.com> wrote:
> > > > Hi Linda,
> > > >
> > > > The data packets can be tunnelled at the ToR over say a GRE
> packet
> > > > and the core is a Layer-3 core
> > > (except for the downstream ports). So we could have encapsulation/ 
> > > decapsulation of L2 over GRE at the ToR.
> > > >
> > > > The very same thing can be done at the hypervisor layer too, in 
> > > > which case the entire DC network
> > > would look like a Layer-3 flat network including the ToR to server 
> > > link and the hypervisor would do the tunneling.
> > > >
> > > > I am not sure if you got the points above or not. I know cloud 
> > > > OS companies that provide the service
> > > and have big announced customers.
> > > >
> > > > Thanks,
> > > > Vishwas
> > > > On Tue, Aug 9, 2011 at 11:51 AM, Linda Dunbar
> <dunbar.ll@gmail.com>
> > wrote:
> > > > Vishwas,
> > > >
> > > > In my mind the bullet 1) in the list refers to ToR switches 
> > > > downstream ports (facing servers)
> > > running Layer 2 and ToR uplinks ports run IP Layer 3.
> > > >
> > > > Have you seen data center networks with ToR switches downstream 
> > > > ports (i.e. facing servers) enabling
> > > IP routing, even though the physical links are Ethernet?
> > > > If yes, we should definitely include it in the ARMD draft.
> > > >
> > > > Thanks,
> > > > Linda
> > > > On Tue, Aug 9, 2011 at 12:58 PM, Vishwas Manral
> > <vishwas.ietf@gmail.com> wrote:
> > > > Hi Linda,
> > > > I am unsure what you mean by this, but:
> > > >   * layer 3 all the way to TOR (Top of Rack switches), We can
> also
> > > > have a heirarchical network, with the core totally Layer-3 (and 
> > > > having seperate
> > > routing), from the hosts still in a large Layer-3 subnet. Another 
> > > aspect could be to have a totally
> > > Layer-3 network.
> > > >
> > > > The difference between them is the link between the servers and
> the
> > ToR.
> > > >
> > > > Thanks,
> > > > Vishwas
> > > > On Tue, Aug 9, 2011 at 10:22 AM, Linda Dunbar
> <dunbar.ll@gmail.com>
> > wrote:
> > > > During the 81st IETF ARMD WG discussion, it was suggested that 
> > > > it
> > is
> > > > necessary to document typical
> > > data center network designs so that address resolution scaling
> issues
> > > can be properly described. Many data center operators have
> expressed
> > that they can't openly reveal their detailed network designs.
> > > Therefore, we only want to document anonymous designs without too
> > much
> > > detail. During the journey of establishing ARMD, we have come
> across
> > the following typical data center network designs:
> > > >   * layer 3 all the way to TOR (Top of Rack switches),
> > > >   * large layer 2 with hundreds (or thousands) of ToRs being 
> > > > interconnected by Layer 2. This
> > > design will have thousands of hosts under the L2/L3 boundary 
> > > router
> > > (s)
> > > >   * CLOS design  with thousands of switches. This design will
> have
> > > > thousands of hosts under the
> > > L2/L3 boundary router(s)
> > > > We have heard that each of the designs above has its own problems.
> > > > ARMD problem statements might
> > > need to document DC problems under each typical design.
> > > > Please send feedback to us (either to the armd email list  or to
> > the
> > > > ARMD chair Benson & Linda) to
> > > indicate if we have missed any typical Data Center network designs.
> > > >
> > > > Your contribution can greatly accelerate the progress of ARMD WG.
> > > >
> > > > Thank you very much.
> > > >
> > > > Linda & Benson
> > > >
> > > >
> > >
> > > _______________________________________________
> > > armd mailing list
> > > armd@ietf.org
> > > https://www.ietf.org/mailman/listinfo/armd
> >
> > _______________________________________________
> > armd mailing list
> > armd@ietf.org
> > https://www.ietf.org/mailman/listinfo/armd
> >
> > _______________________________________________
> > armd mailing list
> > armd@ietf.org
> > https://www.ietf.org/mailman/listinfo/armd
> _______________________________________________
> armd mailing list
> armd@ietf.org
> https://www.ietf.org/mailman/listinfo/armd