Re: [armd] soliciting typical network designs for ARMD

Linda Dunbar <linda.dunbar@huawei.com> Wed, 31 August 2011 19:35 UTC

Return-Path: <linda.dunbar@huawei.com>
X-Original-To: armd@ietfa.amsl.com
Delivered-To: armd@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 32BDD21F8F55 for <armd@ietfa.amsl.com>; Wed, 31 Aug 2011 12:35:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.307
X-Spam-Level:
X-Spam-Status: No, score=-6.307 tagged_above=-999 required=5 tests=[AWL=0.292, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5YJXSE89iKez for <armd@ietfa.amsl.com>; Wed, 31 Aug 2011 12:35:35 -0700 (PDT)
Received: from usaga02-in.huawei.com (usaga02-in.huawei.com [206.16.17.70]) by ietfa.amsl.com (Postfix) with ESMTP id 5794521F8F54 for <armd@ietf.org>; Wed, 31 Aug 2011 12:35:35 -0700 (PDT)
Received: from huawei.com (localhost [127.0.0.1]) by usaga02-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LQT0052D55SNP@usaga02-in.huawei.com> for armd@ietf.org; Wed, 31 Aug 2011 14:37:04 -0500 (CDT)
Received: from dfweml201-edg.china.huawei.com ([172.18.4.104]) by usaga02-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTPS id <0LQT00AM455NNZ@usaga02-in.huawei.com> for armd@ietf.org; Wed, 31 Aug 2011 14:37:04 -0500 (CDT)
Received: from DFWEML402-HUB.china.huawei.com (10.193.5.102) by dfweml201-edg.china.huawei.com (172.18.9.107) with Microsoft SMTP Server (TLS) id 14.1.270.1; Wed, 31 Aug 2011 12:36:58 -0700
Received: from DFWEML503-MBX.china.huawei.com ([169.254.3.37]) by DFWEML402-HUB.china.huawei.com ([::1]) with mapi id 14.01.0270.001; Wed, 31 Aug 2011 12:36:44 -0700
Date: Wed, 31 Aug 2011 19:36:43 +0000
From: Linda Dunbar <linda.dunbar@huawei.com>
In-reply-to: <35BAFA1F-25E8-442E-8FE6-2D5691DCBEAC@kumari.net>
X-Originating-IP: [10.192.11.66]
To: Warren Kumari <warren@kumari.net>, Vishwas Manral <vishwas.ietf@gmail.com>
Message-id: <4A95BA014132FF49AE685FAB4B9F17F6102091D3@dfweml503-mbx.china.huawei.com>
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
Content-language: en-US
Content-transfer-encoding: 7bit
Accept-Language: en-US
Thread-topic: [armd] soliciting typical network designs for ARMD
Thread-index: AQHMVt0qRHXKx8xJ1kiMH1hmlbd0hJUVhDmAgAOGoACAHuGmgP//jx7w
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
References: <CAP_bo1b_2D=fbJJ8uGb8LPWb-6+sTQn1Gsh9YAp8pFs3JY_rrw@mail.gmail.com> <CAOyVPHTLYv=-GbjimpDr5NsxMUeWKtVKzStY9yxQO7s4YD2Ywg@mail.gmail.com> <CAP_bo1Ya7p+OS7fS40jE4+UZuhmeO+MAroC=CZK5sMEE625z8Q@mail.gmail.com> <CAOyVPHTcFr7F4ymQyXyECtS6f8z1XyZn40a_5WcpcjF9y0hZvQ@mail.gmail.com> <CA+-tSzx6DGPptGdtx5awzhnPPJgRHow2SWfuwRP4rwjdN1MXmw@mail.gmail.com> <CAOyVPHRUFrm2xqwrd4OVQbRotae+3+E8xhOF4n1dmWERVdLPEg@mail.gmail.com> <CA+-tSzzvj=eUYT4ZOKiy9yGssmrx71eby2f1xkKKh4NkXL5-Vg@mail.gmail.com> <CAOyVPHS-OF8+GRpmcAxbCj5_HEvgVSOvRMA2hC66v1pxs526Nw@mail.gmail.com> <35BAFA1F-25E8-442E-8FE6-2D5691DCBEAC@kumari.net>
Cc: "armd@ietf.org" <armd@ietf.org>
Subject: Re: [armd] soliciting typical network designs for ARMD
X-BeenThere: armd@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion of issues associated with large amount of virtual machines being introduced in data centers and virtual hosts introduced by Cloud Computing." <armd.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/armd>, <mailto:armd-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/armd>
List-Post: <mailto:armd@ietf.org>
List-Help: <mailto:armd-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/armd>, <mailto:armd-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 31 Aug 2011 19:35:36 -0000

Warren, 

Does the referenced design map the Multicast MAC to Multicast IP?  Or map one multicast in MAC to multiple uni-cast messages? 

Linda

> -----Original Message-----
> From: armd-bounces@ietf.org [mailto:armd-bounces@ietf.org] On Behalf Of
> Warren Kumari
> Sent: Wednesday, August 31, 2011 2:16 PM
> To: Vishwas Manral
> Cc: armd@ietf.org
> Subject: Re: [armd] soliciting typical network designs for ARMD
> 
> 
> On Aug 11, 2011, at 11:40 PM, Vishwas Manral wrote:
> 
> > Hi Linda/ Anoop,
> >
> > Here is the example of the design I was talking about, as defined by
> google.
> 
> Just a clarification -- s/as defined by google/as described by someone
> who happens to work for google/
> 
> W
> 
> > http://www.ietf.org/id/draft-wkumari-dcops-l3-vmmobility-00.txt
> >
> > Thanks,
> > Vishwas
> > On Tue, Aug 9, 2011 at 2:50 PM, Anoop Ghanwani <anoop@alumni.duke.edu>
> wrote:
> >
> > >>>>
> > (though I think if there was a standard way to map Multicast MAC to
> Multicast IP, they could probably use such a standard mechanisms).
> > >>>>
> >
> > They can do that, but then this imposes requirements on the
> > equipment to be able to do multicast forwarding, and even if does,
> > because of pruning requirements the number of groups would be
> > very large.  The average data center switch probably won't handle
> > that many groups.
> >
> > On Tue, Aug 9, 2011 at 2:41 PM, Vishwas Manral
> <vishwas.ietf@gmail.com> wrote:
> > Hi Anoop,
> >
> > From what I know they do not use Multicast GRE (I hear the extra 4
> bytes in the GRE header is a proprietery extension).
> >
> > I think a directory based mechanism is what is used (though I think
> if there was a standard way to map Multicast MAC to Multicast IP, they
> could probably use such a standard mechanisms).
> >
> > Thanks,
> > Vishwas
> > On Tue, Aug 9, 2011 at 2:03 PM, Anoop Ghanwani <anoop@alumni.duke.edu>
> wrote:
> > Hi Vishwas,
> >
> > How do they get multicast through the network in that case?
> > Are they planning to use multicast GRE, or just use directory
> > based lookups and not worry about multicast applications
> > for now?
> >
> > Anoop
> >
> > On Tue, Aug 9, 2011 at 1:27 PM, Vishwas Manral
> <vishwas.ietf@gmail.com> wrote:
> > Hi Linda,
> >
> > The data packets can be tunnelled at the ToR over say a GRE packet
> and the core is a Layer-3 core (except for the downstream ports). So we
> could have encapsulation/ decapsulation of L2 over GRE at the ToR.
> >
> > The very same thing can be done at the hypervisor layer too, in which
> case the entire DC network would look like a Layer-3 flat network
> including the ToR to server link and the hypervisor would do the
> tunneling.
> >
> > I am not sure if you got the points above or not. I know cloud OS
> companies that provide the service and have big announced customers.
> >
> > Thanks,
> > Vishwas
> > On Tue, Aug 9, 2011 at 11:51 AM, Linda Dunbar <dunbar.ll@gmail.com>
> wrote:
> > Vishwas,
> >
> > In my mind the bullet 1) in the list refers to ToR switches
> downstream ports (facing servers) running Layer 2 and ToR uplinks ports
> run IP Layer 3.
> >
> > Have you seen data center networks with ToR switches downstream ports
> (i.e. facing servers) enabling IP routing, even though the physical
> links are Ethernet?
> > If yes, we should definitely include it in the ARMD draft.
> >
> > Thanks,
> > Linda
> > On Tue, Aug 9, 2011 at 12:58 PM, Vishwas Manral
> <vishwas.ietf@gmail.com> wrote:
> > Hi Linda,
> > I am unsure what you mean by this, but:
> > 	* layer 3 all the way to TOR (Top of Rack switches),
> > We can also have a heirarchical network, with the core totally Layer-
> 3 (and having seperate routing), from the hosts still in a large Layer-
> 3 subnet. Another aspect could be to have a totally Layer-3 network.
> >
> > The difference between them is the link between the servers and the
> ToR.
> >
> > Thanks,
> > Vishwas
> > On Tue, Aug 9, 2011 at 10:22 AM, Linda Dunbar <dunbar.ll@gmail.com>
> wrote:
> > During the 81st IETF ARMD WG discussion, it was suggested that it is
> necessary to document typical data center network designs so that
> address resolution scaling issues can be properly described. Many data
> center operators have expressed that they can't openly reveal their
> detailed network designs. Therefore, we only want to document anonymous
> designs without too much detail. During the journey of establishing
> ARMD, we have come across the following typical data center network
> designs:
> > 	* layer 3 all the way to TOR (Top of Rack switches),
> > 	* large layer 2 with hundreds (or thousands) of ToRs being
> interconnected by Layer 2. This design will have thousands of hosts
> under the L2/L3 boundary router (s)
> > 	* CLOS design  with thousands of switches. This design will have
> thousands of hosts under the L2/L3 boundary router(s)
> > We have heard that each of the designs above has its own problems.
> ARMD problem statements might need to document DC problems under each
> typical design.
> > Please send feedback to us (either to the armd email list  or to the
> ARMD chair Benson & Linda) to indicate if we have missed any typical
> Data Center network designs.
> >
> > Your contribution can greatly accelerate the progress of ARMD WG.
> >
> > Thank you very much.
> >
> > Linda & Benson
> >
> >
> 
> _______________________________________________
> armd mailing list
> armd@ietf.org
> https://www.ietf.org/mailman/listinfo/armd