Re: [armd] soliciting typical network designs for ARMD

[Vishwas Manral <[e-mail]>]

Hi Linda/ Anoop,

Here is the example of the design I was talking about, as defined by google.
http://www.ietf.org/id/draft-wkumari-dcops-l3-vmmobility-00.txt

Thanks,
Vishwas
On Tue, Aug 9, 2011 at 2:50 PM, Anoop Ghanwani <[e-mail]>wrote:

>
> >>>>
> (though I think if there was a standard way to map Multicast MAC to
> Multicast IP, they could probably use such a standard mechanisms).
> >>>>
>
> They can do that, but then this imposes requirements on the
> equipment to be able to do multicast forwarding, and even if does,
> because of pruning requirements the number of groups would be
> very large.  The average data center switch probably won't handle
> that many groups.
>
> On Tue, Aug 9, 2011 at 2:41 PM, Vishwas Manral <[e-mail]>wrote:
>
>> Hi Anoop,
>>
>> From what I know they do not use Multicast GRE (I hear the extra 4 bytes
>> in the GRE header is a proprietery extension).
>>
>> I think a directory based mechanism is what is used (though I think if
>> there was a standard way to map Multicast MAC to Multicast IP, they could
>> probably use such a standard mechanisms).
>>
>> Thanks,
>> Vishwas
>>   On Tue, Aug 9, 2011 at 2:03 PM, Anoop Ghanwani <[e-mail]>wrote:
>>
>>> Hi Vishwas,
>>>
>>> How do they get multicast through the network in that case?
>>> Are they planning to use multicast GRE, or just use directory
>>> based lookups and not worry about multicast applications
>>> for now?
>>>
>>> Anoop
>>>
>>>   On Tue, Aug 9, 2011 at 1:27 PM, Vishwas Manral <[e-mail]
>>> > wrote:
>>>
>>>>   Hi Linda,
>>>>
>>>> The data packets can be tunnelled at the ToR over say a GRE packet and
>>>> the core is a Layer-3 core (except for the downstream ports). So we could
>>>> have encapsulation/ decapsulation of L2 over GRE at the ToR.
>>>>
>>>> The very same thing can be done at the hypervisor layer too, in which
>>>> case the entire DC network would look like a Layer-3 flat network including
>>>> the ToR to server link and the hypervisor would do the tunneling.
>>>>
>>>> I am not sure if you got the points above or not. I know cloud OS
>>>> companies that provide the service and have big announced customers.
>>>>
>>>> Thanks,
>>>> Vishwas
>>>>   On Tue, Aug 9, 2011 at 11:51 AM, Linda Dunbar <[e-mail]>wrote:
>>>>
>>>>> Vishwas,
>>>>>
>>>>> In my mind the bullet 1) in the list refers to ToR switches downstream
>>>>> ports (facing servers) running Layer 2 and ToR uplinks ports run IP Layer 3.
>>>>>
>>>>>
>>>>> Have you seen data center networks with ToR switches downstream ports
>>>>> (i.e. facing servers) enabling IP routing, even though the physical links
>>>>> are Ethernet?
>>>>> If yes, we should definitely include it in the ARMD draft.
>>>>>
>>>>> Thanks,
>>>>> Linda
>>>>>   On Tue, Aug 9, 2011 at 12:58 PM, Vishwas Manral <
>>>>> [e-mail]> wrote:
>>>>>
>>>>>> Hi Linda,
>>>>>> I am unsure what you mean by this, but:
>>>>>>
>>>>>>    1. layer 3 all the way to TOR (Top of Rack switches),
>>>>>>
>>>>>> We can also have a heirarchical network, with the core totally Layer-3
>>>>>> (and having seperate routing), from the hosts still in a large Layer-3
>>>>>> subnet. Another aspect could be to have a totally Layer-3 network.
>>>>>>
>>>>>> The difference between them is the link between the servers and the
>>>>>> ToR.
>>>>>>
>>>>>> Thanks,
>>>>>> Vishwas
>>>>>>   On Tue, Aug 9, 2011 at 10:22 AM, Linda Dunbar <[e-mail]>wrote:
>>>>>>
>>>>>>> During the 81st IETF ARMD WG discussion, it was suggested that it is
>>>>>>> necessary to document typical data center network designs so that address
>>>>>>> resolution scaling issues can be properly described. Many data center
>>>>>>> operators have expressed that they can't openly reveal their detailed
>>>>>>> network designs. Therefore, we only want to document anonymous designs
>>>>>>> without too much detail. During the journey of establishing ARMD, we have
>>>>>>> come across the following typical data center network designs:
>>>>>>>
>>>>>>>    1. layer 3 all the way to TOR (Top of Rack switches),
>>>>>>>    2. large layer 2 with hundreds (or thousands) of ToRs being
>>>>>>>    interconnected by Layer 2. This design will have thousands of hosts under
>>>>>>>    the L2/L3 boundary router (s)
>>>>>>>    3. CLOS design  with thousands of switches. This design will have
>>>>>>>    thousands of hosts under the L2/L3 boundary router(s)
>>>>>>>
>>>>>>> We have heard that each of the designs above has its own problems.
>>>>>>> ARMD problem statements might need to document DC problems under each
>>>>>>> typical design.
>>>>>>> Please send feedback to us (either to the armd email list  or to the
>>>>>>> ARMD chair Benson & Linda) to indicate if we have missed any typical Data
>>>>>>> Center network designs.
>>>>>>>
>>>>>>> Your contribution can greatly accelerate the progress of ARMD WG.
>>>>>>>
>>>>>>> Thank you very much.
>>>>>>>
>>>>>>> Linda & Benson
>>>>>>>
>>>>>>
>