Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

[Vishwas Manral <[e-mail]>]

Murari think IP mobility. :)

On Thu, Sep 22, 2011 at 5:00 PM, Murari Sridharan <[e-mail]>wrote:

>  Do you have a scenario in mind?
> ------------------------------
> From: Vishwas Manral
> Sent: 9/22/2011 4:55 PM
>
> To: Murari Sridharan
> Cc: Narasimhan Venkataramaiah; Linda Dunbar; [e-mail];
> [e-mail]
> Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00"
> advertise its external facing hosts' IP addresses to external world?
>
>  Hi Murari,
>
> Yes that is what I mean.
>
> Thanks,
> Vishwas
> On Thu, Sep 22, 2011 at 4:50 PM, Murari Sridharan <[e-mail]>wrote:
>
>>  You mean not an Ethernet frame but some IP payload?****
>>
>> ** **
>>
>> *From:* Vishwas Manral [mailto:[e-mail]]
>> *Sent:* Thursday, September 22, 2011 4:49 PM
>> *To:* Murari Sridharan
>> *Cc:* Narasimhan Venkataramaiah; Linda Dunbar; [e-mail];
>> [e-mail]
>>
>> *Subject:* Re: [armd] how does "draft-sridharan-virtualization-nvgre-00"
>> advertise its external facing hosts' IP addresses to external world?****
>>
>>   ** **
>>
>> Murari,****
>>
>>  ****
>>
>> What I am saying is the inner header should be allowed to be L3. ****
>>
>>  ****
>>
>> From the diagram you have that does not seem to be the case. Am I missing
>> it totally?****
>>
>>  ****
>>
>> Thanks,****
>>
>> Vishwas****
>>
>> On Thu, Sep 22, 2011 at 4:43 PM, Murari Sridharan <[e-mail]>
>> wrote:****
>>
>> Vishwas, Thanks for the feedback we will definitely consider adding that.
>> I am not sure what you mean by doing L3 instead of L2. We allow any
>> arbitrary virtual topology including L3. ****
>>
>>  ****
>>
>> Thanks****
>>
>>  ****
>>
>> *From:* Vishwas Manral [mailto:[e-mail]]
>> *Sent:* Thursday, September 22, 2011 4:19 PM ****
>>
>>
>> *To:* Narasimhan Venkataramaiah
>> *Cc:* Linda Dunbar; Murari Sridharan; [e-mail]; [e-mail]
>> *Subject:* Re: [armd] how does "draft-sridharan-virtualization-nvgre-00"
>> advertise its external facing hosts' IP addresses to external world?****
>>
>>  ****
>>
>> Hi Simha,****
>>
>>  ****
>>
>> I see this as the only difference between VXLAN and the NVGRE solution
>> (besides ofcourse that TNI needs to be parsed in the intermediate device for
>> hashing and using lesser number of bytes).****
>>
>>  ****
>>
>> I would think you should add it to your draft immediately. With
>> tunneling you consolidate the addresses visible to the core and by providing
>> a hash mechanism, you are providing some level of randomness.****
>>
>>  ****
>>
>> The other thing you should look at is L3 (IPv4/ IPv6) over NVGRE instead
>> of L2 alone. I guess it would be the same comment for the VXLAN proposal
>> too.****
>>
>>  ****
>>
>> Thanks,****
>>
>> Vishwas****
>>
>> On Thu, Sep 22, 2011 at 4:11 PM, Narasimhan Venkataramaiah <
>> [e-mail]> wrote:****
>>
>> The draft mentions exactly this as one use of the reserved 8 bits in
>> Section 4. An NVGRE endpoint could use the 8 bits to further distribute
>> flows belonging to a particular TNI and the switches use all 32 bits to get
>> entropy. One step further would be for the switches to get full entropy from
>> the inner Ethernet frame. I take it that your comment would be to make it
>> explicit in the draft. Right?****
>>
>>  ****
>>
>> One****
>>
>>    such example could be to use the upper 8 bits of the Key field to****
>>
>>    add flow based entropy and tag all the packets from a flow with an
>> entropy label.****
>>
>>  ****
>>
>> Simha****
>>
>>  ****
>>
>> *From:* Vishwas Manral [mailto:[e-mail]]
>> *Sent:* Thursday, September 22, 2011 4:04 PM
>> *To:* Narasimhan Venkataramaiah
>> *Cc:* Linda Dunbar; Murari Sridharan; [e-mail]; [e-mail]
>> *Subject:* Re: [armd] how does "draft-sridharan-virtualization-nvgre-00"
>> advertise its external facing hosts' IP addresses to external world?****
>>
>>  ****
>>
>> Hi Simha,****
>>
>>  ****
>>
>> The main (Standards Track) change in your draft is the addition of TNI.**
>> **
>>
>>  ****
>>
>> A question I have is a TNI identifies a particular tenant and all flows
>> from/to a tenant will be hashed to the same path (even with the changes in
>> switches to do hashing to use TNI).****
>>
>>  ****
>>
>> Why do you not use the last 8 bits which you have kept as reserved for
>> providing the randomization for hashing flows between same to/from on
>> different paths?****
>>
>>  ****
>>
>> Thanks,****
>>
>> Vishwas****
>>
>> On Sun, Sep 18, 2011 at 11:01 AM, Narasimhan Venkataramaiah <
>> [e-mail]> wrote:****
>>
>> The easiest from the point of view of configuration would be to route
>> everything back through the enterprise - not necessarily the optimal from
>> the enterprise point of view. Are you referring to a scenario where the VMs
>> subnet is split between the cloud and the enterprise? Otherwise I don't see
>> the implication on virtualization as its no different than getting the
>> traffic routed to the enterprise in the first case.
>>
>> Simha
>>
>> ________________________________________
>> From: [e-mail] [[e-mail]] on behalf of Linda
>> Dunbar [[e-mail]]
>> Sent: Sunday, September 18, 2011 7:06 AM
>> To: Murari Sridharan; [e-mail]; [e-mail]
>> Subject: [armd] how does "draft-sridharan-virtualization-nvgre-00"
>> advertise its external facing hosts' IP addresses to external world?****
>>
>>
>> Hi Murari,
>>
>> Thank you very much for sharing the presentation.
>>
>> One question:
>>
>> For a host within an Enterprise site which needs to communicate with
>> external peers, the host either uses public IP address which is visible to
>> external peers or uses private IP address which is translated to public
>> address at the Enterprise site's gateway.
>>
>> When this host is moved to "Cloud data center", will the "Cloud Data
>> center" advertise this host address to external peers? Or will all external
>> peers go through enterprise's gateway to reach this host which is no longer
>> residing in the enterprise site?
>>
>> Thanks, Linda
>>
>> > -----Original Message-----
>> > From: [e-mail] [mailto:[e-mail]] On Behalf Of
>> > Murari Sridharan
>> > Sent: Saturday, September 17, 2011 3:02 PM
>> > To: [e-mail]; [e-mail]
>> > Subject: Re: [armd] soliciting typical network designs for ARMD
>> >
>> > FYI, here is a talk that I gave last week in relation to the nvgre
>> > draft below.
>> > http://channel9.msdn.com/Events/BUILD/BUILD2011/SAC-442T
>> >
>> > Thanks
>> > Murari
>> >
>> > -----Original Message-----
>> > From: [e-mail] [mailto:[e-mail]] On Behalf Of
>> > [e-mail]
>> > Sent: Friday, September 16, 2011 6:14 AM
>> > To: [e-mail]
>> > Subject: Re: [armd] soliciting typical network designs for ARMD
>> >
>> > And two more drafts on this topic:
>> >
>> > http://www.ietf.org/id/draft-mahalingam-dutt-dcops-vxlan-00.txt
>> > http://www.ietf.org/id/draft-sridharan-virtualization-nvgre-00.txt
>> >
>> > The edge switches could be the software switches in hypervisors.
>> >
>> > Thanks,
>> > --David
>> >
>> >
>> > > -----Original Message-----
>> > > From: [e-mail] [mailto:[e-mail]] On Behalf
>> > > Of Warren Kumari
>> > > Sent: Wednesday, August 31, 2011 3:16 PM
>> > > To: Vishwas Manral
>> > > Cc: [e-mail]
>> > > Subject: Re: [armd] soliciting typical network designs for ARMD
>> > >
>> > >
>> > > On Aug 11, 2011, at 11:40 PM, Vishwas Manral wrote:
>> > >
>> > > > Hi Linda/ Anoop,
>> > > >
>> > > > Here is the example of the design I was talking about, as defined
>> > by google.
>> > >
>> > > Just a clarification -- s/as defined by google/as described by
>> > someone
>> > > who happens to work for google/
>> > >
>> > > W
>> > >
>> > > > http://www.ietf.org/id/draft-wkumari-dcops-l3-vmmobility-00.txt
>> > > >
>> > > > Thanks,
>> > > > Vishwas
>> > > > On Tue, Aug 9, 2011 at 2:50 PM, Anoop Ghanwani
>> > <[e-mail]> wrote:
>> > > >
>> > > > >>>>
>> > > > (though I think if there was a standard way to map Multicast MAC to
>> > > > Multicast IP, they could
>> > > probably use such a standard mechanisms).
>> > > > >>>>
>> > > >
>> > > > They can do that, but then this imposes requirements on the
>> > > > equipment to be able to do multicast forwarding, and even if does,
>> > > > because of pruning requirements the number of groups would be very
>> > > > large.  The average data center switch probably won't handle that
>> > > > many groups.
>> > > >
>> > > > On Tue, Aug 9, 2011 at 2:41 PM, Vishwas Manral
>> > <[e-mail]> wrote:
>> > > > Hi Anoop,
>> > > >
>> > > > From what I know they do not use Multicast GRE (I hear the extra 4
>> > > > bytes in the GRE header is a
>> > > proprietery extension).
>> > > >
>> > > > I think a directory based mechanism is what is used (though I think
>> > > > if there was a standard way to
>> > > map Multicast MAC to Multicast IP, they could probably use such a
>> > standard mechanisms).
>> > > >
>> > > > Thanks,
>> > > > Vishwas
>> > > > On Tue, Aug 9, 2011 at 2:03 PM, Anoop Ghanwani
>> > <[e-mail]> wrote:
>> > > > Hi Vishwas,
>> > > >
>> > > > How do they get multicast through the network in that case?
>> > > > Are they planning to use multicast GRE, or just use directory based
>> > > > lookups and not worry about multicast applications for now?
>> > > >
>> > > > Anoop
>> > > >
>> > > > On Tue, Aug 9, 2011 at 1:27 PM, Vishwas Manral
>> > <[e-mail]> wrote:
>> > > > Hi Linda,
>> > > >
>> > > > The data packets can be tunnelled at the ToR over say a GRE packet
>> > > > and the core is a Layer-3 core
>> > > (except for the downstream ports). So we could have encapsulation/
>> > > decapsulation of L2 over GRE at the ToR.
>> > > >
>> > > > The very same thing can be done at the hypervisor layer too, in
>> > > > which case the entire DC network
>> > > would look like a Layer-3 flat network including the ToR to server
>> > > link and the hypervisor would do the tunneling.
>> > > >
>> > > > I am not sure if you got the points above or not. I know cloud OS
>> > > > companies that provide the service
>> > > and have big announced customers.
>> > > >
>> > > > Thanks,
>> > > > Vishwas
>> > > > On Tue, Aug 9, 2011 at 11:51 AM, Linda Dunbar <[e-mail]>
>> > wrote:
>> > > > Vishwas,
>> > > >
>> > > > In my mind the bullet 1) in the list refers to ToR switches
>> > > > downstream ports (facing servers)
>> > > running Layer 2 and ToR uplinks ports run IP Layer 3.
>> > > >
>> > > > Have you seen data center networks with ToR switches downstream
>> > > > ports (i.e. facing servers) enabling
>> > > IP routing, even though the physical links are Ethernet?
>> > > > If yes, we should definitely include it in the ARMD draft.
>> > > >
>> > > > Thanks,
>> > > > Linda
>> > > > On Tue, Aug 9, 2011 at 12:58 PM, Vishwas Manral
>> > <[e-mail]> wrote:
>> > > > Hi Linda,
>> > > > I am unsure what you mean by this, but:
>> > > >   * layer 3 all the way to TOR (Top of Rack switches), We can also
>> > > > have a heirarchical network, with the core totally Layer-3 (and
>> > > > having seperate
>> > > routing), from the hosts still in a large Layer-3 subnet. Another
>> > > aspect could be to have a totally
>> > > Layer-3 network.
>> > > >
>> > > > The difference between them is the link between the servers and the
>> > ToR.
>> > > >
>> > > > Thanks,
>> > > > Vishwas
>> > > > On Tue, Aug 9, 2011 at 10:22 AM, Linda Dunbar <[e-mail]>
>> > wrote:
>> > > > During the 81st IETF ARMD WG discussion, it was suggested that it
>> > is
>> > > > necessary to document typical
>> > > data center network designs so that address resolution scaling issues
>> > > can be properly described. Many data center operators have expressed
>> > that they can't openly reveal their detailed network designs.
>> > > Therefore, we only want to document anonymous designs without too
>> > much
>> > > detail. During the journey of establishing ARMD, we have come across
>> > the following typical data center network designs:
>> > > >   * layer 3 all the way to TOR (Top of Rack switches),
>> > > >   * large layer 2 with hundreds (or thousands) of ToRs being
>> > > > interconnected by Layer 2. This
>> > > design will have thousands of hosts under the L2/L3 boundary router
>> > > (s)
>> > > >   * CLOS design  with thousands of switches. This design will have
>> > > > thousands of hosts under the
>> > > L2/L3 boundary router(s)
>> > > > We have heard that each of the designs above has its own problems.
>> > > > ARMD problem statements might
>> > > need to document DC problems under each typical design.
>> > > > Please send feedback to us (either to the armd email list  or to
>> > the
>> > > > ARMD chair Benson & Linda) to
>> > > indicate if we have missed any typical Data Center network designs.
>> > > >
>> > > > Your contribution can greatly accelerate the progress of ARMD WG.
>> > > >
>> > > > Thank you very much.
>> > > >
>> > > > Linda & Benson
>> > > >****
>>
>>  ****
>>
>> ** **
>>
>>
>