Re: [rbridge] Can someone explain Multi-Topology...its purpose, use, etc?

inline below

> Hi Jon,
>
>
>> 1.) For SANs it's just about logical separation. It's about physical
>> separation. No electrons bumping into each other between path A and path B.
>>
>> I have customers today that do this today with FCoE. This issue becomes
>> that you now have two interfaces on a host from two different networks.
>> FCoE doesn't care as the multipathing driver takes care of it, and it's all
>> layer 2. However many applications are not designed to listen on two
>> different networks. There has been talk of a vRouter at a hypervisor level
>> to solve this for virtual hosts, but it's early.
>>
> I know SAN vendors would want to have separate interfaces altogether on
> servers. However using a CNA and a single Ethernet interface on the host,
> the nirvana we were supposed to achieve with FCoE.
>
> I agree in a lot of cases even if we use FCoE, we would want separate
> networks (after first hop) for different kinds of traffic. In which case we
> would have some parts of the network, that could  be common and some that
> would not be. Am I missing the point you are making?
>

The One Cable idea is a myth. It's nice Marketing. Not even one card with
dual ports will do. Two cards, one cable each is the minimum for most
customers I work with. This is about failure domains, statistics & MTBFs.
This has nothing to do with what SAN vendors want. This has to do with IT
folks who will lose their job if they lose data. If a SAN vendor sells a
dual switch setup connected or air separated, they still just sell 2
switches. This comes from the paranoia and experience of running
datacenters. The last time I had filesystem damage due to single path
failure I was restoring data for 26hrs. It's not worth it.

If you are going to be accessing a remote filesystem most customers have
learned that two physically separate paths, are the only way to provide
permanent availability.

but keep in mind, I work with some very large customers, your local flower
shop might feel very differently.

>
>>
>> 2.) While I agree with Donald that TRILL should play well with DCB. The
>> fact is that both Cisco and Brocade offer FCoE over their "TRILL-ish"
>> products. And they both only support FCoE with DCB. So you can use PFC &
>> ETS without interacting with TRILL at all. Not saying that it's optimal,
>> but I can personally say it works.
>>
>> I guess what you are talking about is not end-to-end Ethernet forwarding
> with FCoE, but a hop by hop, where the forwarding is done by FCF (and not
> Ethernet switching).
>

> We are talking about a case where we have Ethernet forwarding as a lot of
> the Enterprises may like. We do not need TRILL to be DCB aware except for
> the case of end-to-end messaging like PFC.
>
>
I am talking about both "sparse" and "dense" style FCoE. TRILL can provide
a multi-hop multi-path layer for which DCB runs on top. PFC can send it's
pause frames and cause upper level protocols to do what they need to and
TRILL just keeps on moving.

It's like freeways today. Do freeways need to be aware of traffic flow and
congestion to work? No. Would they work much better with that knowledge.
Yes.

> 3.) If you are from the camp that believes in CN then you really also want
>> some interaction between TRILL and CN. There is a lot of value in that.
>>
>> Correct.
>
>
>> However since there are no working implementations of CN today, it's not
>> yet a problem.
>>
> Ok.
>
>
>>
>> While there is not a lot of FCoE deployed today, there is some. My guess
>> is less than 100 production sites, but that is a guess.
>>
>> Do you mean end-to-end FCoE here or do you mean single hop?
>

I mean both TOR->FC single hop and TOR->Aggregation/Core-> FC dual hop.

This does not include all the blades servers out there using FCoE
internally.

The number could be higher, but the truth is no one running FCoE wants
anyone to know they are running FCoE so getting data you can talk about is
very difficult =)
On Mon, Mar 5, 2012 at 12:52 PM, Vishwas Manral <vishwas.ietf@gmail.com>wrote:

> Hi Jon,
>
>
>> 1.) For SANs it's just about logical separation. It's about physical
>> separation. No electrons bumping into each other between path A and path B.
>>
>> I have customers today that do this today with FCoE. This issue becomes
>> that you now have two interfaces on a host from two different networks.
>> FCoE doesn't care as the multipathing driver takes care of it, and it's all
>> layer 2. However many applications are not designed to listen on two
>> different networks. There has been talk of a vRouter at a hypervisor level
>> to solve this for virtual hosts, but it's early.
>>
> I know SAN vendors would want to have separate interfaces altogether on
> servers. However using a CNA and a single Ethernet interface on the host,
> the nirvana we were supposed to achieve with FCoE.
>
> I agree in a lot of cases even if we use FCoE, we would want separate
> networks (after first hop) for different kinds of traffic. In which case we
> would have some parts of the network, that could  be common and some that
> would not be. Am I missing the point you are making?
>
>
>>
>> 2.) While I agree with Donald that TRILL should play well with DCB. The
>> fact is that both Cisco and Brocade offer FCoE over their "TRILL-ish"
>> products. And they both only support FCoE with DCB. So you can use PFC &
>> ETS without interacting with TRILL at all. Not saying that it's optimal,
>> but I can personally say it works.
>>
>> I guess what you are talking about is not end-to-end Ethernet forwarding
> with FCoE, but a hop by hop, where the forwarding is done by FCF (and not
> Ethernet switching).
>
> We are talking about a case where we have Ethernet forwarding as a lot of
> the Enterprises may like. We do not need TRILL to be DCB aware except for
> the case of end-to-end messaging like PFC.
>
>
>> 3.) If you are from the camp that believes in CN then you really also
>> want some interaction between TRILL and CN. There is a lot of value in
>> that.
>>
>> Correct.
>
>
>> However since there are no working implementations of CN today, it's not
>> yet a problem.
>>
> Ok.
>
>
>>
>> While there is not a lot of FCoE deployed today, there is some. My guess
>> is less than 100 production sites, but that is a guess.
>>
>> Do you mean end-to-end FCoE here or do you mean single hop?
>
>> -J
>>
>
> Thanks,
> Vishwas
>
>
>>
>> On Fri, Mar 2, 2012 at 1:49 PM, Vishwas Manral <vishwas.ietf@gmail.com>wrote:
>>
>>> Agree with you totally here Donald.
>>>
>>> -Vishwas
>>>
>>>
>>> On Fri, Mar 2, 2012 at 12:23 PM, Donald Eastlake <d3e3e3@gmail.com>wrote:
>>>
>>>> Some people think congestion notification (CN) is important and some
>>>> think it isn't.
>>>>
>>>> If you are just talking about Priority Based Flow control (PFC, aka
>>>> "per priority pause") or just PFC and ETS, they are down in the
>>>> queueing control at ports and orthogonal to what sort of switch
>>>> protocol you are running above the ports. Could be a bridge, a layer 3
>>>> router, an RBridge, or whatever. PFC and ETS are pretty much the same.
>>>>
>>>> Thanks,
>>>> Donald
>>>> =============================
>>>>  Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
>>>>  155 Beaver Street, Milford, MA 01757 USA
>>>>  d3e3e3@gmail.com
>>>>
>>>> On Fri, Mar 2, 2012 at 2:48 PM,  <david.black@emc.com> wrote:
>>>> > That was a timely submission.
>>>> >
>>>> > My understanding is that while congestion notification was originally
>>>> part of DCB, that's
>>>> > no longer considered to be the case.  Deployments of DCB Ethernet for
>>>> FCoE typically do
>>>> > not include congestion notification.
>>>> >
>>>> > Thanks,
>>>> > --David
>>>> >
>>>> >> -----Original Message-----
>>>> >> From: Donald Eastlake [mailto:d3e3e3@gmail.com]
>>>> >> Sent: Friday, March 02, 2012 2:39 PM
>>>> >> To: Black, David
>>>> >> Cc: narten@us.ibm.com; rbridge@postel.org
>>>> >> Subject: Re: [rbridge] Can someone explain Multi-Topology...its
>>>> purpose, use, etc?
>>>> >>
>>>> >> On Fri, Mar 2, 2012 at 1:21 PM,  <david.black@emc.com> wrote:
>>>> >> > Note that using FCoE over TRILL essentially requires that TRILL
>>>> support DCB Ethernet.
>>>> >>
>>>> >> See
>>>> https://datatracker.ietf.org/doc/draft-eastlake-trill-rbridge-dcb/
>>>> >>
>>>> >> Thanks,
>>>> >> Donald
>>>> >> =============================
>>>> >>  Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
>>>> >>  155 Beaver Street, Milford, MA 01757 USA
>>>> >>  d3e3e3@gmail.com
>>>> >>
>>>> >> > Thanks,
>>>> >> > --David
>>>> >> >
>>>> >> >> -----Original Message-----
>>>> >> >> From: rbridge-bounces@postel.org [mailto:
>>>> rbridge-bounces@postel.org] On Behalf Of Thomas Narten
>>>> >> >> Sent: Friday, March 02, 2012 8:26 AM
>>>> >> >> To: Radia Perlman
>>>> >> >> Cc: rbridge@postel.org
>>>> >> >> Subject: Re: [rbridge] Can someone explain Multi-Topology...its
>>>> purpose, use, etc?
>>>> >> >>
>>>> >> >> > 3) Specifically for TRILL, why would multi-topology be useful?
>>>> >> >>
>>>> >> >> Disjoint multipathing for FCoE. Today, many FC deployments use
>>>> >> >> physically replicated SANs. Twice the cabling/cost/etc.
>>>> >> >>
>>>> >> >> But, there is real value in terms of fault isolation in having
>>>> >> >> completely disjoint paths.
>>>> >> >>
>>>> >> >> TRILL, today, does not really do this.
>>>> >> >>
>>>> >> >> Thomas
>>>> >> >>
>>>> >> >> _______________________________________________
>>>> >> >> rbridge mailing list
>>>> >> >> rbridge@postel.org
>>>> >> >> http://mailman.postel.org/mailman/listinfo/rbridge
>>>> >> >
>>>> >> >
>>>> >> > _______________________________________________
>>>> >> > rbridge mailing list
>>>> >> > rbridge@postel.org
>>>> >> > http://mailman.postel.org/mailman/listinfo/rbridge
>>>> >
>>>>
>>>> _______________________________________________
>>>> rbridge mailing list
>>>> rbridge@postel.org
>>>> http://mailman.postel.org/mailman/listinfo/rbridge
>>>>
>>>
>>>
>>> _______________________________________________
>>> rbridge mailing list
>>> rbridge@postel.org
>>> http://mailman.postel.org/mailman/listinfo/rbridge
>>>
>>>
>>
>>
>> --
>> "Do not lie. And do not do what you hate."
>> "Don't worry, be happy"
>>
>
>

-- 
"Do not lie. And do not do what you hate."
"Don't worry, be happy"