Re: [Tsvwg] RE: multihoming issues via SCTP

[Randall Stewart <[e-mail]>]

Ivan:

Ok it is time for me to jump in here .. having just completed
most of the kernel output routines for SCTP in the KAME-SCTP
stack I am working on.. (note: I have rough framed them but
they hopefully will complile by the end of today :0)


[e-mail] wrote:
> 
>         Hi all!
> 
>         See some comments inline...
> 
> > > > > This means there have to be more paths:
> > >
> > > > > A1->X->Y->B1
> > > > > A1->X->B2
> > > > > A2->Y->B1
> > > > > A2->Y->X->B2
> > >
> > > > yes,  but does routing allow them to be used? We got the
> > > impression that at
> > > > a certain time you can have only one path through the
> > > network for a certain
> > > > destination address.
> > >
> > > If we define a path as being from a source interface to a
> > destination
> > > interface, then the number of paths is only limited by the
> > > product of the
> > > number of interfaces on both sides. Obviously, many of those
> > > paths will
> > > share a lot of infrastructure, but they won't be identical.
> > >
> > > I think the main problem is that in most cases a default
> > route will be
> > > used, which is tied to one interface. That means the paths
> > > originating at
> > > the different interfaces are effectively identical.
> >
> > Hmm, not exactly the effect we are looking for.
> 
>         I was playing with that source address selection issue, and even if
> you can set the source IP address in the outgoing datagrams, the source
> network card (and so the path) used is something SCTP can't control at all.
> So in the worst case, you would be using one network card for the outgoing
> packets (artificially setting the source address, when the corresponding NIC
> is not used because the routing tables say you should use the other one),
> and receiving the SACKs through the other card (as they are directed to the
> source address you set in your outgoing datagram)... In that case,
> multihoming not only doesn't help, but makes things worse (as the failure of
> any of your two source cards/paths would mean that the whole association
> will be down...) :-/

It is really quite SILLY to think that a kernel land SCTP can NOT select
the source address. Even most user land version can do this since it is
using raw sockets. I just finished writting a "source-address-selection"
module for the SCTP stack I am working on it:

If bound to all addresses
A) Will set the source address to 0, allowing the IP stack
   to select the address of the interface that it emmits the
   packet on. This is how UDP packets bound to all work as well.

If not bound to all interaces
B) If bound to a sub-set of the machines addresses it will generate
   the route and then look to see if the interface is one of the
   bound subset. If so it SETS the source address to the address
   of this outbound interface.

C) If we are bound to a sub-set and the interface we will emmit
   the packet on is NOT bound, then the implementation will rotate
   the source address amongst its bound interfaces by setting the
   source address of the packet round robin amongst the bound
   interfaces.

Now TCP can NOT do this since it only has one address bound. Where
the packet is emitted which has nothing to do with the source address
selected...since this is set as part of the connection.

Now another issue with this that people keep forgetting is that
in general the source address of the packet DOES NOT HELP the
IP stack choose the interface the packet will be emitted on.
Yes, you can in some instances turn on source based routing
but this IS STILL A REASEARCH issue and is not generally done.

So we launch packets out an interface paying NO attention to
the source address. The source address is ONLY used by
the remote endpoint to:

1) Help find the association
2) Know the prefered return address to use. An endpoint
   MAY NOT use this in SOME cases as documented in RFC2960

Now it is true that the routing infrastructure of the
local host controls a lot of issues... 

If I am using a single default route (a somewhat bad idea
if I am really concerned about redundancy) then I will
get very little benefit from my multi-homing. But has
Andreas as pointed out, if you are going to the extra
effort of putting in two ISP connections and wanting
to communicate across the network to a peer that
is likewise setup, one would expect you to apply a bit
of "engineering sense" and setup routing table entries
to this peer that lead out the appropriate paths and
NOT use the "default route".

One could even image a system that allows two default
routes. FreeBSD currently does not support this .. I
will have to check and see if KAME's modifications do...
With two default routes one could even build some intelegence
into route selection to get the second default route for
an SCTP endpoint... hmmm I need to go have a look at this
and see if I can't put it in my work.... IF KAME has
expanded the ability to have multiple default routes :>
In linux I think this is possible and one could invision
setting up routing entries in SCTP so that two destinations
DON'T share the same default route outwards... this of
course is an implementation issue that I think I will look
into for KAME...

> > >
> > > > > However, this assumes both that the output interface stays
> > > > > the same over
> > > > > the life time of a connection and that the end host is the
> > > > > one making the
> > > > > decision how packets are routed to the outside. Both are
> > > > > unlikely.
> > >
> > > > Well, but that is how it is done with SCTP. SCTP specifies
> > > the destination
> > > > address  and IP should try to route based on that IP address to B.
> > >
> > > Right. So the decision about the output path is done in IP and SCTP
> > > probably can't control this.
> > > > If B1 is taken as destination address then the source
> > > address should be the
> > > > corresponding A1(and that is something done by IP in host A).
> > >
> > > I suppose it is possible to do this, although TCP doesn't.
> 
>         I might be completely wrong, but I strongly believe you can't...
> unless you go outside SCTP and change your routing tables...

see above :>

> 
> > It is not TCP nor SCTP that have to do selection of the
> > source address, They
> > cannot do this, it is the IP layer who has to do this.
> > >
> > > > > If an
> > > > > output interface fails, hosts may simply reroute outgoing
> > > > > packets over the
> > > > > other interface.
> > >
> > > > That is a possibility, but that depends on who finds it
> > > first. If SCTP
> > > > detects that the interface has gone(via the heartbeats that aren't
> > > > acknowledge), then it will changeover to another
> > > destination address(thus
> > > > taking the other interface)
> > >
> > > The fact that host B is suddenly sending packets to interface
> > > 2 instead of
> > > 1 of host A, doesn't influence outgoing packets on host A.
> 
>         Not for the DATA chunks, but it affects to the SACKs...
> 
> snip...
> 
> > > > > Also, it won't do much good
> > > > > unless there are three or more paths of which at least
> > two fail, a
> > > > > situation that is not likely to be very common.
> 
>         Well... given that the paths used will be in part identical, I don't
> see this as that uncommon...
> 
> > > > Murphy law: Failures of links, interfaces and routers will
> > > happen.(once upon
> > > > a time, we believed this tale that failures wouldn't
> > > happen, but then
> > > > reality and a lot of users thought us otherwise)
> > >
> > > I'm not saying failures won't happen, just that in most cases the
> > > heartbeats won't help you. If there are only two paths,
> > > you'll have to use
> > > the second one when the first one fails. So why bother to see if the
> > > second path is alive? You'll find out soon enough.
> >
> > In my view, if the second path is not alive then everything
> > is allready lost
> > and the net result is that I gain a lot of very, very angry customers.
> > That is my view and , agreed, for some/many/most/all?(make
> > your choice)
> > applications that need not to be the case.
> >
> >
> > > If you
> > > have three paths
> > > and one fails, then each of the others still work so no need to use
> > > heartbeats again. Only when two paths fail at the same time,
> > > it helps to
> > > know which secondary path has also failed so you know to
> > > avoid that one
> > > and pick the other.
> > >
> > > If doing something and doing nothing yield the same results,
> > > the latter is
> > > preferable.
> >
> > Depends on the view taken. If you are happy which the situation as you
> > described then OK, then it is better to do nothing(that is
> > why the heartbeat
> > is optional). But you will find people(like me) doing
> > heartbeats(and yes,
> > there are not many of us)
> 
>         I think heartbeats are useful. If you have two paths, and you are
> using your primary normally, and you discover that the secondary is death,
> you can always reuse the primary for retransmissions (the other one seems to
> be even in worse situation). Or simply, you can discover quicker that the
> peer is unreachable... inform to the upper user, and get another angry
> customer faster :-)
> 
>         When having more than two addresses, if one of the backup addresses
> fails you will know it, and if your timer expires in your primary (which
> doesn't really mean that it is down, maybe just the path is congested) you
> won't send the retransmitted packet to a death address but to the other...
> 
A bunch of us in sigtran (if I remember right including Vern Paxson) did
discuss this very issue on having HB's and if the overhead was worth it.
As I remember it was thought critcal to maintain a good RTO estimate
on the backup path(s) and that this mechanism was not "wasted overhead"
but very good information to maintain.

Originally Vern proposed doing a HB every RTO time with a 1 second
random jitter. I was the one who objected to this and pointed out
that this would be WAY TO MUCH and got the current
RTO+HB.Interval+Jitter
put in. With a HB.Interval around 1 minute or so...can't remember
without looking at  RFC :>

I must agree with Vern (and Dis-agree with you Iljitsch) that HB's
provide VALUABLE information and SHOULD NOT be turned off. As
Michael as stated however, you are more than welcome to use
the API and turn off your HB's on SCTP. I know I will NOT!!

R
> > >
> > > Iljitsch
> > >
> >
> > Seems like we are in between a rock and a hard place. Maybe
> > I'll try to come
> > up with something.
> > Thanks anyway for the comments. They cleared some things for me.
> >
> > Yours sincerly,
> > Lode
> 
>         By the way Lode, wouldn't it be interesting including something
> about the use of IPsec in the draft and the problems with it regarding
> multiple addresses?
> 
>         BR Iván Arias Rodríguez
> 
> _______________________________________________
> tsvwg mailing list
> [e-mail]
> http://www1.ietf.org/mailman/listinfo/tsvwg

-- 
Randall R. Stewart
[e-mail] 815-342-5222 (cell phone)

_______________________________________________
tsvwg mailing list
[e-mail]
http://www1.ietf.org/mailman/listinfo/tsvwg