Re: [re-ECN] FW: ConEx BoF announcement text

[" Ilpo Järvinen " <[e-mail]>]

On Thu, 22 Oct 2009, Fred Baker wrote:

> well, we disagree on some points.

I guess we agree quite much yes. However, what I'm pretty much still 
lacking the connection you made between Cubic/Reno and throughput 
reduction because of "the cliff". I oppose such an universal statement you 
made earlier about the cliff being there if we keep pushing the network. 
E.g., you put earlier Cubic among the "bad ones" who push the network, but 
now your scenario below doesn't have anything that even remotely resembles 
Cubic/Reno but something totally unresponsive (I'm assuming SACK based 
recovery is there with this defination of Reno, and that router buffers 
are large enough to accommodate MD-madness "recovery"). ...That's the 
connection I want to fully tear down. Don't first put Cubic or Reno into 
bads than show an example with something completely different, it isn't 
very fair to blame something from others faults like that and justify 
their "badness" that way. This is very important because this 
mis-connection is there in many minds that when enough congestion, 
throughput automatically dramatically drops (like in the Jain's graph). As 
a result, we hear these "congestion is the problem", and "more aggressive 
means more congestion" type of an attitude. Just to make sure, I didn't 
mean to say here that I'd think that Cubic/Reno is something perfect 
either.

...The rest is just some comments to your rather intimidated scenario :-).

> I don't know that perfect agreement is necessary, and certainly not on 
> this thread. That said...
> 
> Regarding behavior beyond the cliff, it might be instructive to take the most
> extreme possible case - a dumbbell network and a really obnoxious file
> transfer algorithm.
> 
>         +-+ 10*N   +-+ 10*N  +-+   N  +-+ 10*N  +-+
>         |A|--------|B|-------|C|------|D|-------|E|
>         +-+        +-+       +-+      +-+       +-+
>                10*N |         |10*N
>                   +--+      +--+
>                   | F|      | G|
>                   +--+      +--+
> 
> Presume that the capacity from C to D is N, and all other capacities are 10*N.
> Presume that some application at A wants to send something really big to a
> peer at E, and some other application at F wants to send something else to a
> peer at G.
> 
> Presume also that the way file transfers work is that the sender puts
> appropriate headers on all of the indicated packets, queues them up to its
> link layer chip, and spews them as fast as it can. If it's a 100 gigabyte
> file, it queues up 100 gigabytes and watches it heat the "wires". The peer
> receives what it receives and tells the sender what it didn't. The sender now
> repeats the exercise with anything his peer didn't receive.
>
> BTW, that is in fact how a file transfer done in a certain research
> environment on very noisy lines is proposed to work. I didn't just dream that
> up.
> 
> Obviously, links A-B, B-C, F-B, and C-D will be fully utilized. Since B-C has
> less (half) capacity than A-B + F-B, each of those flows will drop about half
> of its traffic. The transfer rate F-G should be around 5*N. C-D will also drop
> 80% of the remaining traffic, resulting in an arrival rate at E of N. A will
> go through at least ten iterations of its transfer, and F at least two.
>
> The first down side is that while A-B and F-B are fully utilized, most 
> of that capacity is wasted.

Yes, that's the nature of unresponsiveness you added by yourself, and now 
you say that it is a down side?!? ...Nobody (including the network) can 
prevent an application from doing stupid things like wasting the resources 
of the first link :-). I understand though that you could construct a more 
complex topology where it wouldn't be the first link.

Instead of tackling that possibility, I take another extreme example to 
stress the point of mine: An application could cause the very same 
symptoms as were with the classical congestion collapse (as per Nagle) 
just by keeping resending just the very same segment even though it made 
through (and ACK was received), only multiple copies are then in the 
network and the throughput of that particular flow will be dramatically 
affected. But it's just plain stupid behavior which no network can prevent 
you from doing. Of course nobody else would have to be part of that 
experiment and they can operate with good throughputs. In a thought 
experiment we can put a TCP implementation with SACK and lots of receiver 
window into the age of Nagle and prove that it keeps operating with very 
good throughput regardless of what the stupid senders do as long as the 
transfer isn't unlucky to run of data to send while recovering.

...Secondly, I don't find this non-optimal use of excess capacity to be 
the reason why one would need/want Re-ECN.

Another stupidity of the application/protocol in this example is the way 
of feedback which exaggerates anomalities nearby the end of a transfer.

> This impacts F's ability to deliver traffic to G; it
> should have achieved 90% throughput rate (the rate C-G less the rate C-D), and
> in fact achieved 50% or less. With the right mechanism (ack clocking comes to
> mind, and there are other approaches), the capacity used by the file transfer
> on A-B is only 10% used and there is at most a 10% impact on the file transfer
> F-G.

...Right, would there be responsiveness, all changes. And I guess we then 
pretty much agree.

> The second down-side is that the traffic arriving at G and E will be 
> scattered all through the respective files, meaning that they have to 
> store the bits and pieces and do a fair bit of searching and sorting to 
> determine what they have not yet received and to reconstitute the file. 
> This is a waste both of memory and CPU resources.

This has nothing to do with the throughput reduction anymore, don't you 
think? Also, to me these seem "extras" again are because of stupid 
behavior introduced by you rather than anything very close to what I was 
arguing against.

But in general I agree with you that loss recovery even without any 
throughput loss introduces these space constraints (though in a lot less 
severe way than in your extreme example) and one would want to avoid that 
if at all possible. But IMHO that again has rather little to do with "the 
cliff" and the throughput reduction (that supposedly happens because of 
it).

> Voila, we come back pretty quickly to wanting the transmission rate of A to
> approximate the reception rate at E and the transmission rate at F to
> approximate the reception rate at G, and wanting the window to not be
> excessively large. An excessively large window wastes bandwidth before the
> bottleneck and potentially creates unnecessary bottlenecks, impacts 
> other file transfers, imposes a memory/algorithm burden on the receiver, 
> and for all of those costs accomplishes nothing that a smaller window 
> would not have accomplished better.

Agreed. ...Noted though that you have for some reason cleverly milded 
the "reduces throughput" into "_potentially_ creates unnecessary 
bottlenecks" (emphasis mine) here which I can certainly agree much
more ;-).

...In fact, I'd even not choke because of your example here, if given in 
less extreme form, as we have CBR type of non-responsiveness present too 
for real, and thus dead packets wasting capacity (without ECN) and so on.

> A window that is too large is a bad thing, and the congestion it causes is a
> bad thing. Not that the presence of congestion is bad - it is, as Bob says, a
> natural side effect of what happens. But congestion that is poorly managed by
> the endpoint and/or the network is a very bad thing. re-ecn does in fact try
> to give the sender an incentive to approximate the knee (the smallest window
> that achieves the potential throughput rate), and increasing it to or beyond
> the cliff by definition doesn't increase it's rate materially beyond that
> point.

Agreed. Like you pointed out earlier, there's no increase in throughput 
beyond that point not matter what trick one tries to play and I agree.
My point was just to say that with responsive transfers there isn't such 
magic cliff (always), but the throughput would just keep that constant 
line even if drops start to occur. And especially mentioning Cubic/Reno, 
which are responsive, in this context of "cliff" seemed very misleading.


--
 i.


> On Oct 22, 2009, at 12:36 PM, Ilpo Järvinen wrote:
> 
> >On Thu, 22 Oct 2009, Fred Baker wrote:
> >
> > >I'll argue something slightly different. It comes to the same endpoint, but
> > >through a different observation. Some of us, you Bob especially, are very
> > >familiar with this; others are perhaps less so. Bear with me.
> > >
> > >Jain's 1994 patent on congestion control defines two terms: the "cliff" and
> > >the "knee" of a throughput curve.
> > >
> > > |
> > >     T   |      availabile capacity
> > >     h   | ---------------------------------------
> > >     r   |       +---+
> > >     u   |  knee/     \cliff
> > >     p   |     /       \
> > >     u   |    /         \
> > >     t   |   /           \
> > > |/             \
> > > |/               \
> > > |/                 ----------------
> > >         -----------------------------------------
> > >                   Window  --->
> > >
> > >As the TCP/SCTP window increases from zero to some value, throughput also
> > >increases. That stops when the available capacity has been consumed; at
> > >that
> > >point, even if the window grows, throughput does not. What increases
> > >instead
> > >is RTT, because a queue grows.
> >
> >Up to this point I agree with this paragraph...
> >
> > >If window continues to increase, at some point the queue starts dropping
> > >traffic, and throughput degrades.
> >
> >...However, this is not a true claim. As long as no unnecessary work is
> >not done at any bottleneck (and we don't have a shared resource such as
> >a wireless channel), thoughput does _not_ degrade (at all!), even though
> >many keep asserting that without much of a thought nowadays. Of course
> >non-infinitely long flows would introduce some anomalities to calculations
> >through which one could use to show that this claim is "true" but I think
> >that's hardly what people usually mean when they make that claim that
> >dropping implies throughput degradation.
> >
> >Somewhat related, for some reason there's a very common misconcept that
> >drops imply poor performance. That is often stated as a fact without
> >looking deeper into what really was the cause for sub-optimal
> >performance, often that turns out to be something else when looked
> >closely enough. I'd say that very often losses were, in fact, innocent.
> >Quite often the blame would fall on constant factor MD (if one is honest
> >enough to admit).
> >
> >Btw, your graph is not accurate (wrt. Jain), he's drawing a much sharper
> >drop than you do at his "cliff" (so with ascii you'd have to use | at
> >first to match him). I've some doubts about validity of that sharp
> >angle too, so I find your graph is much more sensible (that is, as long as
> >the resource is shared).
> >
> > >By definition, the window at the "knee" is smaller than the window at
> > >the "cliff", but throughput is the same.
> > >
> > >Common TCP congestion control algorithms such as Reno and Cubic tune to the
> > >cliff. That has the upside of maximizing throughput; it has the downside
> > >that
> > >it is abusive to other applications such as voice/video (increased and
> > >variable RTT, and induces loss), and to other TCP sessions. #include
> > ><discussion of bittorent and why other users are negatively impacted by it>
> >
> >I think Bob doesn't agree with you here, he isn't saying that congestion
> >is something bad but something nearly opposite. Bob's own words:
> >
> >"It would contradict this to say congestion is a problem - it's not - it's
> >healthy and natural in a data network."
> >
> >...It's just that if you keep doing that too much (beyond the congestion
> >volume allocated for you), you would "suffer (if I've understood him
> >right). ...So it has very little to do with operating in the "cliff" or
> >"knee" in short timescales (unless of course you've already ran out of
> >your quota).
> >
> > >Other TCP congestion control algorithms based on ECN, CalTech FAST, etc,
> > >try
> > >to tune to the knee. This gets them exactly the same throughput including
> > >support of very high rate applications but with a smaller window value and
> > >therefore lower queue depth and lower probability of loss - both for
> > >themselves and their competitors. It does so without negatively, or at
> > >least
> > >AS negatively, impacting applications they compete with beyond seeking to
> > >share the available capacity with them.
> >
> >I certainly agree with you here that low queuing delay is a desirable
> >property too. ...However, I don't see how e.g., ECN alone could achieve
> >low queue delay and high throughput at the same time, you'd need something
> >more than that (mainly to remove the 0.5 factored MD). And after removing
> >that, we'd get something close to what Matt Mathis is suggesting and that
> >then does not anymore depend on ECN to work (though I think one could
> >certainly avoid lots of loss recovering by signalling in early with
> >ECN).
> >
> >-- 
> >i.
> 
> 

-- 
 i.