[Tsvwg] RE: HEADS UP: consensus call on draft-floyd-tsvwg-cc-alt

["Arjuna Sathiaseelan" <[e-mail]>]

Regarding fairness,

One question that may sound silly is when we say we should be fair to TCP,
should we be fair to which version of TCP? TCP Tahoe, Reno, New Reno, SACK
or other versions? I was told that TCP Reno should be the target, but I hear
people saying that New Reno, is the most used version of TCP in the
Internet! And I strongly believe this is also SUBJECT to change. IMHO,
benchmarking fairness with "some" protocol that does not perform well to
"certain" conditions is questionable.

Another thing we need to note is, that the limitations in the routers (in
packets per second or bytes per second etc). So if a protocol "X" sends 4
packets with 1500 byte segments, and a protocol "Y" sends 8 packets with 160
byte packets, and if "a single" router in the path drops packets of "Y" due
to the pps limitation causing Y to behave badly, do we say, that protocol
"X" is unfair to "Y"?

One silly question that still lurks in my mind is, say a protocol "X", does
not do slowstart, and sends at a desired rate, but on seeing congestion,
reduces the rate accordingly. Is this a "fair" protocol? I think this is
definitely better than UDP and when we still have UDP, why not have protocol
"X". I know that I will be beaten if I go with this protocol to the IETF :) 

This is just a novice's point of view, please pardon me if I sounded
ignorant, and please do correct me if I am wrong :)

 

-Arjuna

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] 
Sent: 23 February 2007 21:37
To: [e-mail]
Subject: tsvwg Digest, Vol 34, Issue 25

Send tsvwg mailing list submissions to
	[e-mail]

To subscribe or unsubscribe via the World Wide Web, visit
	https://www1.ietf.org/mailman/listinfo/tsvwg
or, via email, send a message with subject or body 'help' to
	[e-mail]

You can reach the person managing the list at
	[e-mail]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of tsvwg digest..."


Today's Topics:

   1. Re: HEADS UP: consensus call on  draft-floyd-tsvwg-cc-alt
      (Bob Briscoe)


----------------------------------------------------------------------

Message: 1
Date: Fri, 23 Feb 2007 21:36:53 +0000
From: Bob Briscoe <[e-mail]>
Subject: Re: [Tsvwg] HEADS UP: consensus call on
	draft-floyd-tsvwg-cc-alt
To: Sally Floyd <[e-mail]>
Cc: tsvwg <[e-mail]>, Mark ALLMAN <[e-mail]>
Message-ID: <[e-mail]>
Content-Type: text/plain; charset="us-ascii"; format=flowed

Sally,

I've added Mark to the distr. more inline...

At 23:12 22/02/2007, Sally Floyd wrote:
>Bob -
>
>My apologies for not getting to your email earlier.
>
>>The doc seems reasonable. We have to be v careful writing these 
>>guidelines that aim to constrain future standards activity. I think this 
>>one has got the balance about right, except I'm very concerned in a 
>>couple of areas:
>>- fairness
>>- constraining 'congestion' control when there's spare capacity
>>
>>The draft could become a WG item so we deal with these issues formally as 
>>WG business. However, you might have noticed 
>><draft-briscoe-tsvarea-fair-00> I'm not at all happy (typical British 
>>understatement!) about the prevailing fairness thinking at the IETF. So 
>>I'd rather see at least an attempt to try to work out how we might reach 
>>consensus on fairness /before/ this becomes a WG item.
>>
>>I believe the above tsvarea-fair I-D showed beyond doubt that comparing 
>>flow rates is irrelevant to real-world fairness, because fairness can 
>>only be between real-world entities like users, not inanimate objects 
>>like flows---users can create any number of flows, for any duration. This 
>>cc-alt I-D came out in parallel, so I'd like to hear if I've made any 
>>impression yet?
>
>Well, the I-D is aimed not at the next generation Internet, but at the 
>current, real-world Internet, where there
>are many congested links that aren't covered by ingress and egress access 
>routers legislating for fairness.  And in this world,
>for traffic that could traverse these congested links, I believe that it 
>is necessary for transport protocols to provide some form
>of per-flow fairness.  As we state in draft-floyd-tsvwg-cc-alt.

When people are proved incorrect, they are meant to stop saying the 
incorrect thing. That's how we reach consensus. Rather than just asserting 
there is merit in carrying on regardless (just like this whole sorry story 
has carried on for 9yrs since Kelly proved it wrong), please can we see an 
argument against the points in my I-D, before we discuss putting anything 
more about fairness in any RFCs.

Let's please just accept that the form of fairness you and many many others 
have defended throughout your careers isn't actually any form of fairness 
at all.{*} Then we can have a proper debate about where we should go from
here.

This isn't personal - it may seem recently I'm been jibing at you 
personally but that's because I usually say nothing when I agree (which is 
most of the time). My issue is with an idea that has been held much more 
widely than just you personally of course, but at last more and more people 
are abandoning it. In fact, I greatly admire everything you have done for 
the Internet, but on this one point on fairness, I'm not saying anything 
you did was wrong at the time, but it's actually been a mirage.

The reason this needs facing now is that this whole flow rate fairness 
charade is stopping progress. It used to provide some degree of restraint 
among those that didnt' understand. But the world is way beyond that now. I 
don't believe pretending TCP is fair is achieving anything useful any more.

What we do have is a definition of fairness where allocations maximise 
global utility. And it is likely that any other reasonable fairness 
definitions will result in allocations close to this.

When two flows happen to run at the same rate through the same bottleneck, 
in  99.9999% of cases this situation will be extremely unfair by this 
definition, when taken in the context of what has gone before, and what the 
same user is doing on other unrelated routers not even on this path. This 
is mostly because flows that go on thousands of times longer than other 
flows are effectively allocated thousands of times more resources.

On the basis that TCP's allocations might then be thousands of times 
different from those that would maximise utility, two things can be
concluded:
- because the fairness outcome is completely arbitrary, depending on 
parameters TCP doesn't control, TCP can't claim to produce a fairness
outcome
- the outcome that happens to result isn't even close to 'reasonable'
anyway.

But, as long as apps are using some form of TCP-like congestion control 
/dynamics/, the Internet is at least safe even though it's not fair.

>Yes, I know about "The Amorphous Problem of Fairness", which I discussed 
>in Section 3.3 of  RFC 3714 back in March 2004,
>and I know about the well-known problems about the granularity of a flow 
>that I discussed in Section 3.2 on "Fairness" in RFC 2914
>in September 2000.

Yup, I appreciated these RFCs said these things. I should ref them in the 
next rev of the fairness draft.

>I have never claimed that per-flow fairness was perfect.  Just that it was 
>necessary, for best-effort traffic.

How can per-flow fairness, which is actually unfairness by all 'reasonable' 
definitions, be necessary?

I'm afraid, this isn't just a minor point. It really does need a re-think 
of everything ever said about TCP fairness. It cannot go on any longer 
without a proper case being made for why we should pretend it's fair when 
it isn't.

>My bottom line about per-flow fairness is that it is a little like
democracy:
>"It has been said that democracy is the worst form of government except 
>all the others that have been tried." - Sir Winston Churchill.

It's not a form of government at all. It doesn't govern anything. It's 
complete anarchy, but TCP (and MulTCP) is at least *safe*.

>>Here's two questions as acid-tests of whether the bullets of section 3 
>>will allow the evolution of congestion control that we need to allow:
>>
>>Q1. If I wrote an I-D defining MulTCP, would it satisfy this I-D's 
>>fairness criteria?
>
>Like much else, draft-floyd-tsvwg-cc-alt leaves such decisions as 
>judgement calls, that would have to be made
>by the IETF at the time.  I personally  wouldn't consider the fairness of 
>MulTCP as sufficient fairness for best effort
>traffic to receive a sentence in the abstract saying that "this is 
>considered ok for widespread deployment as best-effort
>traffic in the current Interent".  Other people might call it differently, 
>who knows?
>
>...
>>The point I'm making here is that fairness between users (as opposed to 
>>between flows) /requires/ the mean rates of flows between different users 
>>to all be very different. This isn't a case of two forms of fairness. 
>>It's two mutually incompatible positions that cannot both be correct. The 
>>IETF cannot use both - it has to decide between them.
>
>What, you are planning on having a flag day, when all links not covered 
>for fairness between users, as opposed to fairness
>between flows, are required to be turned off?  And all transport protocols 
>that deliver some form of fairness between flows,
>instead of fairness between users, are not allowed to transmit packets on 
>the Internet?  Really?  I don't think that Internet
>evolution works that way.

Perhaps you think I have been irresponsible knocking down the only 
touchstone we had, before we had anything else to bolster fairness. But I 
thought about it very hard before going public with that fairness draft.

I realised that there is no need for a flag day. Because we aren't 
transitioning from any form of fairness at all.{*} So any moves towards 
starting to protect links to ensure fairness between users as opposed to 
so-called 'fairness' between flows will be an improvement over nothing. 
Even if we find there's a big security flaw in re-ECN, we won't have lost 
anything, because we didn't have anything{*} wrt fairness in the first
place.

The transports that deliver so-called fairness between flows would work 
just fine, while this new policed environment materialises, because 1-TCP 
works just as well as w-TCPs in this new environment. Unless the user 
causes too much congestion over a sustained period, or is running a 
bot-agent into a DoS attack.

Then, if the network operator chooses to, it can bring in some throttling, 
just like the vast majority of ISPs are doing already to heavy users (at 
least over here in Europe they are). But with congestion information, they 
will be able to throttle much more fairly than they can now. And they will 
only need to throttle packets running into congested paths (e.g. DoS 
attacks). The user will still be able to run flows into low congestion 
paths perfectly fast.

A lot of ISPs are using volume as a metric, but volume's a v poor proxy for 
congestion and therefore fairness. Volume does no harm when there's no 
congestion and volume at some times or on some paths does a lot more harm 
than volume at other times or on other paths, depending on how much 
congestion there is. Worse, plenty of ISPs are throttling traffic based on 
inferring which apps most often cause congestion, using deep packet
inspection.

This is why we need to stop pretending TCP fairness is the solution and 
really face up to the problem properly. We can't blame these people for 
screwing up the Internet with middleboxes, if we've not done our job
properly.


>Another alternative would be that the IETF could have one set of standards 
>for best-effort traffic, and other standards
>for traffic other than best effort.  That is, the IETF could require 
>per-flow fairness for transport protocols used for connections
>when it is possible that some congested link on the end-to-end path is not 
>protected by some other mechanism
>(i.e., diffserv, intserv, some other QoS, fairness enforced by ingress and 
>egress routers, or whatever.)  That is what would
>make sense to me.

Actually, I believe it's this alternative model of the world that is /not/ 
how Internet evolution works. No-one expects the IETF to legislate on the 
policies you've outlined above, which are about how networks should be 
operated or used.

No-one expects the IETF to legislate on how operators write acceptable use 
policies, or for that matter how they use Diffserv codepoints, or what they 
put in SLAs. The IETF is respected for standardising what vendors 
implement. But no-one expects the IETF to legislate on operational polices. 
Or how we all use the Internet.

That's why, even though the IETF has legislated instantaneous TCP fairness 
in RFC2581, ISPs still deploy DPI boxes and throttle flows down if the user 
runs too many TCPs for too long. Do these ISPs make sure these flows comply 
with RFC2581? No. They deliberately break RFC2581. Because in these cases 
RFC2581 flows are pushing the system well away from maximum user utility (= 
max customer value). Why should ISPs comply with an RFC that would push the 
Internet to a point that massively reduces customer value?

That's the message of tussle in cyberspace that you re-inforced yourself in 
(the excellent) RFC3426. If we're forcing people and businesses into places 
the market or society doesn't want to go to, we can't be surprised if our 
rules are broken and ignored.

>I will add a sentence to draft-floyd-tsvwg-cc-alt clarifying that some of 
>its concerns only apply to best-effort traffic.

I'm afraid that's not the issue. At the moment TCP is being used within BE 
to create much better QoS for long lived flows than would be fair. I know 
this sounds bizarre, but it has to be seen from what fairness /would/ be 
like if it was 'reasonably' fair.


>>The dynamics /are/ a legitimate IETF concern. MulTCP responds very safely 
>>in the face of congestion. MulTCP is no less safe than the multiple TCPs 
>>from different apps we have on the Internet today. Why should the problem 
>>be any different if MulTCP emulates opening multiple TCPs but happens to 
>>label all the packets with the same flow ID? The rate behaviour is 
>>exactly the same as if they were w separate TCP flows? Browsers use w=4 
>>already. 20 is just as safe. Or 20,000 for that matter.
>
>I happen to disagree.  I think that an end-node opening 20,000 concurrent 
>flows, as best-effort traffic with no other guarantees
>or protection about available bandwidth, could be causing very bad packet 
>loss rates for the competing traffic in the Internet.  The
>good news is that a concerned router in the Internet could detect this and 
>take some protective action.  And a vendor that designed
>its web browser this way, opening 20,000 concurrent flows as a default 
>behavior for best-effort traffic, would have to
>answer to the considered views of the rest of the Internet community.

Of course.

My point was that this is already possible on the current Internet. 
Certainly I agree that a MulTCP API would be misunderstood by many 
application developers without general deployment of network protections 
against excess congestion, as above. And, true (as I just said in the 
version of my fairness presentation I just gave at ICCRG), we shouldn't 
encourage an API to MulTCP until we have all the standards in place and 
wide deployment of congestion protections.

But that's /not/ because of the potential for an excessively large 
converged rate. It's because the slow start and back-off behaviour of each 
of the TCP's in MulTCP would need to be scaled back to not be the same as 
the back-off behaviour of w TCPs. I certainly agree on that point. But 
that's the neat thing about putting all the separate flows together. MulTCP 
only needs one set of back-off probes, not w.

This is why I've put an asterisk after a few of the times I've said we have 
no fairness at all.{*} Because we do have some form of fairness during 
exponential back-off, and that's important.

Related to this, if you search for hints on how to improve BitTorrent 
performance, you'll get advice to install a patch for WinXP that allows 
many more half connections to open at one time, then it tells you how to 
config BitTorrent so it opens many more connections much faster. It also 
tells you to ignore the warnings from your virus protection software about 
opening up the number of half-connections allowed. It admits this is the 
OS's protection against fast-spreading worms but "it doesn't matter" 
because "you need it for BitTorrent".

Also, have you seen Mustang TCP? It's a start-up company aiming to sell a 
version of TCP that is deliberately unfair to regular TCP flows, for 
hi-speed video download, etc.

This is the reality that ordinary Internet users are used to already. It's 
not that these folks wouldn't want to be fair if they understood. In fact, 
I recently picked up a paper about BitTorrent fairness, but it wasn't about 
fairness to other Internet users, it was about fairness between uploads and 
downloads within the community of users. Most users are meticulous about 
fairness within the community, but they really just don't realise how many 
other users outside their community they are swamping.

And why should they? That's our job - they assume the transport must be 
handling that - why would any lay person imagine it doesn't? Plenty of 
folks at the IETF but outside the transport area don't understand this 
themselves, at all.


>...
>>Q2. Would PCN (pre-congestion notification) based admission control 
>>satisfy these fairness criteria
<draft-briscoe-tsvwg-cl-architecture-04.txt>?
>
>I think of PCN as a form of QoS protection (at least for some of the links 
>on the end-to-end path - I forget if it covers
>all of them).  So I would say that some parts of draft-floyd-tsvwg-cc-alt 
>don't necessarily apply to traffic covered by PCN,
>since some parts of draft-tsvwg-cc-alt only apply for best-effort traffic.

A form of words that distinguishes protected (or perhaps third-party 
mediated) congestion control from voluntary would be useful, to be able to 
say what is out of scope.

But for what remains in scope---on paths where congestion control is 
voluntary---I really don't think it is right that the IETF should legislate 
what is fair. Even if the IETF did legislate, it certainly shouldn't say 
equal flow rates are fair. The IETF might just as well legislate that 
causing equal congestion per flow is fair (over the duration of the flow).

Instead of just carrying on regardless, it would be progress to debate 
whose concern fairness should be. I have proposed the IETF should provide 
the machinery for others to decide on fairness. We need to stop writing 
RFCs that legislate policies that should be in the hands of others.


>...
>>I haven't seen any defence of why it's still appropriate for the IETF to 
>>judge fairness using flow rates, either on this list or TSV-AREA. So this 
>>cc-alt I-D either needs to acknowledge that a major U-turn on fairness is 
>>required, or someone should construct an argument for flow rate fairness 
>>before continuing to use it in this I-D.
>
>I obviously don't believe that the IETF should throw out flow rate 
>fairness for best effort traffic, so I will say so
>more explicitly in draft-floyd-tsvwg-cc-alt.

I obviously differ strongly, because it is actually saying we must have 
allocations that are very very likely to be extremely unfair (by a 
scientific definition of the word).


>..
>>While there is spare capacity, that implies no-one wants any more than 
>>they are using, so it shouldn't concern them that someone else takes more 
>>(unless we want to condone envy for the sake of it). It might conceivably 
>>concern a network operator, perhaps trying to make people pay more for 
>>free beer, I suppose. But that doesn't seem to be a practice that the 
>>IETF needs to concern itself with.
>
>I would leave bullet (2) about "Using Spare Capacity" in.  It was 
>motivated by Quick-Start, whose entire purpose is to
>use spare capacity.  It was also motivated by High-Speed TCP, which had as 
>a motivation using spare capacity.
>I think it is worthwhile that  say that successfully using space capacity 
>is a good thing, but it doesn't eliminate the
>requirement for fairness.

Let's try to untangle what might be the disagreement here. Perhaps, you are 
thinking of fairness as a rate thing so you have to think of fairness over 
a duration. I think that's why you aren't able to separate the period when 
there's spare capacity from when the system spills over into congestion.

If instead you think of fairness in terms of volume of congestion caused, 
there is a very distinct immediate boundary between the two situations. 
Then fairness becomes irrelevant at every instant when the system is 
uncongested. But the shares of capacity users have in flight at the instant 
a resource becomes congested determine how much of the sudden incidence of 
congestion is attributed to each of them. The relation between the 
uncongested and congested periods is one of risk of how much congestion 
traffic in flight will cause before the source gets feedback.

Then fairness is very strictly only an issue from the instant congestion 
starts to the instant it stops - the congested episode.

>...
>>Lastly, how about an extra bullet?:
>>
>>* New cc proposals should explain how the protocol avoids loss of 
>>congestion event information (e.g. NACKs or congestion reports might get 
>>lost etc).
>
>I think that is covered already in bullets (1), (2), and (5).  If the 
>protocol can't avoid the loss of congestion event information,
>then it can't provide either fairness or protection against congestion 
>collapse.

I strongly believe it would be worth saying this very explicitly, not 
assume people understand. It is this more than anything that determines 
whether algorithm behaviour runs to collapse or pulls back from collapse if 
load suddenly surges.


>>Nits
>
>Thanks, I will take care of them.
>
>(I should say that I haven't passed this answer by my co-author, Mark 
>Allman, so he could have different opinions about things...)

Cheers


Bob


____________________________________________________________________________
Notice: This contribution is the personal view of the author and does not 
necessarily reflect the technical nor commercial direction of BT plc.
____________________________________________________________________________
Bob Briscoe,                           Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    [phone]






End of tsvwg Digest, Vol 34, Issue 25
*************************************