Re: [Tsvwg] A coment on "draft-briscoe-tsvwg-relax-fairness-00" re file-sharing

[Bob Briscoe <[e-mail]>]

Robb,

At 23:06 14/04/2008, Robb Topolski wrote:
> > ... How many active (unchoked) connections
> > you get initially depends on configured upload slots, which is indeed
> > about 3-4 per swarm (gets linearly larger for faster access lines -
> > see graph in presentation below).
>
>Bob, this may be quite valid, and I'm not sure whether the BitTorrent
>development community has put much thought into this.  As residential upload
>pipes grow larger, it is possible under BitTorrent's current configuration
>recommendations to have many more parallel tasks going at once.

There is a world beyond the borders of Oregon.

>The 3-4
>upload slots could become 30-40 (assuming 512 Kbps upload becomes 5 Mbps).
>It's still not the scenario of 100 or more, but 30-40 is a sizable departure
>from 3-4 and BitTorrent developers might need to consider how today's TCP
>stacks would treat that in a congestion situation.  Do you think this is
>worthy of discussing?

Need you ask? - it's my current mission in life :)

>Still, I don't think the goal is "fairness."

So what is it?

>I can't think of anyone who
>wants P2P uploads to compete "fairly" with online games, telephone calls, or
>active web surfing.

An operator has to see things from all its customers' points of view. 
Those gaining and those losing out.

There is a way to consider all of these within the same reasoning 
framework so you can reason about fairness, and allow different 
communities to maintain different standards of fairness between their 
members and fairness between communities. See my paper here:
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/pubs.html#rateFairDis>
(Originally an I-D in this w-g, but I let it expire)


>What I honestly don't understand is why we can't mark these packets using
>any of the Internet Standard, vetted, approved methods so that applications
>give ISPs instruction and permission to handle them on a "weather
>permitting" basis.

My own company (not me personally) is developing a Diffserv-based 
approach to this issue for our production network (no promises mind - 
it's not yet committed to as a service).

But it won't be perfect, because ultimately we would need 
inter-provider DS and a generally recognised API to DS. Many of us - 
many on this list - have been trying to sort out all the issues with 
inter-provider DS and with an API to DS.

But there's another approach that might get there first (because it's 
simpler, even tho it's less advanced through standards). It uses 
weighted congestion controls with the ISP enforcing congestion limits 
(that's the bit that requires standards). I prefer it because it can 
be fully controlled by the app/OS, and it's possible to prove the ISP 
is being 'neutral' (or not). But that's another subject.

>Currently, we have ISPs that ignore these flags, inspect
>the packets, and pass their own brand of judgment upon them.

As you rightly point out, a DS-based approach gives a 
'weather-permitting' treatment, so it won't solve this problem of the 
ISP using its independent judgement either.

I have been working on this since 1997, and have finally devised a 
simple way for ISPs (and users) to be accountable for their actions 
(see I-D on re-ECN). <http://www.cs.ucl.ac.uk/staff/B.Briscoe/projects/refb/>


>If the network operators enable that on their networks, I think the apps
>will follow.  And yes, there will be some end-user abuse of these flags --
>but that's true of almost any technical change.

Nope - we've been designing networks and systems for years that a 
robust to abuse.

>The great majority won't abuse it.

It must be lovely to live in such a state of innocence :)


> > * But if peers stays on line after they have finished (seeders),
> > other peers still downloading will activate (unthrottle) connections
> > to nearly as many seeders as they can find, irrespective of the
> > upload slots limit.
> >
> > This is because upload capacity is the bottleneck (as you say). But
> > once you're no longer downloading, BitTorrent no longer needs to
> > preserve your upload capacity for the ack stream that was keeping
> > your download going.
>
>Your understanding has some errors.
>
>First, the downloaders don't activate the seeders, the downloaders only
>indicate "Interest."  The seeder client is in positive control.   Before any
>transfer can take place, the seeder must unchoke that peer so that peer can
>make a request.  The seeder does not unchoke all of the downloaders, it only
>unchokes when it has a slot available.  (Sometimes, your uploads are going
>to people who can't download fast -- such as dial-up users.  Many clients
>have a feature that will automatically open an additional slot if this is
>detected.)

OK - yes - you're correct. But my error on which end has final say 
doesn't change the outcome - see below.


>You are correct that a downloader will indicate "Interest" to all seeders,
>and that many seeders might unchoke that downloader and allow it to make
>requests.  But in that case, those flows are downward in direction --
>currently not the big concern of residential ISP congestion.  Meanwhile the
>seeding clients that are transferring to him still maintain control over the
>number of uploading flows from their nodes so that they can respond to signs
>of congestion.

You're right, except 'uploading seeders responding to signs of 
congestion' is actually one of the biggest aspects of /the/ problem. 
Most{*} currently just use normal TCP congestion control, and the 
more connections they have open the greater share of the pipe they 
get. Yes, each responds to congestion. But the set overall causes a 
much greater share of the upstream congestion precisely as described in my I-D.

{* e.g. Joost doesn't use congestion control for media streaming. DNA 
seems to be TCP-friendly but over UDP.}

And,... focusing solely on flow counts still overlooks the much 
higher activity factor, which multiplies the effect.

>Since congestion is detected and responded to by the sender (the seeder),
>there is nothing inappropriate with this model since the seeder is
>controlling the number of outgoing flows it has.

Define inappropriate.

A pure uploader (seeder) uploading with 15 TCPs active 100% of the 
time causes 150 times more congestion than a user uploading with 2 
TCPs active 5% of the time.


>A BitTorrent client will still respect the upload limit specified either at
>install time or modified by the user, even if it's done downloading.
>However, in many clients there is the option to specify both conditions --
>an upload limit while downloading and upload limit while only seeding.  It's
>against the users interests to turn this up higher -- for reasons similar to
>the ones you mentioned.  The user still wants to surf and use other services
>on the 'net.

But the user's own interest gets hurt long long after other users' 
interests have been hurt to the same degree.


> > My presentation of that draft to the Transport Area in Vancouver
> > <http://www.cs.ucl.ac.uk/staff/B.Briscoe/present.html#0712relax-fairness>
> > shows a screen shot of a colleague's Azureus client (taken randomly
> > at the time I asked him for it). He had 3 swarms running (fairly
> > typical) and 105-114 active TCP data transfer connections, with only
> > about 10-15 choked. One of the swarms is shown. You can see ~40
> > connections are active in this one swarm. This merely implies there
> > were a lot of nice friendly seeders in his swarm.
>
>I think you misinterpreted that display, perhaps.  Notice that the upload
>speed of all of these was only a few bytes a second.  That's overhead from
>downloading.  More evidence is the fact that they're all seeders -- so they
>aren't getting any data from you (they already have all of the pieces).
>Client wide, it doesn't appear that he was uploading anything data all,
>since he limited his upload to 11 KB/s (lower right corner) yet was only
>sending out 2.2 KB/s -- that might be one upload slot or might all be
>overhead chatter.

None of this contradicts anything either of us have said.

It shows a downloader can open many more than 3-4 connections (114 or 
so in this case) when it can find plenty of pure uploaders (seeders).

This in itself /is/ a problem as described in the draft.

Certainly it doesn't show whether pure uploaders can have many 
connections open. But it doesn't show they can't either.

Flows have two ends. If some (and possibly many) downloaders have 
over 100 connections open, is it really likely that all uploaders of 
the same torrent have no more than 3 or 4 connections open?...

...No need to answer that - as it's not actually a water-tight 
argument. There could be only a few downloaders and many seeders for 
one torrent. I'll see if we can find some evidence on upload connection no's.

Nonetheless, I also said in the draft:
"This scenario is not meant to be an accurate model of the current 
Internet, for instance:
...
     * Upstream not downstream constrains most p2p apps on DSL (but 
not all fixed & wireless access technologies). "

The point of the draft was to use p2p to illustrate that the IETF 
needs a new way of thinking. You've shown that what I've said in the 
draft was not incorrect. I can halve the numbers and still make the 
point I was trying to make.


Bob

>He is also misconfigured.  With a global upload limit of 11 KB/s, one might
>suggest he try eMule instead, or only run one torrent at a time and let the
>others queue.  In the long run, it is more efficient.
>
>HTH
>
>Robert M. "Robb" Topolski <[e-mail]>
>Hillsboro, Oregon USA
>
>Variables won't; constants aren't.
>Osborn
>
> > -----Original Message-----
> > From: Bob Briscoe [mailto:[e-mail]]
> > Sent: Monday, April 14, 2008 5:07 AM
> > To: Robb Topolski
> > Cc: 'IETF Transport Area working group mailing list'
> > Subject: Re: [Tsvwg] A coment on "draft-briscoe-tsvwg-relax-fairness-00"
> > re file-sharing
> >
> > Robb,
> >
> > I've split out this more detailed response to your mail - see my
> > previous posting for these two responses:
> > a) Could my scenario become common, even if it's not currently prevalent?
> > b) The discussion is over what the correct fairness /goal/ should be
> >
> > c) Who is actually correct for today's Internet?
> > ---------------------------
> > The 3-4 active connections statement is just as much a fallacy as the
> > hundreds of active connections statement. Which you get depends on
> > the evolution of the situation over the lifetime of a swarm:
> > * If the swarm is immature (ie few peers have finished) your peer
> > will limit open connections. How many active (unchoked) connections
> > you get initially depends on configured upload slots, which is indeed
> > about 3-4 per swarm (gets linearly larger for faster access lines -
> > see graph in presentation below).
> > * But if peers stays on line after they have finished (seeders),
> > other peers still downloading will activate (unthrottle) connections
> > to nearly as many seeders as they can find, irrespective of the
> > upload slots limit.
> >
> > This is because upload capacity is the bottleneck (as you say). But
> > once you're no longer downloading, BitTorrent no longer needs to
> > preserve your upload capacity for the ack stream that was keeping
> > your download going.
> >
> > In some communities you get a lot of seeders staying online. In
> > others, they're all mingey gits who disconnect once they've got what
> > they want.
> >
> > Also, if you're going for niche ('long-tail') content, of course you
> > won't get many active connections, as it's hard to find peers.
> >
> > My presentation of that draft to the Transport Area in Vancouver
> > <http://www.cs.ucl.ac.uk/staff/B.Briscoe/present.html#0712relax-fairness>
> > shows a screen shot of a colleague's Azureus client (taken randomly
> > at the time I asked him for it). He had 3 swarms running (fairly
> > typical) and 105-114 active TCP data transfer connections, with only
> > about 10-15 choked. One of the swarms is shown. You can see ~40
> > connections are active in this one swarm. This merely implies there
> > were a lot of nice friendly seeders in his swarm.
> >
> >
> > Bob
> >
> > At 05:30 12/04/2008, Robb Topolski wrote:
> > >Comment on
> > >
> > >Problem Statement: We Don't Have To Do Fairness Ourselves
> > >draft-briscoe-tsvwg-relax-fairness-00
> > >
> > >
> > >Dear Working Group members,
> > >
> > >Thank you for your time working on this committee and issues such as
> > these.
> > >
> > >The above quoted paragraph requires a number of corrections.  This
> > paragraph
> > >is a basis for the problem statement that follows it -- so getting the
> > basis
> > >down right is key to analysing the issue and sizing up the problem.  The
> > >corrections that I hope to effectively communicate seem to me to reduce
> > the
> > >stated size of the problem.
> > >
> > >--Quote--
> > >Even though file-sharing generally uses TCP, it uses the
> > >well-known trick of opening multiple connections--currently around
> > >100 actively transferring over different paths is not uncommon. A
> > >competing Web application might open a couple of flows at a time, but
> > >perhaps only actively transfer data 1-10% of the time (its activity
> > >factor). Combining 50x less flows and 10-100x lower activity factor
> > >means the traffic intensity from the Web app can be 500-5,000x less.
> > >However, despite being so much lighter on the network, it gets 50x
> > >less bit rate through the bottleneck.
> > >--EndQuote--
> > >
> > >I am relatively certain that the draft is talking about BitTorrent above,
> > >and it unforunately misinterprets what it does.  BitTorrent is the #1 P2P
> > >File Sharing protocol in use today.  Furthermore, neither #2 P2P file
> > >sharing protocol ED2K (aka eDonkey, eDonkey2000, eMule, or MFTP) nor #3
> > >Gnutella (aka G1, G2, or Gnutella2) work like BitTorrent, nor as
> > Briscoe's
> > >draft described above.
> > >
> > >In the case of BitTorrent, between 35-70 TCP connections are generated.
> > >However, only three or four of these connections are actively
> > transferring
> > >data.  Therefore, any benefit to the "trick" is erased.  The reason the
> > >remainder sit idle is described in the specification
> > >(http://wiki.theory.org/BitTorrentSpecification#Choking_and_Optimistic_Un
> > cho
> > >king), chief of which is the desire that BitTorrent NOT interfere with
> > TCP
> > >congestion control.
> > >
> > >It is important to remember that P2P users are largely residential users,
> > >most using connections that are configured with asymmetric connections
> > where
> > >the maximum download bandwidth allowance is several times that of their
> > >upload allowance.  In all of the discussions about "fairness" that I've
> > been
> > >involved in, it has to do with sharing the tiny upload pipe.  Filling the
> > >generously-configured download pipe is seldom a user fairness concern (it
> > is
> > >also very difficult to control at the receiving end).  A trend is barely
> > >beginning toward more symmetrical residential accounts, but it is
> > probably
> > >too small to accurately determine changes in usage patterns created by
> > it.
> > >
> > >That BitTorrent uses "hundreds of connections" that are simultaneously
> > >active is a common misperception.  While the draft likely did not mean to
> > >inject emotionalism, the label of "trick" is both inaccurate and
> > >inflammatory.  The reason that BitTorrent makes these idle connections is
> > to
> > >allow for one connection to "hunt" for the best reciprocating peer, which
> > >allows two efficiently located peers to detect and reciprocate with each
> > >other.  If not interfered with by network operators, this pretty much
> > >ensures that the lowest-latency peers will eventually find one another
> > and
> > >engage in a long-term exchange of data.  This is not only fast for the
> > user,
> > >choosing the most efficient (lowest latency) pairings is friendliest to
> > the
> > >local ISP network and the Internet as a whole.
> > >
> > >All three of the top P2P protocols do open multiple upload connections.
> > >ED2K seems happiest with 3-4 KB/s upload flows, Gnutella with 10 KB/s
> > upload
> > >flows, and BitTorrent with anywhere from 3 KB/s to 7 KB/s flows.
> > Therefore,
> > >the number of upload flows that a single peer might generate (think
> > >different archives being sent by the same client simultaneously) is a
> > >function of dividing the configured upload bandwidth by the amounts
> > >decribed.
> > >
> > >An example:
> > >
> > >BitTorrent:  Upload limit set to 20 KB/s, 3-6 upload TCP connections
> > >(remainder are idle or downloading only)
> > >ED2K:  Upload limit set to 20 KB/s, 5-6 uploading TCP connections
> > (remainder
> > >of connections are downloading only or performing light-bandwidth
> > >housekeeping, such as queue placement requests or updates)
> > >Gnutella:  Upload limit set to 20 KB/s, 1-2 uploading TCP connections
> > >(remainder, same as for ED2K).
> > >
> > >All three of the above have a UDP component, which is used for searching
> > or
> > >other light network housekeeping.  There is an emerging use of UDP for
> > "NAT
> > >hole punching" but these "connections" are still quite rare.
> > >
> > >I am happy to answer any questions or further clarify anything that I
> > can.
> > >
> > >Sincerely,
> > >
> > >Robert M. "Robb" Topolski
> > >[e-mail]
> >
> > __________________________________________________________________________
> > __
> > Notice: This contribution is the personal view of the author and does
> > not necessarily reflect the technical nor commercial direction of BT plc.
> > __________________________________________________________________________
> > __
> > Bob Briscoe,                           Networks Research Centre, BT
> > Research
> > B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473
> > 645196
> >

____________________________________________________________________________
Bob Briscoe, <[e-mail]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    [phone]