Re: [p2pi] Real life torrent statistics

[Laird Popkin <[e-mail]>]

You're right that swarms with 10 peers can't be optimized - the p2p network will connect all 10 peers to each other and move as much data as possible. 



That being said, I suspect that a small number of very popular swarms that can be optimized would give significant impact on overall data flow, because one swarm with 20,000 peers balances out a lot of 'long tail'. Haiyong Xie of Yale mentioned to me that there was an analysis of this last year, by spidering one of the popular torrent web sites, and their conclusion was that well over 50% of the downloaders were in swarms with 100+ active peers, which is where they estimate that P4P optimization applies. Similarly, when I was in the music business and tracked such things, 2% of the files in the p2p networks accounted for the large majority of the download activity. 



I think that it would be a very interesting analysis, spidering some torrent web sites and seeing how the distribution of bandwidth is balanced between the 'head' and the 'long tail'. 


- Laird Popkin, CTO, Pando Networks 
  mobile: [phone] 

----- Original Message ----- 
From: "The 8472" <[e-mail]> 
To: "Stas Khirman" <[e-mail]> 
Cc: [e-mail], [e-mail] 
Sent: Tuesday, August 19, 2008 4:40:39 PM (GMT-0800) America/Los_Angeles 
Subject: Re: [p2pi] Real life torrent statistics 

Stas Khirman wrote: 




To estimate a feasibility of ALTO/P4P for real life torrents , I collected <ip,port> information for peers from one of the most popular “PirateBay” torrents ( almost 20k peers) and maped their IPs to corresponded ASs. Please find attached my working notes with some interesting statistics. Ahh, there is a problem with this one. With torrents you have a significant long tail when it comes to swarm sizes and content. I'm not certain about distribution, but the long tail will probably outweight the... let's say top 100 torrents. Torrents with only 10-20 peers spread throughout several ASNs is a much harder to optimize than the top 100. 
This problem is aggreviated by swarm fragmentation due to private trackers and since bittorrent does not aim to coalesce all torrents with the same content, i.e. due to different piece sizes, file names etc. 







Also, I find it surprising geo distribution of the peers – majority were in UK , not in US (probably because content is available in US theaters).  Places 3-5 taken by Sweden, Poland and Canada (in total – more peers then in US). This will probably be different if you sample torrents from regional trackers or torrents aimed at other audiences. During some DHT tracing on the weekend i saw a significant proportion of DHT traffic coming from asian countries, though i suspect an inefficient implementation of the DHT by a client that's popular in china to play some role in this distribution. 









Certainly, observed “heavy” neighboring of peers is a function of swarm size. I intend to investigate a few medium/small size swarms to have a multi-point picture for any future discussions. as i mentioned above we should try to get the big picture, i.e. how relevant the long tail is, measured in aggregate bandwidth. If the small torrents actually make up the bulk of the traffic then any solution will require a high degree of cooperation between ISPs, e.g. caches that cooperate with each other. 

-- 
The 8472 
independent developer for the Azureus Vuze Bittorrent client 

_______________________________________________ p2pi mailing list [e-mail] https://www.ietf.org/mailman/listinfo/p2pi

_______________________________________________
p2pi mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/p2pi