Message-Id: <A7A8A99C-B203-4A5E-AB44-BCE7C49AB2BE@cdt.org>
From: Alissa Cooper <acooper@cdt.org>
To: p2pi@ietf.org
Mime-Version: 1.0 (Apple Message framework v919.2)
Date: Mon, 9 Jun 2008 11:53:48 -0400
Subject: [p2pi] More workshop notes
Precedence: list
Content-Type: text/plain; charset="windows-1252"
Content-Transfer-Encoding: quoted-printable
Sender: p2pi-bounces@ietf.org
Errors-To: p2pi-bounces@ietf.org

This is a bit late in coming, but I also took notes at the workshop on  =

May 28 and wanted to share them with the list (my colleague John  =

Morris was on the program committee). I tried to capture anything that  =

wasn't on a slide (and some of what was on the slides if it was  =

necessary to understand the rest), so my notes ended up quite a bit  =

longer than Spencer's.

I grouped paragraphs and back-and-forths by topic, so a blank line  =

between paragraphs indicates a shift in the topic of conversation.

Being somewhat unfamiliar with the IETF community, there were several  =

speakers whose names I didn't catch. They are listed simply as  =

"Speaker" (or "Speaker 1" and "Speaker 2" where they might be confused  =

for the same person within a back-and-forth).

Cheers,
Alissa


IETF p2pi Workshop
May 28, 2008
Cambridge, MA

3.1 Welcome/Note Well/Intro Slides =96 Cullen Jennings

Goals for the meeting:
1) Understand the technical problem behind Comcast/BitTorrent
2) Understand what IETF could do about the problem
	=95 figure out where in IETF this could happen
	=95 tasks that are feasible to finish
	=95 engineering, not research or politics
	=95 applicable to the entire Internet

We=92re not dealing with illegal traffic.
Jessop (RIAA): We should deal in an engineering sense with illegal  =

traffic.
Cullen: We have a congestion problem either way, and since we=92re  =

international, illegal/legal distinctions are tricky.


3.2 Comcast - Jason Livingood/Rich Woundy [16]

Livingood:

Apps providers learning they need to be more friendly, networks  =

learning they need to be more apps friendly.

Comcast=92s commitments:
1) protocol-agnostic network management
2) new focuses on transparency, disclosure, openness, fairness
3) DOCSIS 3.0 roll out

Woundy =96 What the Comcast network looks like:

In 2005-06 Comcast was getting lots of complaints about Vonage, other  =

VoIP, some gaming. There was a congested link between Comcast  =

transport provider and Vonage transport provider in Chicago area. At  =

the same time, p2p traffic was growing, as was the correlation between  =

heavy p2p and packet loss.

Comcast started managing traffic in 2006 (not an early adopter among  =

ISPs). Customer complaints dropped as a result. Last official  =

communication from Vonage was March 2006.

DOCSIS architecture: CPE --> CM --> HFC --> CMTS --> onward. CMTS  =

allocates slots to transmit upstream. A single CMTS serves thousands  =

of homes, has multiple DOCSIS domains. Usually a domain is single  =

downstream with a number of upstream links.

Limiting resource is typically bandwidth. That=92s what Comcast measures  =

for capacity planning. CMTS has a notion of mini-slots. Large packets  =

take up a lot of mini-slots. It=92s not like ATM cells, or like WiFi  =

where dominant cost is acquiring the link. Some discussion about size  =

of mini-slots =96 answers offered are 6.25 microseconds worth of time,  =

64 bytes (minimum Ethernet MTU), or 16 bytes (according to Google).

Any cable modem can make a request to CMTS, with requests randomized  =

to minimize collisions. CM may piggyback next request in data PDU  =

(next request is attached to previous data getting sent). CMTS sees  =

these requests in reliable manner

With many CMs trying to do upstream requests, they collide, resulting  =

in delays. Packet could be queued up to be next packet to CMTS for a  =

full second.

Stanislav: Some motorola cable modems have a 4-second buffer
Speaker: If you measure capacity and speed, buffer size is always 32  =

or 64 KB.

Back to Woundy:
It=92s vendor-dependent. DOCSIS specification does not specify.

How TCP flow fairness applies in upstream direction is unclear. CMTS  =

sees a bunch of service flows =96 could be many TCP flows or one (or  =

UDP). CMTS doesn=92t know source IP, dest IP, etc. until it arrives at  =

the port, at which point flow has already been transmitted upstream.

Key point: With a lot of upstream traffic, piggybacking happens more,  =

which sends heavy upstream users to front of line, over top of  =

interactive but less-heavy apps. For example: With MAPs coming down  =

between 1 and 3 ms, all requests are going through. If we assume  =

packetization delay for a Vonage customer is 20-30 ms, the queue gets  =

cleared out. Then when you go back to the request process, you get  =

back into contention mode. This could explain how heavy users dominate  =

upstream traffic more than interactive users.

A particular cable modem can have multiple service flows. Inbound  =

traffic can be classified to particular service flow. Take the example  =

of CM which is also a VoIP endpoint (e.g., for Comcast voice). You can  =

have a service flow provisioned for voice so that it wouldn=92t go  =

through the upstream process just described =96 it wouldn=92t need to keep  =

requesting upstream space. There would be a separate service flow for  =

regular best efforts.

Henning: How can Vonage request to be put in that service flow for VoIP?
Woundy: A cable company in Canada lets people pay extra for a service  =

and it will provision a separate service flow for it (could be used  =

for Vonage). The other way is to create a service flow on the fly, but  =

Comcast doesn=92t do this.

Cullen: Would looking at the DiffServ bits coming up from the CPE  =

equipment be enough to classify into two different flows?
Woundy: Yes.

Speaker: Does Comcast voice service get affected by upstream  =

congestion? What about for emergency calls?
Woundy: Each time transmissions are allowed, we will provision a  =

reserved lane in next frame for voice.

Clark: Can you distinguish between interfering with yourself and  =

interfering with your neighbor? Which problem do you think you have?  =

Priority bits solve the first problem, not the second problem.
Woundy: We have the neighbor problem.

Livingood:

Engineering Activities
1) Congestion Management Improvements
2) P2P optimizations
	=95 near-term: tracker optimizations, caching, p2p client optimizations
	=95 long-term: signaling from network to clients (increase  =

localization), signaling from clients to network (can declare yourself  =

a scavenger service), improved resource allocation
3) DOCSIS 3.0

Speaker 1: We already have ECN and no one uses it. We also could  =

resurrect scavenger class work.
Speaker 2: Woundy said problem is on access link, but tracker  =

optimizations are for backhaul. And caching only helps on upstream.
Woundy: Yes, tracker optimizations are only for backhaul, but we still  =

have to pay for transit. Caching could improve either upstream or  =

downstream.

Robb: I=92m worried about caching and physical network capacity. Peak-to- =

trough ratios will be higher with caches, and there=92s more likelihood  =

for contention with caches. If contention is from HFC and you=92re  =

trying to get timeslots on RF network, contention will be greater  =

because you have caching.

Back to Livingood:
Will begin rolling out protocol-agnostic network management very soon.  =

With this new net management, all traffic will be marked =93priority.=94  =

When reaching =93near congestion state,=94 we will start marking traffic  =

down to best efforts. Threshold will be based on usage over certain  =

length of time (=93Long Duration Bulk Consumption State=94). Length will  =

be varied in trial. =93Near congestion state=94 can occur on upstream or  =

downstream.

Speaker: What feedback will users get? Anything a user can do to find  =

out his/her traffic has been marked down to best efforts?
Livingood: There=92s no mechanism for feedback right now.

Cullen: How do you get out of the penalty box?
Livingood: People come out of it quickly, but this is part of what we  =

want to discover in trial.

Henning: I can see how this can be done for downstream. But how do you  =

lower priority on upstream?
Woundy: We=92re creating a new default DOCSIS service flow that has same  =

provisioned bandwidth but lower priority.

Bob: This was an interesting decision to change QoS rather than vary  =

bandwidth. BT is varying bandwidth allocation on its DSL network. Why  =

did you chose QoS over reducing bandwidth?
Livingood: Reducing bandwidth is perceived negatively by users to  =

reduce bandwidth.
Bob: But with your system, if an app wants to set its own priority, it  =

has lost that ability, whereas reducing bandwidth doesn=92t affect that.
Woundy: If congestion works itself out in 5 or 7 minutes it won=92t be a  =

problem.

Speaker: Will you factor in the subscriber=92s tier?
Livingood: If you have 8 Mbit/s downstream service, you=92ll have a  =

different thresholds than lower service tiers.

Henning: So many tweakable parameters! There=92s a lot of work in  =

autonomic networking, using control theory approaches to adjust  =

parameters based on feedback. Lots of people want to do this kind of  =

traffic research, but problem is often unavailability of traffic load  =

data.
Livingood: We want to work with research community on these things.

Oran: Does my VPN traffic not get marked down to best efforts when my  =

son is using p2p upstairs?
Woundy: That is a desirable angle, but not something that we will do  =

this year. DiffServ would be helpful with this.

3.3 BitTorrent -  Stanislav Shalunov [22]

Most important problem is increased end-user latency. Uplink is  =

250-500 KB/s. Buffer is 32-64KB. Cheap devices don=92t do active queue  =

management (AQM). So TCP fills the buffer, and you end up with a 1-4  =

second delay, which is a catastrophe from end-user standpoint. Mostly  =

you=92re interfering with yourself, but on cable you might affect the  =

neighborhood.

BitTorrent peer selection is very random by design. =93Rarest first=94 is  =

useful because if original seed goes away, swarm will be able to  =

continue; max amount of capacity can be registered away from original  =

seed; and the rarer the pieces you get, the more likely you=92ll have  =

pieces somebody wants.

Tracker doesn=92t track global piece distribution. =93Rarest=94 is out of  =

the peers that you=92re talking to. The best approximation of global  =

rarest first is an unbiased random sample.
Having a large selection of peers averages out problems. Best peers  =

can be far away (connect to Japan or campus network).

Solving the problem of =93RTT in Seconds:=94 We won=92t get rid of the 4- =

second buffer, practically speaking. We won=92t upgrade every end node  =

in the Internet. But there are practical things that can be done
1) congestion control that avoids filling the buffer
2) scavenger service
3) more explicit congestion notification (not necessarily the two-bit  =

ECN in IP header)

There=92s a good reason to make peer selection random, so everyone  =

should realize that not doing rarest first is a trade-off. But if we  =

want to make overlays more efficient, the simplest way would be to  =

prefer peers in same AS. Since they=92re close, they might be faster. We  =

have already added the IP to AS mapping table to our client. Not sure  =

how useful this will be.

Could also prefer peers in a set of ASes (ISP could have multiple  =

ASes), or
prefer peers in list of IP prefixes. For both of these, client needs  =

to know the lists.

Could also run a cost minimization algorithm, but we need to know the  =

costs to do this.

We did a case study with the four most popular torrents. With >=3D 20  =

peers in an AS, they can just trade with each other. 50% of peers were  =

in ASes like this. Two thirds of interdomain traffic went away. Note  =

that user experience could get worse if ASes are asymmetric  =

(inequality of upload/download) and there=92s a flash crowd.

Since the most popular swarm size we had was 9984, the probability of  =

two peers being in same neighborhood is so low that there=92s no point  =

in trying out localization on the neighborhood level first. Simple  =

theories of localization should be tried first, and there should be  =

substantial gains for doing this kind of localization.

Henning: Do you know the swarm size distributions?
Stanislav: Looks like polynomial distribution. Straight line on log- =

log scale. Not sure about what the tail looks like.

Stanislav: If last mile uplink is most expensive part, caching reduces  =

uplink utilization.
Henning: Caches only help if they=92re not part of the HFC, outside the  =

access network. They can be run by anybody except someone who is on a  =

cable modem.

Back to Stanislav:
For 30% hit rate inside 1 AS, cache need to be 1 TB; for 80% hit rate  =

inside 1 AS, 100 TB.

Oran: What=92s the bandwidth on and off the cache? We may not be storage- =

limited, we may be hotspot limited. You can=92t just plunk down a 100 TB  =

cache.

BitTorrent congestion control is already in 7 million clients in the  =

wild. Using delay instead of loss as the metric. Continuously estimate  =

one-way delay, separating propagation from queuing delay. Ramps up  =

faster than TCP, but will yield to anything else that is not delay- =

based. Approximates scavenger service class with end-to-end congestion  =

control.

Laird: Do you see congestion control as a best practice or standard  =

technique?
Stanislav: It may be useful to standardize an experimental framework  =

where things are not proprietary. Doesn=92t make sense to standardize  =

particular settings.
Lars: We already have an area in IETF for new congestion control  =

protocols, this could be presented there.

Back to Stanislav:
Lower-hanging fruit for this group is DNS-based cache discovery  =

protocol. Also getting information about the network for smarter peer  =

selection =96 perhaps an export of something from BGP communities, or  =

anything that can give clients cost information about different paths.

Clark: Back to question of whether you interfere with your neighbors.  =

CMTS is involved in deciding whether to hand out slots to you or your  =

neighbor. BitTorrent only works on certain slot allocation algorithms.

Oran: I=92d like to know how this looks to a 10x under provisioned DSL  =

network.
Bob: We run a DSL network that doesn=92t have the same problems as cable.

Back to Stanislav:
IETF could also standardize a way for applications to find cache that  =

provider might have installed. BitTorrent Engineering Proposal (BEP)  =

22, which came out last week, is an attempt to sketch a way of how  =

this could be done. We want better ideas of how to do it without DNS,  =

without rewriting packets on the fly.


3.4 Lightning Talks

3.4.1 Robb Topolski [10]

P2P doesn=92t really open hundreds of active connections or trick  =

congestion control. It=92s not designed to take up all bandwidth.

We should not rush to solutions before we understand the problems --  =

why are networks so congested? Where is the problem? How bad is it,  =

how often does it occur, how long does it last, what are the trends?  =

What are the effects and deeper root causes? Can we leverage prior  =

experiences and insight from the history of the Internet?

Quick fixes will only solve a problem for a particular application,  =

but we should fix problems as close to root as possible.

3.4.2 Nick Weaver [1]

P2P does cost-shifting, not cost savings, and it does so  =

inefficiently. P2P has as much data crossing ISP boundaries as  =

uncached HTTP traffic, and for unpopular files you always have 2x bits  =

crossing a boundary as HTTP because of uploading.

Caches are not the solution because of legal liability. ISPs also have  =

no incentive to build caches to help someone shifting costs onto them.

3.4.3 Leslie Daigle [18]

We want continued creativity and innovation on the Internet. This  =

conversation is not strictly about p2p.

There is an assumption that a network operator has no incentive to  =

adjust network to meet demand.

We need to leave today understanding what problem we need to solve:
	=95 Make p2p work better?
		=95 Is it enough? Or what happens with next kind of app that comes
		=95 along?
	=95 Make networks and p2p apps aware of each others=92 requirements?
	=95 General case of bandwidth-intensive apps, beyond p2p networks?
	=95 Deal with in-network issues, or impact of cross-network traffic?


3.5 Localization

3.5.1 P4P - Laird Popkin, Haiyong Xie [13/14]

Haiyong:

Caveats: P4P is not just for p2p. Localization is just a by-product of  =

p4p, not a goal. It=92s also not a tracker optimization approach.

Traditional feedback to apps is limited =96 routing is limited, rate  =

control through TCP is coarse. At application level, TCP hides  =

everything except success, failure and responsiveness. We want to  =

expose more of what=92s going on to apps.

Bob: But TCP hides it because it takes care of it.
Laird: If I=92m trying to pick peers this might be useful information.

P4P Control Plane consists of an iTracker with interfaces for static  =

topology, policy, provider capability, virtual costs, others (it=92s  =

extensible). iTracker can
be identified through DNS SRV, or run by third parties.

Virtual costs are provided by ISPs. Can reflect network status and  =

policy (OSPF weights, higher costs on links with higher utilization,  =

etc.).

Bob: How dynamic are costs expected to be?
Haiyong: We can compute costs by querying the router SNMP database, so  =

can be very dynamic, or can be more static, it=92s up to the ISP.
Laird: One ISP is instrumenting in real time (every 5 minutes). Others  =

have precomputed daytime vs. nighttime distributions. Communication  =

channel can update very frequently.

Laird:

Ran a test with Verizon and another ISP (missed the name?). With  =

regular p2p, 98% of traffic is from sources external to ISP. With P4P:  =

29% local, 21% more internal, 50% external, where =93internal=94 means IP  =

for both source and dest were within AS of Verizon. =93Local=94 vs.  =

=93internal=94 was based on map of IPs provided by Verizon. This was a  =

large swarm =96 35,000 nodes global peak, 1000 nodes after that. 300-500  =

peers internally on Verizon.

Nick: Would transit across border be same volume as uncached HTTP?
Laird: Yes, but nobody pays for transit that way.

Back to Laird:
P4P is a trade group, not a standards group, so it=92s looking at ways  =

to extend IETF protocols. A few ideas:
	=95 trackerless p2p: some mechanism (DNS?) to find iTrackers
	=95 tracker-based p2p: mechanism for peers to find their ASNs more  =

easily than IP-lookup mechanism
	=95 enable p2p to =93play nice=94
	=95 enable apps to determine ISP usage policies and consumption against  =

quota (like cell phone minutes)

Jon: Does P4P have its own DNS resource records?
Haiyong: It=92s only a proposal.
Jon: What about your own TCP header? Anything else you=92re doing with  =

IETF standards?
Laird: The TCP header is for flagging information as being not-time- =

sensitive.

Henning: The peer discovery aspect relates to many other things (STUN,  =

geo-location). If we discover things in one way, it makes sense to  =

leverage that. We shouldn=92t rush to deploy solutions.
Laird: We=92re validating the premise, but turning it into a standard  =

requires a lot more consensus. All we=92ve proved is that localizing  =

traffic improves performance and reduces cost.

3.5.2 Microsoft - Yu-Shun Wang [12]

Localization is useful for several reasons:
	=95 shifting costs to where we can get comfortable with current  =

capacity of network
	=95 mitigate asymmetric capacity provisioning
	=95 limit traffic within an administrative scope =96 QoS, throttling,  =

admission control

However, localization provides a non-uniform user experience (depends  =

on your scope or location).

Designed a Multi-Layer Tracker-Based Architecture. Global tracker can  =

delegate or redirect a peer to a local tracker. Provision local caches  =

among seeds and peers. Nothing to prevent operator from setting up  =

further sub-trackers. Content server and tracker could sit at same  =

place (could be a hybrid CDN).

Good work for IETF to do may be tracker redirection/delegation. Could  =

standardize a generic tracker format =96 can think of it in the abstract  =

as going from global entity to local entity.

Haiyong: Don=92t ISPs want to dump traffic off to other ISPs? Why would  =

they localize?
Yu-Shun: What I=92m hearing today is that localization could help.
Jon: Localization would not be the sole factor, just something to  =

balance against other considerations.

Speaker: Everyone has a disincentive to do this kind of protocol.
Jon: Yu-Shun chopped it up into workable pieces. IETF is not going to  =

take everything on. The tracker formats look like a good piece.
Henning: This is as much an economics problem. If there are incentive  =

mechanisms available, then we don=92t have to pick one =96 customer care  =

is a good example. These won=92t be the same everywhere, but there are  =

plausible incentives out there.

Bob: Difficult decision for IETF =96 ISP won=92t want to use this if you  =

pick all the swarm hosts and work out the costs between them, because  =

then the ISP is complicit in copyright infringement.
Speaker: Is merely answering a localization request about an IP  =

address a legal issue? Without knowing who requests what?
Bob: We need a lawyer.

3.5.3 Vinay Aggrawal [28]

P2P is doing what link layer should have been doing (except at app  =

layer). To us this means we need better ISP-P2P cooperation. ISP knows  =

bandwidth, geography, service class, OSPF/BGP metrics, distance to  =

peers, routing policy, etc. Instead of trying to reverse-engineer  =

these, P2P should be able to find out some of this information.

Oracle service is hosted by ISP at known location. It gets list of IPs  =

from each p2p node, sorts the list, and gives list back to node.  =

Criteria is up to the ISP.

Henning: It would be in the interest of an ISP with upstream problems  =

is to offload all of them onto Columbia University, and Oracle could  =

facilitate that. Also, with oracle list mechanism, tracker gives list  =

of randomly selected nodes =96 swarm can be large but tracker only picks  =

a small handful to give to oracle. 10 out of 1000s, and you can=92t send  =

1000s of IPs to be sorted.

Oran: By rooting source at each individual swarm, result is not close  =

to optimal Steiner tree for entire swarm. Input could instead be  =

everyone else=92s maps.

Laird: What are the scaling properties of this approach? In large  =

swarm every peer will need a unique recommended list.
Vinay: Perhaps tracker could come to Oracle instead of peers dealing  =

with Oracle. Scaling may need some DNS help for look-ups. Oracle is  =

designed to be app-independent =96 helps ISPs remain outside copyright  =

liability. Consulting Oracle also does not imply participation in file- =

sharing.
Speaker: Need to focus on scalability in light of growth of legal  =

content. 9000 is not the top end of swarm size.
Stanislav: When you first join a swarm you would need to rank about  =

200 peers, and once you=92re established (within 1 minute), you will  =

know of 2500 peers.

Jon: Modularity and application-independence are the appeal here. If  =

IETF can leverage that, there may be a lot we can do. The computation  =

that an Oracle does would not necessarily be an IETF focus.
Speaker: With localization built into routing layer already, this kind  =

of system can scale even with very large lists.
Vinay: One possibility is to standardize the interface for the kinds  =

of messages that could be sent between ISPs, peers, trackers.

Speaker: US and EU both have legal regimes that say caching is  =

acceptable, as long as it looks just like Web caching.
Bob: It=92s not about whether you supply the caching service, but if  =

you=92re pointing to content, that may be illegal even if you=92re not the  =

one caching it.

Haiyong: Peer ranking is a special case of virtual cost. Could use  =

virtual cost to rank on the peers. How to handle interdomain ranking?
Vinay: Two ways. ISP could rank its own peers first, or could base  =

ranking on AS number. Or, you could get multiple ISPs working  =

together, have Oracles send lists to each other.

Speaker: When ISP is supposed to expect that clients will magically  =

follow their guidance, that seems too much to expect. Why would  =

clients do it?
Jon: There is an argument that there is an incentive to do this.


3.5.4 Cullen =96 What could IETF do on localization? [Summary slide from  =

the morning]

Leslie: What problem are we trying to solve?

Stanislav: Finding AS numbers is not as valuable as other things that  =

are on the slide because we already know how to do that.
Others: We don=92t know how to do it.

Speaker: We=92ve heard about three kinds of things: making clients less  =

aggressive in bandwidth usage, moving data into caches so it=92s not  =

consuming uplink and downlink, and variety of ways to open up consumer  =

backhaul capacity. It=92s important to ask which of these three general  =

approaches is worthwhile to do. Reducing backhaul doesn=92t seem to be  =

the highest priority. I=92m terrified of legal issues with caching.

Robb: AS number discovery seems like a good idea.
Speaker: AS doesn=92t help mobile IP and mobile =96 AS has nothing to do  =

with where you actually are in the world.

Henning: Suggest that we follow the do-no-harm principle. Not  =

everybody is going to be upstanding. If we provide info that cannot be  =

verified by end user, can lead to undesirable behavior (hot potato  =

routing magnified). Don=92t want to provide info to users that gets them  =

into trouble either.

Cullen: What interest is there in standardized tracker formats?
Stanislav: Don=92t understand the notion of global tracker. Whatever  =

local trackers there could be would be in addition to global  =

(original) tracker?
Yu: Take current tracker format and make it recursive, to make  =

selection easier. Stay within the local tracker once you=92re there (can  =

fall back to global tracker if necessary).
Lisa: HTTP servers often build their own indexes of mirrors in  =

completely non-standard human-readable formats. This seems like  =

another use case for this kind of solution.
Lars: Don=92t need to do anything new, just deploy what we have.
Robb: This is very application-specific.
Stanislav: Would be cool to consult another tracker to find other  =

peers, but for most tracker operators redirecting to other peers/ =

trackers is not going to happen.

Greg: Why don=92t we consider the mechanism as a whole as being  =

standardized? IM was successful, but there were boundaries about  =

interoperability, so we brought it into the IETF.
Jon: The manner in which we brought IM into IETF was by modularizing.  =

We brought in core pieces first to foster interoperability.
Lars: Transport area is doing some related stuff, like AQM schemes to  =

help create shorter queues. Hasn=92t been deployed yet because long  =

queues weren=92t a problem. Did DiffServ but no incentive to mark stuff  =

if network doesn=92t respond, and no incentive to respond if end points  =

don=92t mark. Experimental congestion control for high-speed long-delay  =

networks is in the works, but nothing stops us from doing friendlier- =

than-TCP congestion control. Incentives are better for those sorts of  =

protocols because they don=92t require network to change with it.

Jon: A lot of people in industry talk about localization as a help to  =

ISPs. People are going to be doing this regardless. Are people  =

refuting the claim that localization is useful?
Speaker: It=92s possible that localization has value, but we had a  =

specific concern about a particular traffic management practice.
Jon: We=92re looking for incrementally beneficial solutions.

Livingood: Next month Comcast is starting a localization trial and  =

will be willing to share a great deal of data.
Jon: We need data.
Speaker: We have a localization implementation. We need to know who is  =

requesting localization and who is delivering.
Henning: Alternative is to guess at data when we don=92t have data.  =

Lying about your own AS is detectable. Lying about which IP addr is  =

closer is hard to detect. Not purely a problem related to p2p, we will  =

be facing this is p2psip =96 how to find relay nodes so I don=92t go to  =

Australia to make a call to NY.

Bob: Network cannot say what costs are in other networks. Overlay can  =

talk about costs in whole paths. Shouldn=92t necessarily assume that  =

network is source of information =96 overlay may be able to furnish that  =

data across networks.

Lisa: Some things are feasible until you try to standardize them.  =

Localization may be one of them. Some things work when you constrain  =

it to one ISP and one kind of modem, but not more broadly. Problems  =

with trust, motivation, and accuracy =96 how could oracles share lists  =

even when they trust each other?


3.6 Congestion

3.6.1 Bob Briscoe [32]

Compare DiffServ and weighted congestion control. IETF shouldn=92t judge  =

balance of control between network and hosts =96 there is a role for  =

this to be done on hosts if we get it right

DiffServ
	=95 APIs
		=95 DiffServ is done without APIs, not done on hosts.
		=95 App would need to know what the class choices are.
	=95 Control
		=95 Need API for apps to be able to do this, otherwise it=92s network- =

controlled.
		=95 Even with APIs, how to know network is complying?
	=95 Policing
		=95 Wasn=92t designed to work down to granularity of individual users
		=95 Could be done by volume? time of day? on-net vs. off-net?
			=95 But the whole problem is that peak period will shift once it=92s  =

declared.
		=95 Totally sender-based, but in p2p world people talk about limiting  =

download, which is on receiver side.
	=95 Interconnect
		=95 if two networks work out deal, and I=92m a light user sending to a  =

heavy user, I use up my budget to be brought down in their network  =

because the recipient is getting reduced priority.
	=95 Moving the problem: with heavy hitters in each class, you end up  =

with class for every sort of traffic.

Weighted Congestion Control
	=95 APIs
		=95 Just a matter of setting the weight =96 extension of socket API.
		=95 Need consensus for it to be portable across OSes.
		=95 IETF doesn=92t standardize socket APIs typically.
	=95 Control
		=95 Unambiguously with the host.
	=95 Policing
		=95 Why not always set weight to max? how to deal with this?
		=95 Don=92t want light traffic bursts to puncture real-time apps.
		=95 Sender-based issue just life DiffServ.

There are a few features of good solutions. ISP should not have to  =

fiddle around inside the user=92s envelope (but this doesn=92t mean ISPs  =

can=92t offer optional prioritization). There should also be no =93wriggle- =

room=94 for suspicion about what ISP is doing.

We can view congestion as a harm (cost) metric, where the cost equals  =

the amount of data that wasn=92t served to its destination. This follows  =

=93no wriggle-room=94 =96 cost is greatest when peaks are greatest. Using  =

this metric can allow for a policy that says not to send too much data  =

during congestion, without specifying exactly how much.

Danny: Can you relate this metric to Comcast=92s metric? Part of  =

thinking about what is important to do from standardization  =

perspective has to take into account what is going to be done by ISPs.
Bob: Two ways of looking at it. One is to set volume limits  =

arbitrarily, but other is to set limits based on lengths of queues.

ISPs can=92t limit what the can=92t see. Need to give information about  =

gaps to the network. This is the idea behind re-ECN, which makes it in  =

sender=92s interest to reveal the congestion it knows about.

Quick fixes need to be compatible with long-term goals. IETF/IRTF  =

should setup longer-term design team to asses the general resource  =

sharing problem and the long-term effects of any proposed quick fixes.

Lars: You did not mention p2p, which suggests that the problems that  =

we have are with massive uses of bandwidth, not just p2p.


3.6.2 Marcin Matuszewski

Interviewed 20 ISPs. Their profit margins are low, with acquisition  =

cost higher than monthly revenue. Cost structure varies (fixed vs.  =

mobile). Heavy users are a perceived problem.

Tools available to ISPs (though none widely used): volume-based  =

accounting, fixed price up to certain volume, shaping/blocking/ =

charging for excessive traffic, higher priority assigned to new flows,  =

DPI, adding capacity, limiting subscriber flows, banning servers from  =

residential access, superpeers/content caches, blocking heavy users.

Bob: BT=92s retail department uses DPI and they=92re open about it, and  =

they think it=92s 10x more cost effective than adding bandwidth.

Conclusion: heavy hitters are a business/marketing problem, not a  =

technical problem.


3.7 QoS

3.7.1 Mary Barnes [24]

DPI and traffic analysis are not viable in long term. DiffServ Code  =

Points (DSCP) could be helpful.

DS devices at edges mark packets so nodes don=92t need to remember. This  =

could support charging based on usage, which would be helpful.

Peterson: How does DSCP particularly lend itself to the p2p problem  =

space?
Bob: P2P traffic is less than it used to be, relative to stream  =

traffic. Problem has been around for 7 years, why the rush to solve it  =

now?
Jon: Because of what happened last year.

Clark: Academics have done a lot of hand-wringing about lack of DS  =

deployment, but DS is used in 50% of Cisco enterprise customers. We  =

have an economic problem to solve on the broader internet.
Oran: Actually it=92s larger than that now because VoIP is so popular.  =

But some DSL networks are just not ready for DS. There are not a lot  =

of layer 2 people in the room, but there are severe problems in layer  =

2 networks. How do you trust the markings for DS? That issue will  =

cause everything to come to a screeching to a halt.
Mary: That=92s a common problem to a lot of these solutions.
Robb: Incentive for a host to do the right thing needs to be because  =

it will benefit him to do the right thing.

3.7.2 Henning Schulzrinne [25]

Just minimizing cost to network provider while maximizing cost to  =

content provider doesn=92t help because content provider will just move  =

networks.

It=92s most useful to talk about total cost of delivery, including cost  =

of media storage, media server bandwidth, and delivery bandwidth. Do  =

you re-use existing components or use new components (e.g., personal  =

DVR vs. cache)?

Rational cost shifting exists where people who can save the money  =

funnel savings to those who can=92t save the money. Rentable caches are  =

an example =96 ISPs get paid to lend storage area so they are incented  =

to keep caches. In some versions this is the Akamai model.

Transit bandwidth has flattened out, is no longer getting cheaper.  =

Cost of providing content must take into account different costs on  =

different links (costs look like a step function).

Bob: Cost of empty links is different than cost with traffic on the  =

links. Whether you use long link or short link if it=92s empty you pay  =

the same.
Henning: Assume an end-user centric model where you buy bandwidth by  =

the bit. The price that is charged to the customer and the price you  =

pay for transit are not the same. In a fixed-price market there is not  =

incremental cost. Each additional bit costs nothing until you reach  =

capacity. Congestion doesn=92t play a factor in cost today =96 you pay the  =

same amount of money regardless of congestion on the link.
Bob: That is only true in the US.

Back to Henning:
One question is whether localization matters with long tails. One  =

percent of the U.S. population saw Superbad on one weekend. If this  =

was VoD, each neighborhood would have a copy.

With diurnal variations on some networks, average verses peak times  =

doesn=92t leave much left to shift demand around. On Columbia network  =

only trough is in early morning.

There=92s a general discovery problem for things like STUN, HELD, LoST,  =

SIP local network config. These could all be informed by ISP. There  =

are a number of discovery mechanisms: DHCP, DNS (SRV, NAPTR), anycast.

Scavenger service helps avoid self-interference and requires no  =

admission control =96 can only do worse than you were doing before. The  =

missing piece is to define the code point -- don=92t want to hand code  =

for each network.

For volume-based pricing, we need application-visible charging  =

indicator. App could offer to defer a data transfer until later when  =

bandwidth gets cheaper. Perhaps IETF could help here? Although  =

business models are infinite. Also, charging or rate-limiting exposes  =

users to risk of malware running up your traffic bill.


Conclusion: simple mechanisms are needed that help prevent self- =

interference and work for both symmetric and asymmetric. We also need  =

more data about content distribution.


3.8 Conclusion and Wrap-Up

Jon: This idea of a generalized way to talk to your local network  =

keeps coming up.
Henning: It would be useful to be able to ask a node how close I am to  =

the limit.

Speaker: VoD costs or the quality of a VoIP call are all application- =

specific. Too hard to create generic transport-layer framework.
Henning: The ability to query local network provider to be able to  =

detect what I=92m getting myself into can be generic. Right now service  =

providers cannot easily add metered services that are observable by a  =

user (like electricity). Without a meter, you have no scalable way to  =

built apps.
Jon: Even in P4P, ability for service providers to express policy is  =

at the root of this. We want to extract out where the policies come  =

from, interconnection charges, etc. Abstract it into a black box or  =

express it as a consequence.
Speaker: Network can say it=92s expensive or not, but not sure how much  =

more granular we can get.

Cullen: We heard people saying there would be bandwidth caps.  =

Scavenger class might be worth implementing if it doesn=92t count  =

against your bandwidth limit.

Robb: Two issues: congestion and cost. Congestion in last mile is a  =

technical issue. Cost issue is a choice or decision not to part with  =

your money.
Jon: But they=92re related.
Robb: But the root cause matters. No reasonable amount of money will  =

solve the congestion problem. But the existence of available bandwidth  =

that is unused is a cost issue. Congestion control doesn=92t fix the  =

cost issue. When we don=92t continue to grow the network, we=92re just  =

deciding how to order the packets at a gateway. IETF should decide  =

what is strictly within business purview. IETF should focus on  =

congestion issues.
Jon: This does not all fall into one bucket. People have raised the  =

problem of how to make inelastic traffic compete reasonably with  =

elastic traffic.

Danny: Big divide here in design areas to work in =96 pricing/signaling  =

about what things cost and signaling about congestion and fairness  =

behavior. There=92s a long history of failure to match up network  =

protocols with economic terms, so I=92m wary of the first bucket. Skype  =

figured it out, but it=92s very hard to get it right (think  =

micropayments). P3P tried to get between social and economic  =

interactions and we didn=92t get it right. More modest task of signaling  =

fairness is more realistic.
Jon: Where does scavenger class fall?
Danny: If marking is related to action ISP will take, it=92s in the  =

first category. If it=92s proposed as a currency, it=92s in second category.
Clark: I=92m uncomfortable with division between cost and congestion.  =

Congestion which arises when there=92s a scarcity needs to be resolved  =

through some cost mechanism. People seem to be making value judgments  =

that a p2p person will suffer less than other people. If in times of  =

congestion I get to use the network and you don=92t, you paid because  =

you didn=92t get to use it (exclusion). Can push back by asking users to  =

pay, or slow down. Distinction between cost and congestion is false.  =

ISPs are adding more capacity =96 they=92re paying to relieve congestion.  =

Many cost propositions haven=92t worked, but that doesn=92t mean they  =

won=92t in the future.
Danny: There=92s a relationship between congestion and cost. We don=92t  =

understand it or how to communicate it in a transparent manner, and  =

that=92s the challenge. Cost questions may get worked out out-of-band  =

from here.
Bob: If congestion isn=92t limited, then cost and congestion aren=92t the  =

same thing. When congestion becomes part of the contract of what  =

you=92re not meant to do, then they become the same thing. Because an  =

operator can=92t see congestion, it can=92t make it part of the contract.  =

Can=92t see congestion that user is causing elsewhere.
Speaker: An IRTF workgroup should go back to economics. In economics  =

pricing is a protocol. It indicates when we should spend money. The  =

problem we have is that people have discovered the Internet =96  =

assumption was that grandma would get a big web page back, but now she  =

has a server.

Greg: There are a decent number of people interested in looking at  =

caching, at least as a superpeer. Optimizing peer selection is  =

something IETF could look at. I also encourage us to look at bringing  =

the entire thing in, like with SSL and Jabber. As a vendor, DSCP is  =

costly because whatever is in provider edge box has to do application  =

identification in order to trust DS markings.
Cullen: Latest Windows Vista API is one where voice traffic is marked  =

CS7, above priority reserve for router traffic. Every switch is going  =

to have to remark all traffic from all Windows boxes, which means  =

every box because you can=92t distinguish a Windows box.

Nick: We might end up in a situation where every party is not  =

necessarily happy. Instead the question is how to minimize damage in  =

war between ISPs and p2p. Notions towards user fairness enforced in  =

the network are where IETF should focus. That seems to be the least  =

unpleasant for users who aren=92t heavy users. Bandwidth caps, etc.  =

cause those people harm.
Other Danny (not Weitzner): I agree with Nick. IETF might want to  =

consider application employment of transport layer functions =96 apps  =

opening all those TCP connections isn=92t necessarily a good thing. IETF  =

could say something about that. Mentality on mobile side =96 nights,  =

weekends, distances for billing. All of this is transparent on  =

network, but it costs more to deliver bits further away, and some ISPs  =

are thinking about pricing based on some of these factors.

Lars: RFC 2914 explains what congestion control is good for. 1) Back  =

off and, 2) being sort of fair to other users. Much of what we brought  =

up today is not specific to p2p. If you=92re a heavy user and you build  =

up a queue, you will have delay =96 same for IPTV. I wonder if we can  =

isolate these problems away from p2p. Also fairness =96 need to deal  =

with heavy vs. light whether you=92re using p2p or not.
Jon: Coming back to modularity and how to unify the modules. All the  =

components we=92ve talked about add up to p2p system.
Lars: No, the current cause is p2p. In the future the cause could be  =

other things. Solving them for p2p might not solve them for next thing.

Laird: Issue of p2p opening lots of connections should end up in  =

summary notes.
Stanislav: Multiple connections is not the cause of the problems. If  =

you open one connection, you will get same delay that=92s equal to  =

buffer size in bottleneck link divided by bitrate of bottleneck link.  =

Caused by Facebook photo uploader in same way as BitTorrent.
Nick: That is only true if you are the only person the link. With  =

multiple users sharing one link, TCP flow fairness gives people with  =

multiple open connections a larger portion of the bandwidth. Users  =

with multiple torrents open can saturate the network.
Cullen: Gmail, Gmaps, Mosaic browser use 4-5 TCP connections. It=92s not  =

specific to BitTorrent.
Nick: How BitTorrent is commonly used by users is 4 flows per torrent  =

with multiple torrents.
Oran: There are non-evil reasons to open multiple connections and you  =

can=92t tell the difference.
Stanislav: Piece dissipation is the reason for opening multiple  =

connections. If you only trade with one person you end up with  =

disjoint pairs. Your graph looks better with 3-4 peers.
Livingood: Ted Hardie talked about unattended apps =96 it=92s not just  =

p2p. P2P is as important as all the other apps out there.
Bob: I=92m not saying multiple flows is a problem =96 the internet is a  =

problem because it can=92t judge how many flows people are using.
Jon: If we do not solve the problem for p2p, then we have not solved  =

the problem.





_______________________________________________
p2pi mailing list
p2pi@ietf.org
https://www.ietf.org/mailman/listinfo/p2pi