Re: [re-ECN] FW: ConEx BoF announcement text

[Fred Baker <[e-mail]>]

I'll argue something slightly different. It comes to the same  
endpoint, but through a different observation. Some of us, you Bob  
especially, are very familiar with this; others are perhaps less so.  
Bear with me.

Jain's 1994 patent on congestion control defines two terms: the  
"cliff" and the "knee" of a throughput curve.

            |
        T   |      availabile capacity
        h   | ---------------------------------------
        r   |       +---+
        u   |  knee/     \cliff
        p   |     /       \
        u   |    /         \
        t   |   /           \
            |  /             \
            | /               \
            |/                 ----------------
            -----------------------------------------
                      Window  --->

As the TCP/SCTP window increases from zero to some value, throughput  
also increases. That stops when the available capacity has been  
consumed; at that point, even if the window grows, throughput does  
not. What increases instead is RTT, because a queue grows. If window  
continues to increase, at some point the queue starts dropping  
traffic, and throughput degrades. By definition, the window at the  
"knee" is smaller than the window at the "cliff", but throughput is  
the same.

Common TCP congestion control algorithms such as Reno and Cubic tune  
to the cliff. That has the upside of maximizing throughput; it has the  
downside that it is abusive to other applications such as voice/video  
(increased and variable RTT, and induces loss), and to other TCP  
sessions. #include <discussion of bittorent and why other users are  
negatively impacted by it>

Other TCP congestion control algorithms based on ECN, CalTech FAST,  
etc, try to tune to the knee. This gets them exactly the same  
throughput including support of very high rate applications but with a  
smaller window value and therefore lower queue depth and lower  
probability of loss - both for themselves and their competitors. It  
does so without negatively, or at least AS negatively, impacting  
applications they compete with beyond seeking to share the available  
capacity with them.

On Oct 22, 2009, at 11:07 AM, Bob Briscoe wrote:

> Leslie,
>
> I accept all the changes you've made except one, which we really  
> must stop saying.
>
> 1/ "However, TCP alone is unable to prevent bandwidth intensive  
> applications"
>
> TCP *does* prevent high bandwidth apps. That's the problem.
>
> And implying intensive bandwidth usage needs to be prevented is  
> rubbish marketing for the idea behind ConEx, given it *enables*  
> higher bandwidth apps.
>
> I believe congestion exposure will enable an Internet where high  
> bandwidth and high volume apps can co-exist optimally. I want to  
> encourage high bandwidth apps *and* high volume apps, not prevent  
> either.
>
> I sometimes think there really are people on this list who believe  
> that too much use of the Internet is the problem. Or congestion is  
> the problem. No. No. No. Rubbish capacity sharing is the problem.
>
> This is the same point as moving away from TCP-friendly, which is  
> what LEDBAT aims for. LEDBAT deliberately allows interactive stuff  
> to go faster while background stuff temporarily goes slower. That is  
> the opposite of TCP-friendly, which is better overall.
>
> Data point: In the transport area plenary in March '09, Matt Mathis  
> asked for a hum on "Is TCP-friendly a way forward?".
> Yes: zero; No: most of the hall.
>
> [It's minuted as 'all hummed no' but I'd say that was over- 
> enthusiastic minute-taking as I'm sure there were plenty of the  
> obligatory silent observers you get these days at the IETF, who were  
> sitting on their hands <http://www.ietf.org/proceedings/74/minutes/tsvarea.txt 
> >]
>
> With congestion exposure, we will be able to have weighted  
> congestion control. Then we should be able to repeat the LEDBAT  
> trick recursively as different size flows arrive. Ie. imagine three  
> types of flows co-existing:
> - small foreground flows
> - medium middleground flows
> - larger background flows
> The larger background can be background to the medium middleground,  
> and both can be background to the small foreground. This can  
> continue with any number of recursions. All transfers complete much  
> faster, except the very largest flows which should complete in not  
> much more time than they do now.
>
> 2/ User experience: I agree; hard to quantify, but not impossible.  
> It's a mix of faster at same price / cheaper at same quality.
>
> Aspects of user experience that are really hard to quantify: less  
> complexity / more future innovation.
>
>
>
> Bob
>
> At 02:55 20/10/2009, Leslie Daigle wrote:
>
>> Bob,
>>
>> First -- thanks for making the suggestions as marked up text:  it  
>> makes it easier for all of us to track what is (and is not)  
>> changing in the text.
>>
>> Having seen no further discussion of the changes, I've made the  
>> changes as follows.  I've also indicated where I've not made the  
>> changes, in case there is further discussion (in-line):
>>
>>
>> [e-mail] wrote:
>>> Bob
>>> Some (personal) comments in-line
>>> Best wishes,
>>> phil
>>> -----Original Message-----
>>> From: [e-mail] [mailto:[e-mail]] On  
>>> Behalf
>>> Of Bob Briscoe
>>> Sent: 18 October 2009 19:47
>>> To: Leslie Daigle
>>> Cc: [e-mail]
>>> Subject: Re: [re-ECN] FW: ConEx BoF announcement text
>>> Leslie,
>>> I like the announcement text now (it was pretty good already).
>>> I have a few suggestions that seem like nits, but they are  
>>> important (to
>>> me).
>>> I'm assuming you "hold the token" on the text at the mo. So I've  
>>> pasted it from the wiki below, and added my comments.
>>> = 
>>> = 
>>> = 
>>> = 
>>> ====================================================================
>>> ====
>>>> Congestion Exposure (ConEx?) is a proposed new IETF activity to  
>>>> enable congestion to be exposed along the forwarding path of the  
>>>> Internet. By revealing expected congestion in the IP header of  
>>>> every
>>> packet,
>>> s/every packet/packets/
>>
>> Done
>>
>>> [phil] agree with your change
>>> [[[Reasoning: We shouldn't imply we have ambitions to ever make  
>>> all Internet traffic expose congestion. I defined the re-ECN  
>>> protocol so re-ECN and non-re-ECN packets can be distinguished.  
>>> Then different apps can choose to use it or not. And different  
>>> ISPs can choose to separately account for these two main types of  
>>> traffic (or not). This is what we should mean by permanent partial  
>>> deployment.
>>> For example, Sally Floyd was much less concerned about re-ECN once  
>>> I had made this design goal clear.
>>> Anyway, on practicality grounds, an ISP can't force all IP traffic  
>>> to use ConEx, particularly if we've only initially defined it for  
>>> some transports like TCP and not others like DNS/UDP. Obviously,  
>>> if an ISP uses congestion exposure in the future to limit heavy  
>>> sources of congestion, then the ISP is likely to severely limit  
>>> non-re-ECN traffic. But we should leave that up to each ISP.
>>> ]]]
>>>
>>>> congestion exposure provides a generic network capability which  
>>>> allows greater freedom over how capacity is shared. Such  
>>>> information could be used for many purposes, including congestion  
>>>> policing, accountability and inter-domain SLAs. It may also open  
>>>> new approaches to QoS and traffic engineering.
>>>>
>>>> The Internet is, in essence, about pooling resources. The ability  
>>>> to share capacity has been paramount to its success and has  
>>>> traditionally been managed through the voluntary use of TCP  
>>>> congestion control. However, TCP alone is unable to prevent  
>>>> bandwidth intensive applications,
>>> s/bandwidth intensive applications/applications transferring high  
>>> traffic volumes/
>>
>> Not done -- agree with Phil.
>>
>>> [phil] personally I don't find your version any more or less clear  
>>> than
>>> the current text. maybe a 3rd version is needed!
>>> [[[Reasoning: TCP *can* limit bandwidth, but it cannot arbitrate  
>>> continuously heavy use of bandwidth over time.
>>> ]]]
>>>
>>>> such as peer-to-peer or streaming video, from causing enough  
>>>> congestion
>>> i/over time /
>>
>> Done.
>>
>>>
>>>> to severely limit the user-experience of many other end-hosts.
>>> s/user-experience of many other end-hosts/experience of many other
>>> users/
>>> [phil] ok
>>
>> Done.  As a bit of a gripe:  "user experience" is not  
>> quantifiable.  It would be nicer if we could actually express this  
>> in terms of some other quantifiable impact at the end hosts.  I am  
>> not inspired to suggestion, however.  (So, I did the mod as  
>> described :-) ).
>>
>>> [[[hosts don't have user-experiences
>>> ]]]
>>>
>>>> This has led ISPs to deploy ad-hoc solutions such as volume  
>>>> accounting, rate policing and deep packet inspection in an  
>>>> attempt to distribute capacity differently. The consequences of  
>>>> such practices are increasingly leading to calls for government  
>>>> regulations and stifling innovation at the transport and  
>>>> application layer (see for example, the problem statement I-D  
>>>> (ref below) and
>>> RFC5594).
>>>> We believe these problems stem from the lack of a network-layer  
>>>> system for accountability -- among all parties -- for sending  
>>>> traffic which causes congestion.
>>> s/sending/forwarding/
>>> [phil] maybe this depends whether you think forwarding is a subset  
>>> or
>>> sending, sending is a subset of forwarding, or neither. Perhaps the
>>> easiest solution is to say "sending or forwarding"
>>
>> Changed to sending or forwarding.
>>
>>> [[[Reasoning: applies equally to networks or senders
>>> ]]]
>>>
>>>> We propose a metric where IP packets carry information about the  
>>>> expected rest-of-path congestion, so that any network node may  
>>>> estimate how much congestion it is likely to cause by forwarding  
>>>> traffic. A network operator can then count the volume of  
>>>> congestion about to be caused by an aggregate of traffic as  
>>>> easily as it can count the volume of bytes entering its network  
>>>> today. Once ISPs can see rest-of-path congestion, they can  
>>>> actively discourage
>>> d/actively /
>>
>> Done.
>>
>>> [phil] ok
>>> [[[Reasoning: passively (e.g. pricing) or actively (e.g. policing)
>>> ]]]
>>>
>>>> users from causing large volumes of congestion, discourage other  
>>>> networks from allowing their users to cause congestion, and more  
>>>> meaningfully differentiate between the qualities of services  
>>>> offered from potential connectivity partners. Meanwhile end-hosts  
>>>> may be freed from rate restrictions where their traffic causes  
>>>> little
>>> congestion.
>>> i/In this environment the self-imposed constraint of TCP- 
>>> friendliness could be relaxed, allowing a richer variety of  
>>> application behaviours to evolve that would still prevent  
>>> congestion collapse./
>>> [phil] if this is inserted, there is no point having the last  
>>> sentence
>>> ["Meanwhile..."], as they say the same thing. I don't care much  
>>> between
>>> them.
>>> [[[Your original had wording around this idea that has got lost in  
>>> translation.]]]
>>
>> No change made -- I think positing TCP-friendliness could be  
>> relaxed is probably overreaching (happy to be educated otherwise),  
>> and largely think the rest is covered in the "Meanwhile" sentence.
>>
>>>
>>>> The purpose of the BoF is to explore the support for and  
>>>> viability of pursuing an IETF activity to define a basic protocol  
>>>> to expose the expected rest-of-path congestion in the IP header.  
>>>> Any such protocol should work with minimal changes to the  
>>>> existing network, in particular it should work with unmodified  
>>>> routers.
>>> d/Any such protocol should work with minimal changes to the  
>>> existing network, in particular it should work with unmodified  
>>> routers./
>>> [phil] I have no strong preference where this text appears
>>> [[[Reasoning: A BoF announcement shouldn't contain strong  
>>> requirements-text. A little lower down, I've introduced text that  
>>> conveys the same sentiment without making it a strong requirement.
>>> ]]]
>>
>> Done -- I do think the BoF announcement should give some sense of  
>> boundaries that exists.  Having said that, I don't care where it  
>> is, and am happy to migrate it closer to the proposal text.
>>
>>>
>>>> There is already one existing proposal that builds on ECN to  
>>>> provide rest-of-path congestion information in every IP header
>>> s/every IP header/IP headers/
>>
>> Done.
>>
>>>
>>>> and other proposals may come forward.
>>>>
>>>> If supported, an eventual WG would focus on the development of that
>>> protocol
>>> s/that protocol/its chosen congestion exposure protocol/
>>>
>>>> as its main work item.
>>> i/The chosen protocol will need to be deployable with minimal  
>>> changes to the existing Internet and compare well against the  
>>> existing proposal, which works with unmodified routers./
>>
>> I inserted:
>>
>> "The chosen protocol will need to be deployable with minimal  
>> changes to the existing Internet, preferably working with  
>> unmodified routers."
>>
>> I deleted the competition with the existing proposal -- either  
>> that's obvious, or something else is at play from which the BoF/WG  
>> should not be constrained, so it seemed an unnecessary restriction  
>> in the text.
>>
>>>
>>>> Additional work items could include detailing the motivations for  
>>>> congestion exposure, a threat analysis of the subsequent  
>>>> protocol, providing feedback on experimental trials and  
>>>> describing deployment considerations.
>>> s/providing feedback on/reporting on/
>>
>> Done.
>>
>>> [phil] ok
>>>
>>>> Importantly, the proposed WG would encourage experimentation but  
>>>> not deliberate on how congestion exposure should be used: our  
>>>> concern would be how flexibly the resulting protocol can support  
>>>> differing needs at run-time, rather than dictating a particular  
>>>> usage at design
>>> time.
>>
>>
>> Thanks,
>> Leslie.
>>
>>>
>>> Bob
>>> At 04:02 17/10/2009, Leslie Daigle wrote:
>>>
>>>> Hi,
>>>>
>>>> I've posted a description on the BoF wiki page:
>>>>
>>>> http://trac.tools.ietf.org/area/tsv/trac/wiki/re-ECN
>>>>
>>>> I think I mostly followed the structural edits, but not the  
>>>> wording changes within sentences, because I felt there were  
>>>> important (if subtle) changes that I wasn't convinced about/would  
>>>> want further input from the list for.   But -- it's a wiki.  We  
>>>> can change the text, if there's need to do so!
>>>>
>>>> Thanks,
>>>> Leslie.
>>>>
>>>>> Suggested edited version (slightly shorter and it moves  
>>>>> references to re-ECN further towards the end. Also tries to  
>>>>> tighten up the text a
>>> bit):
>>>>> Congestion Exposure (ConEx) is a proposed new IETF activity that  
>>>>> reveals congestion along the forwarding path of the Internet. By  
>>>>> revealing the expected congestion in the IP header of every  
>>>>> packet, congestion exposure provides a new generic network  
>>>>> capability. We believe such information could be used for many  
>>>>> purposes, including congestion policing, accountability and  
>>>>> inter-domain SLAs. It may also open new approaches to QoS and  
>>>>> traffic engineering.
>>>>>
>>>>> The Internet is, in essence, about pooling and sharing  
>>>>> resources. This has been paramount to its success and has  
>>>>> traditionally been managed through the voluntary use of TCP  
>>>>> congestion control.  However, TCP alone is unable to prevent  
>>>>> bandwidth intensive applications, such as peer-to-peer or  
>>>>> streaming video,
>>>> >from causing enough congestion to severely limit the
>>>>> user-experience of many other end-hosts.  This has led ISPs to  
>>>>> deploy ad-hoc solutions such as volume accounting, rate policing  
>>>>> and deep packet inspection in an attempt to distribute capacity  
>>>>> differently. Such practices are leading to calls for government  
>>>>> regulation as well as stifling innovation at the transport and  
>>>>> application layer (see for example, the problem statement I-D  
>>>>> (ref below) and RFC5594).
>>>>>
>>>>> We believe these problems stem from the lack of accountability  
>>>>> for causing congestion at the network layer. We propose a metric  
>>>>> where all IP packets carry information about the expected rest- 
>>>>> of-path congestion, so that any network node may estimate how  
>>>>> much congestion it will cause by forwarding traffic. This will  
>>>>> allow network operators to count the volume of congestion about  
>>>>> to be caused as easily as the volume of bytes in any aggregate  
>>>>> of traffic. Once ISPs can see rest-of-path congestion, they can  
>>>>> actively discourage users from causing excessive congestion,  
>>>>> encourage other networks to control the congestion their  
>>>>> customers cause, and more meaningfully differentiate between the  
>>>>> qualities of services offered by potential connectivity  
>>>>> partners. Meanwhile end-hosts can be freed from rate  
>>>>> restrictions so long as they control the overall congestion they  
>>>>> cause.
>>>>>
>>>>> The purpose of this BoF is to explore whether the IETF community  
>>>>> agrees this lack of congestion exposure is a problem and to  
>>>>> gauge the support for and viability of pursuing an IETF activity  
>>>>> to define a basic protocol to expose the expected rest-of-path  
>>>>> congestion in the IP header.  Any such protocol should work with  
>>>>> minimal changes to the existing network; in particular it should  
>>>>> work with unmodified routers. There is already one existing  
>>>>> proposal that builds on ECN to provide rest-of-path congestion  
>>>>> information in every IP header and other proposals may come  
>>>>> forward.
>>>>>
>>>>> If supported, an eventual WG would focus on developing such a  
>>>>> protocol as its main work item.  Additional work items could  
>>>>> include detailing the motivations for congestion exposure, a  
>>>>> threat analysis of the protocol, providing feedback on  
>>>>> experimental trials and describing deployment considerations.  
>>>>> Importantly, the proposed WG would encourage experimentation but  
>>>>> not deliberate on how congestion exposure should be used: our  
>>>>> concern would be how flexibly the resulting protocol can support  
>>>>> differing needs at run-time, rather than dictating a particular  
>>>>> usage at design time.
>>>>>
>>>>>
>>>>> *From:* [e-mail] [mailto:[e-mail]]  
>>>>> *On Behalf Of *[e-mail]
>>>>> *Sent:* 16 October 2009 11:28
>>>>> *To:* [e-mail]
>>>>> *Subject:* [re-ECN] ConEx BoF announcement text
>>>>>
>>>>> Here's a slightly revised version of the announcement text -  
>>>>> thanks to Joao and everyone who worked on it.
>>>>>
>>>>> The main change was to re-write the last 2 paras from the  
>>>>> perspective of the BoF. I also deleted the claim that the work  
>>>>> should be transport-agnostic, as I find 'transport' an ambiguous  
>>>>> word & also think the substantive point is already made by  
>>>>> saying the congestion is revealed in the IP header.
>>>>>
>>>>> Please send any further suggestions asap, so we can circulate to  
>>>>> other mailing lists later today
>>>>>
>>>>> Thanks
>>>>> Phil & Leslie
>>>>>
>>>>> ---
>>>>>
>>>>> Congestion Exposure (ConEx) is a proposed new IETF activity to  
>>>>> enable congestion to be exposed along the forwarding path of the  
>>>>> Internet. By revealing expected congestion in the IP header of  
>>>>> every packet, congestion exposure provides a generic network  
>>>>> capability which allows greater freedom over how capacity is  
>>>>> shared.
>>>>>
>>>>>
>>>>>
>>>>> An existing proposal, building on ECN to reveal "rest-of-path"  
>>>>> information into the IP header, has already demonstrated how  
>>>>> congestion exposure can give an incentive to control one's  
>>>>> impact on the network beside TCP congestion-control. We believe  
>>>>> this "congestion exposure" information may be used for many  
>>>>> purposes, including congestion policing, accountability and  
>>>>> inter-domain SLAs. It may also open new approaches to QoS and  
>>>>> traffic engineering.
>>>>>
>>>>> The Internet is, in essence, about pooling resources. The  
>>>>> ability to share capacity has been paramount to its success and  
>>>>> has traditionally been managed through the voluntary use of TCP  
>>>>> congestion control.
>>>>> However, TCP alone is unable to prevent bandwidth intensive  
>>>>> applications, such as peer-to-peer or streaming video, from  
>>>>> causing enough congestion to severely limit the user-experience  
>>>>> of many other end-hosts.  This has led ISPs to deploy ad-hoc  
>>>>> solutions such as volume accounting, rate policing and deep  
>>>>> packet inspection in an attempt to distribute capacity  
>>>>> differently. The consequences of such practices are increasingly  
>>>>> leading to calls for government regulations and stifling  
>>>>> innovation at the transport and application layer (see for  
>>>>> example, the problem statement I-D (ref below) and RFC5594).
>>>>>
>>>>> We believe these problems stem from the lack of a network-layer  
>>>>> system for accountability -- among all parties -- for sending  
>>>>> traffic which causes congestion. We propose a metric where IP  
>>>>> packets carry information about the expected rest-of-path  
>>>>> congestion, so that any network node may estimate how much  
>>>>> congestion it is likely to cause by forwarding traffic. A  
>>>>> network operator can then count the volume of congestion about  
>>>>> to be caused by an aggregate of traffic as easily as it can  
>>>>> count the volume of bytes entering its network today. Once ISPs  
>>>>> can see rest-of-path congestion, they can actively discourage  
>>>>> users from causing large volumes of congestion, discourage other  
>>>>> networks from allowing their users to cause congestion, and more  
>>>>> meaningfully differentiate between the qualities of services  
>>>>> offered from potential connectivity partners. Meanwhile end- 
>>>>> hosts may be freed
>>>> >from rate restrictions where their traffic causes little  
>>>> congestion.
>>>>> The purpose of the BoF is to explore the support for and  
>>>>> viability of pursuing an IETF activity to define a basic  
>>>>> protocol to expose the expected rest-of-path congestion in the  
>>>>> IP header.  Any such protocol should work with minimal changes  
>>>>> to the existing network, in particular it should work with  
>>>>> unmodified routers.
>>>>>
>>>>> If supported, an eventual WG would focus on the development of  
>>>>> that protocol as its main work item.  Additional work items  
>>>>> could include detailing the motivations for congestion exposure,  
>>>>> a threat analysis of the subsequent protocol, providing feedback  
>>>>> on experimental trials and describing deployment considerations.  
>>>>> Importantly, the proposed WG would encourage experimentation but  
>>>>> not deliberate on how congestion exposure should be used: our  
>>>>> concern would be how flexibly the resulting protocol can support  
>>>>> differing needs at run-time, rather than dictating a particular  
>>>>> usage at design time.
>>>>>
>>>>> ----------------------------------------------------------------------
>>> --
>>>>> _______________________________________________
>>>>> re-ECN mailing list
>>>>> [e-mail]
>>>>> https://www.ietf.org/mailman/listinfo/re-ecn
>>>> --
>>>>
>>>> -------------------------------------------------------------------
>>>> "Reality:
>>>>     Yours to discover."
>>>>                                -- ThinkingCat
>>>> Leslie Daigle
>>>> [e-mail]
>>>> -------------------------------------------------------------------
>>>> _______________________________________________
>>>> re-ECN mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/re-ecn
>>> ________________________________________________________________
>>> Bob Briscoe,                                BT Innovate & Design
>>> _______________________________________________
>>> re-ECN mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/re-ecn
>>
>> --
>>
>> -------------------------------------------------------------------
>> "Reality:
>>     Yours to discover."
>>                                -- ThinkingCat
>> Leslie Daigle
>> [e-mail]
>> -------------------------------------------------------------------
>
> ________________________________________________________________
> Bob Briscoe,                                BT Innovate & Design
> _______________________________________________
> re-ECN mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/re-ecn