Re: [PCN] Re: 5.5. Probing functions

Michael Menth <menth@informatik.uni-wuerzburg.de> Tue, 06 November 2007 15:28 UTC

Return-path: <pcn-bounces@ietf.org>
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpQLh-0005bb-5h; Tue, 06 Nov 2007 10:28:17 -0500
Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpQLg-0005ZW-6t for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 10:28:16 -0500
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpQLf-0005Yw-T7 for pcn@ietf.org; Tue, 06 Nov 2007 10:28:15 -0500
Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IpQLa-000760-Co for pcn@ietf.org; Tue, 06 Nov 2007 10:28:15 -0500
Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id AE071F5B7; Tue, 6 Nov 2007 16:28:09 +0100 (CET)
Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id A08BAF5BA; Tue, 6 Nov 2007 16:28:09 +0100 (CET)
Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 45F17F5B7; Tue, 6 Nov 2007 16:28:06 +0100 (CET)
Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lA6FS6h13871; Tue, 6 Nov 2007 16:28:06 +0100
Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id 172906F591; Tue, 6 Nov 2007 16:20:38 +0100 (CET)
Message-ID: <47308628.1010505@informatik.uni-wuerzburg.de>
Date: Tue, 06 Nov 2007 16:20:08 +0100
From: Michael Menth <menth@informatik.uni-wuerzburg.de>
Organization: University of Wuerzburg
User-Agent: Thunderbird 2.0.0.6 (Windows/20070728)
MIME-Version: 1.0
To: "Anna Charny (acharny)" <acharny@cisco.com>
Subject: Re: [PCN] Re: 5.5. Probing functions
References: <BABC859E6D0B9A4D8448CC7F41CD2B07056CAC3F@xmb-rtp-203.amer.cisco.com>
In-Reply-To: <BABC859E6D0B9A4D8448CC7F41CD2B07056CAC3F@xmb-rtp-203.amer.cisco.com>
Content-Type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-Transfer-Encoding: 7bit
X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 8a4bcf8f67063cac573319207fe3db35
Cc: pcn@ietf.org
X-BeenThere: pcn@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: menth@informatik.uni-wuerzburg.de
List-Id: PCN WG list <pcn.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/pcn>, <mailto:pcn-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/pcn>
List-Post: <mailto:pcn@ietf.org>
List-Help: <mailto:pcn-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/pcn>, <mailto:pcn-request@ietf.org?subject=subscribe>
Errors-To: pcn-bounces@ietf.org

Hi Anna, Phil,

Anna, you addressed the impact of ECMP on possible imbalances of link 
utilization. I am with you in the sense that this is only a minor issue 
in networks with high traffic aggregation.

However, the point I wanted to make is that different links can have 
different utilizations in their normal operational mode (e.g. because 
some of them are provisioned with backup capacity for rerouted traffic, 
others aren't). This effect is not caused by ECMP or load balancing in 
general. But the consequence of that effect is that alternative paths 
can have substantially different utilization which possibly becomes 
problematic when ECMP is used in the presence of PCN-based admission 
control.

Regards,

    Michael

Anna Charny (acharny) wrote:
> Hi Phil, Michael,
>
> Based on our own internal study here are a few highlights that can shed
> light on the ECMP accuracy:
>
> 1) once the number of flows (at hash granularity) becomes large
> (~100,000), most hashes perform very well at a single hop for a wide
> range of hash inputs.
> 2) when the number of flows in a hash is smaller (not that small
> actually - on the order of 1000!) practically all hashes display large
> statistical variation for some distributions of the hash inputs (where
> large is on the order 20-40% or more of relative error per bucket).
> This appears to be true for a wide set of hashes, including those shown
> very robust in many academic studies.
>
> The above is for single hop only, and for heterogeneous traffic rates.
> If flow rates distribution is skewed (i.e. there are a few large flows),
> it can become much worse wrt actual bandwidth distribution. Some of the
> known ways to fix this issue is to employ hashes that look deeper into
> the packets so that large flows (e.g. based on (src, dst) hash) get
> split into smaller flows (e.g. look not only at src,dst, but also at
> layer 4 info) - but note that these same methods cause, for our PCN
> purposes, a problem that rsvp messages may no longer follow the same
> path as the data packets. 
>
> So I think realistically, for truly large aggregation at the core, ECMP
> imbalances are likely to be not much of an issue, but for relatively
> lower levels of aggregation one might expect that fairly large ECMP
> inaccuracies in some cases are quite possible.  Note that "relatively
> low aggregation" may occur for tunneled traffic (i.e. lots of small
> flows are aggregated into a relatively small number of large tunnels). 
>
> Anna 
>  
>
>   
>> -----Original Message-----
>> From: philip.eardley@bt.com [mailto:philip.eardley@bt.com] 
>> Sent: Tuesday, November 06, 2007 4:13 AM
>> To: menth@informatik.uni-wuerzburg.de; lars.eggert@nokia.com
>> Cc: pcn@ietf.org
>> Subject: RE: [PCN] Re: 5.5. Probing functions
>>
>> Michael
>>
>> Do you have an idea of how likely this is? will it be the 
>> case that for a small increase in load all/most of the ECMP 
>> paths become full? Does it depend strongly on topology? If 
>> each link is shared by mnay ecmp paths from many different 
>> ingress-egress-aggregates is this less of a risk?
>>
>> phil
>>
>>     
>>> -----Original Message-----
>>> From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de]
>>> Sent: 06 November 2007 08:29
>>> To: Lars Eggert
>>> Cc: Eardley,PL,Philip,CXR9 R; pcn@ietf.org
>>> Subject: Re: [PCN] Re: 5.5. Probing functions
>>>
>>> Hi Lars and Phil,
>>>
>>> I think that one major motivation for probing is to make PCN capable
>>>       
>> to
>>     
>>> deal with multipath routing like ECMP. Some path are empty, 
>>>       
>> others are 
>>     
>>> full. Some flows must be rejected, others can proceed although they
>>>       
>> all
>>     
>>> belong to the same ingress-egress-aggregate.
>>>
>>> Regards,
>>>
>>>     Michael
>>>
>>> Lars Eggert wrote:
>>>       
>>>> Hi,
>>>>
>>>> On 2007-11-5, at 21:30, ext philip.eardley@bt.com wrote:
>>>>         
>>>>> My impression is that probing advocates have 2 reasons:
>>>>>           
>>>> thanks for summarizing this! I think this does capture the main
>>>>         
>> points
>>     
>>>> made.
>>>>
>>>>         
>>>>> - the on-going ingress-egress-aggregate measurement of 
>>>>>           
>> marked pkts 
>>     
>>>>> doesn't work. Because there's often no traffic on this 
>>>>> ingress-egress-aggregate, or no traffic on the ECMP path; AND
>>>>>           
>> simply
>>     
>>>>> admitting the flow has a significant risk of leading to overload
>>>>>           
>> [pkts
>>     
>>>>> dropped &/or flows terminated]
>>>>> - there's no signalling state on the PCN-boundary-nodes for this 
>>>>> ingress-egress-aggregate (and no traffic already on it); AND the
>>>>>           
>> router
>>     
>>>>> behaviour is such that every pkt is marked if the flow should be 
>>>>> blocked; AND just admitting the call is dangerous 
>>>>>           
>> because there's a
>>     
>>> good
>>>       
>>>>> chance of overload in some parts of the nw (near the 
>>>>>           
>> edge where the 
>>     
>>>>> capacity is low). Hence probing creates no extra delay (since you
>>>>>           
>> have
>>     
>>>>> to send a signalling set-up msg anyway). Often this scenario is
>>>>>           
>> people
>>     
>>>>> who envisage PCN going to end terminals or at least further out
>>>>>           
>> than
>>     
>>>>> core nw [I think].
>>>>>           
>>>> Although I agree that we can construct scenarios where "simply 
>>>> admitting the flow has a significant risk of leading to 
>>>>         
>> overload", I 
>>     
>>>> still remain unclear on how likely such cases are in whatever we 
>>>> envision the most likely deployment scenarios to be. Basically, I 
>>>> worry that we're making the architecture complex for what will
>>>>         
>> provide
>>     
>>>> little real benefit in the end.
>>>>
>>>> Also, don't forget that using probing will mean dropping 
>>>>         
>> the packets 
>>     
>>>> of the flow waiting for admission for an RTT *every time* 
>>>>         
>> probing is 
>>     
>>>> used. Whereas "just admitting" may only cause overload sometimes.
>>>> There is a tradeoff here - I believe it's possible to construct a
>>>>         
>> case
>>     
>>>> where probing hurts more than "just admitting."
>>>>
>>>> And I'm still waiting for someone to convince me that 
>>>>         
>> dropping the 
>>     
>>>> initial packets of a flow at the ingress during the 
>>>>         
>> probing RTT will 
>>     
>>>> result in something that is acceptable for apps.
>>>>
>>>>         
>>>>> The first scenario will only happen in narrow conditions [flash
>>>>>           
>> crowds,
>>     
>>>>> many ingress-egress-aggregates without any traffic]. An 
>>>>>           
>> alternative
>> way
>>     
>>>>> of handling this is to limit the rate at which admit new 
>>>>>           
>> flows [at
>>     
>>> least
>>>       
>>>>> this makes the conditions when the problem occurs even narrower].
>>>>>           
>>>> And we should try to determine how narrow these 
>>>>         
>> conditions are. If 
>>     
>>>> they're very narrow, we may be able to avoid them through 
>>>> configuration suggestions (sufficient overprovisioning, 
>>>>         
>> sufficiently 
>>     
>>>> broad marking regions, etc.)
>>>>
>>>>         
>>>>>  The second scenario seems to break the (link) aggregation
>>>>>           
>> assumption?
>>     
>>>>> presumably it could be solved by running different 
>>>>>           
>> mechanism at the
>>     
>>> edge
>>>       
>>>>> compared with at the core. [joe, sorry if you've already 
>>>>>           
>> explained,
>> the
>>     
>>>>> late hour => even poorer memory than usual.]
>>>>>           
>>>> Yes, we seem to be running into the "the number of flows 
>>>>         
>> across any 
>>     
>>>> potential aggregation bottleneck is sufficiently large for
>>>>         
>> stateless,
>>     
>>>> statistical mechanisms to be effective" restriction from 
>>>>         
>> the charter 
>>     
>>>> here. These scenarios seems to require a deterministic mechanism, 
>>>> while PCN is supposed to provide a probabilistic one.
>>>>
>>>> Lars
>>>>
>>>>         
>> --------------------------------------------------------------
>> ----------
>>     
>>>> _______________________________________________
>>>> PCN mailing list
>>>> PCN@ietf.org
>>>> https://www1.ietf.org/mailman/listinfo/pcn
>>>>
>>>>         
>>> --
>>> Dr. Michael Menth, Assistant Professor University of Wuerzburg, 
>>> Institute of Computer Science Am Hubland, D-97074 
>>>       
>> Wuerzburg, Germany, 
>>     
>>> room B206
>>> phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 
>>> mailto:menth@informatik.uni-wuerzburg.de
>>> http://www3.informatik.uni-wuerzburg.de/research/ngn
>>>       
>>
>> _______________________________________________
>> PCN mailing list
>> PCN@ietf.org
>> https://www1.ietf.org/mailman/listinfo/pcn
>>
>>     

-- 
Dr. Michael Menth, Assistant Professor
University of Wuerzburg, Institute of Computer Science
Am Hubland, D-97074 Wuerzburg, Germany, room B206
phone: (+49)-931/888-6644, fax: (+49)-931/888-6632
mailto:menth@informatik.uni-wuerzburg.de
http://www3.informatik.uni-wuerzburg.de/research/ngn



_______________________________________________
PCN mailing list
PCN@ietf.org
https://www1.ietf.org/mailman/listinfo/pcn