RE: [PCN] Re: 5.5. Probing functions

"Anna Charny (acharny)" <acharny@cisco.com> Tue, 06 November 2007 15:39 UTC

Return-path: <pcn-bounces@ietf.org>
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpQWB-0002TO-UP; Tue, 06 Nov 2007 10:39:07 -0500
Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpQWA-0002Sa-Pb for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 10:39:06 -0500
Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpQWA-0002S1-DN for pcn@ietf.org; Tue, 06 Nov 2007 10:39:06 -0500
Received: from sj-iport-5.cisco.com ([171.68.10.87]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IpQW9-0002WF-2Z for pcn@ietf.org; Tue, 06 Nov 2007 10:39:06 -0500
X-IronPort-AV: E=Sophos;i="4.21,378,1188802800"; d="scan'208";a="185116256"
Received: from sj-dkim-4.cisco.com ([171.71.179.196]) by sj-iport-5.cisco.com with ESMTP; 06 Nov 2007 07:39:04 -0800
Received: from sj-core-5.cisco.com (sj-core-5.cisco.com [171.71.177.238]) by sj-dkim-4.cisco.com (8.12.11/8.12.11) with ESMTP id lA6Fd4m5003111; Tue, 6 Nov 2007 07:39:04 -0800
Received: from xbh-rtp-201.amer.cisco.com (xbh-rtp-201.cisco.com [64.102.31.12]) by sj-core-5.cisco.com (8.12.10/8.12.6) with ESMTP id lA6Fcpvu005835; Tue, 6 Nov 2007 15:39:04 GMT
Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-201.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 6 Nov 2007 10:38:56 -0500
X-MimeOLE: Produced By Microsoft Exchange V6.5
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Subject: RE: [PCN] Re: 5.5. Probing functions
Date: Tue, 06 Nov 2007 10:38:55 -0500
Message-ID: <BABC859E6D0B9A4D8448CC7F41CD2B07056CACE9@xmb-rtp-203.amer.cisco.com>
In-Reply-To: <47308628.1010505@informatik.uni-wuerzburg.de>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Thread-Topic: [PCN] Re: 5.5. Probing functions
Thread-Index: Acggiabo63PrGuJISkqoEAqaF1vYSQAAHYmA
From: "Anna Charny (acharny)" <acharny@cisco.com>
To: menth@informatik.uni-wuerzburg.de
X-OriginalArrivalTime: 06 Nov 2007 15:38:56.0864 (UTC) FILETIME=[24B7FE00:01C8208B]
X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15528.002
X-TM-AS-Result: No--21.619600-8.000000-2
X-TM-AS-User-Approved-Sender: No
X-TM-AS-User-Blocked-Sender: No
DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=11553; t=1194363544; x=1195227544; c=relaxed/simple; s=sjdkim4002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20<acharny@cisco.com> |Subject:=20RE=3A=20[PCN]=20Re=3A=205.5.=20Probing=20functions |Sender:=20; bh=V3b1db3gKSOkcC6S5iamERNRS4EbsE2MvW0Me4shHtc=; b=pkEGjCa5qBhtWmf49twavO2KB8Msd0TMwBrlbl8K1PIjLfs11h3iMgIX/KvC28mvD8ybQr9D xYdXYkdOKriBfQSpmHVTgDNItpmnl+9PDmOL0zsaHdRNgTXrKwsexYQP;
Authentication-Results: sj-dkim-4; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/sjdkim4002 verified; );
X-Spam-Score: 0.0 (/)
X-Scan-Signature: c119f9923e40f08a1d7f390ce651ea92
Cc: pcn@ietf.org
X-BeenThere: pcn@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: PCN WG list <pcn.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/pcn>, <mailto:pcn-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/pcn>
List-Post: <mailto:pcn@ietf.org>
List-Help: <mailto:pcn-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/pcn>, <mailto:pcn-request@ietf.org?subject=subscribe>
Errors-To: pcn-bounces@ietf.org

Hi Michael, 

> -----Original Message-----
> From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de] 
> Sent: Tuesday, November 06, 2007 10:20 AM
> To: Anna Charny (acharny)
> Cc: Philip Eardley; Lars Eggert; pcn@ietf.org
> Subject: Re: [PCN] Re: 5.5. Probing functions
> 
> Hi Anna, Phil,
> 
> Anna, you addressed the impact of ECMP on possible imbalances 
> of link utilization. I am with you in the sense that this is 
> only a minor issue in networks with high traffic aggregation.
> 

Actually my point was not only that with sufficiently large aggregations
levels ECMP imbalance is not an practical issue - it was also that there
are realistic circumstances when ECMP imbalance may in fact be an issue
(specifically, deployments with large tunnels)

> However, the point I wanted to make is that different links 
> can have different utilizations in their normal operational 
> mode (e.g. because some of them are provisioned with backup 
> capacity for rerouted traffic, others aren't). This effect is 
> not caused by ECMP or load balancing in general. But the 
> consequence of that effect is that alternative paths can have 
> substantially different utilization which possibly becomes 
> problematic when ECMP is used in the presence of PCN-based 
> admission control.
> 

Would't we expect that in this case PCN thresholds will be chosen as
part of traffic engineering/network planning to reflect the expected
utilisation with and without "planned" failures - so setting these PCN
thresholds (as well as the rest of the network planning tasks) will be
chosen with the expected ECMP load-balancing in mind? So the issue of
imbalances will only/typically occur for "unplanned" multiple failures
or flash crowds for which this planning fails?  

Anna 

Anna 

> Regards,
> 
>     Michael
> 
> Anna Charny (acharny) wrote:
> > Hi Phil, Michael,
> >
> > Based on our own internal study here are a few highlights that can 
> > shed light on the ECMP accuracy:
> >
> > 1) once the number of flows (at hash granularity) becomes large 
> > (~100,000), most hashes perform very well at a single hop 
> for a wide 
> > range of hash inputs.
> > 2) when the number of flows in a hash is smaller (not that small 
> > actually - on the order of 1000!) practically all hashes 
> display large 
> > statistical variation for some distributions of the hash 
> inputs (where 
> > large is on the order 20-40% or more of relative error per bucket).
> > This appears to be true for a wide set of hashes, including those 
> > shown very robust in many academic studies.
> >
> > The above is for single hop only, and for heterogeneous 
> traffic rates.
> > If flow rates distribution is skewed (i.e. there are a few large 
> > flows), it can become much worse wrt actual bandwidth distribution. 
> > Some of the known ways to fix this issue is to employ 
> hashes that look 
> > deeper into the packets so that large flows (e.g. based on 
> (src, dst) 
> > hash) get split into smaller flows (e.g. look not only at 
> src,dst, but 
> > also at layer 4 info) - but note that these same methods cause, for 
> > our PCN purposes, a problem that rsvp messages may no longer follow 
> > the same path as the data packets.
> >
> > So I think realistically, for truly large aggregation at the core, 
> > ECMP imbalances are likely to be not much of an issue, but for 
> > relatively lower levels of aggregation one might expect that fairly 
> > large ECMP inaccuracies in some cases are quite possible.  
> Note that 
> > "relatively low aggregation" may occur for tunneled traffic 
> (i.e. lots 
> > of small flows are aggregated into a relatively small 
> number of large tunnels).
> >
> > Anna
> >  
> >
> >   
> >> -----Original Message-----
> >> From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]
> >> Sent: Tuesday, November 06, 2007 4:13 AM
> >> To: menth@informatik.uni-wuerzburg.de; lars.eggert@nokia.com
> >> Cc: pcn@ietf.org
> >> Subject: RE: [PCN] Re: 5.5. Probing functions
> >>
> >> Michael
> >>
> >> Do you have an idea of how likely this is? will it be the 
> case that 
> >> for a small increase in load all/most of the ECMP paths 
> become full? 
> >> Does it depend strongly on topology? If each link is 
> shared by mnay 
> >> ecmp paths from many different ingress-egress-aggregates 
> is this less 
> >> of a risk?
> >>
> >> phil
> >>
> >>     
> >>> -----Original Message-----
> >>> From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de]
> >>> Sent: 06 November 2007 08:29
> >>> To: Lars Eggert
> >>> Cc: Eardley,PL,Philip,CXR9 R; pcn@ietf.org
> >>> Subject: Re: [PCN] Re: 5.5. Probing functions
> >>>
> >>> Hi Lars and Phil,
> >>>
> >>> I think that one major motivation for probing is to make 
> PCN capable
> >>>       
> >> to
> >>     
> >>> deal with multipath routing like ECMP. Some path are empty,
> >>>       
> >> others are
> >>     
> >>> full. Some flows must be rejected, others can proceed 
> although they
> >>>       
> >> all
> >>     
> >>> belong to the same ingress-egress-aggregate.
> >>>
> >>> Regards,
> >>>
> >>>     Michael
> >>>
> >>> Lars Eggert wrote:
> >>>       
> >>>> Hi,
> >>>>
> >>>> On 2007-11-5, at 21:30, ext philip.eardley@bt.com wrote:
> >>>>         
> >>>>> My impression is that probing advocates have 2 reasons:
> >>>>>           
> >>>> thanks for summarizing this! I think this does capture the main
> >>>>         
> >> points
> >>     
> >>>> made.
> >>>>
> >>>>         
> >>>>> - the on-going ingress-egress-aggregate measurement of
> >>>>>           
> >> marked pkts
> >>     
> >>>>> doesn't work. Because there's often no traffic on this 
> >>>>> ingress-egress-aggregate, or no traffic on the ECMP path; AND
> >>>>>           
> >> simply
> >>     
> >>>>> admitting the flow has a significant risk of leading to overload
> >>>>>           
> >> [pkts
> >>     
> >>>>> dropped &/or flows terminated]
> >>>>> - there's no signalling state on the PCN-boundary-nodes 
> for this 
> >>>>> ingress-egress-aggregate (and no traffic already on it); AND the
> >>>>>           
> >> router
> >>     
> >>>>> behaviour is such that every pkt is marked if the flow 
> should be 
> >>>>> blocked; AND just admitting the call is dangerous
> >>>>>           
> >> because there's a
> >>     
> >>> good
> >>>       
> >>>>> chance of overload in some parts of the nw (near the
> >>>>>           
> >> edge where the
> >>     
> >>>>> capacity is low). Hence probing creates no extra delay 
> (since you
> >>>>>           
> >> have
> >>     
> >>>>> to send a signalling set-up msg anyway). Often this scenario is
> >>>>>           
> >> people
> >>     
> >>>>> who envisage PCN going to end terminals or at least further out
> >>>>>           
> >> than
> >>     
> >>>>> core nw [I think].
> >>>>>           
> >>>> Although I agree that we can construct scenarios where "simply 
> >>>> admitting the flow has a significant risk of leading to
> >>>>         
> >> overload", I
> >>     
> >>>> still remain unclear on how likely such cases are in whatever we 
> >>>> envision the most likely deployment scenarios to be. 
> Basically, I 
> >>>> worry that we're making the architecture complex for what will
> >>>>         
> >> provide
> >>     
> >>>> little real benefit in the end.
> >>>>
> >>>> Also, don't forget that using probing will mean dropping
> >>>>         
> >> the packets
> >>     
> >>>> of the flow waiting for admission for an RTT *every time*
> >>>>         
> >> probing is
> >>     
> >>>> used. Whereas "just admitting" may only cause overload sometimes.
> >>>> There is a tradeoff here - I believe it's possible to construct a
> >>>>         
> >> case
> >>     
> >>>> where probing hurts more than "just admitting."
> >>>>
> >>>> And I'm still waiting for someone to convince me that
> >>>>         
> >> dropping the
> >>     
> >>>> initial packets of a flow at the ingress during the
> >>>>         
> >> probing RTT will
> >>     
> >>>> result in something that is acceptable for apps.
> >>>>
> >>>>         
> >>>>> The first scenario will only happen in narrow conditions [flash
> >>>>>           
> >> crowds,
> >>     
> >>>>> many ingress-egress-aggregates without any traffic]. An
> >>>>>           
> >> alternative
> >> way
> >>     
> >>>>> of handling this is to limit the rate at which admit new
> >>>>>           
> >> flows [at
> >>     
> >>> least
> >>>       
> >>>>> this makes the conditions when the problem occurs even 
> narrower].
> >>>>>           
> >>>> And we should try to determine how narrow these
> >>>>         
> >> conditions are. If
> >>     
> >>>> they're very narrow, we may be able to avoid them through 
> >>>> configuration suggestions (sufficient overprovisioning,
> >>>>         
> >> sufficiently
> >>     
> >>>> broad marking regions, etc.)
> >>>>
> >>>>         
> >>>>>  The second scenario seems to break the (link) aggregation
> >>>>>           
> >> assumption?
> >>     
> >>>>> presumably it could be solved by running different
> >>>>>           
> >> mechanism at the
> >>     
> >>> edge
> >>>       
> >>>>> compared with at the core. [joe, sorry if you've already
> >>>>>           
> >> explained,
> >> the
> >>     
> >>>>> late hour => even poorer memory than usual.]
> >>>>>           
> >>>> Yes, we seem to be running into the "the number of flows
> >>>>         
> >> across any
> >>     
> >>>> potential aggregation bottleneck is sufficiently large for
> >>>>         
> >> stateless,
> >>     
> >>>> statistical mechanisms to be effective" restriction from
> >>>>         
> >> the charter
> >>     
> >>>> here. These scenarios seems to require a deterministic 
> mechanism, 
> >>>> while PCN is supposed to provide a probabilistic one.
> >>>>
> >>>> Lars
> >>>>
> >>>>         
> >> --------------------------------------------------------------
> >> ----------
> >>     
> >>>> _______________________________________________
> >>>> PCN mailing list
> >>>> PCN@ietf.org
> >>>> https://www1.ietf.org/mailman/listinfo/pcn
> >>>>
> >>>>         
> >>> --
> >>> Dr. Michael Menth, Assistant Professor University of Wuerzburg, 
> >>> Institute of Computer Science Am Hubland, D-97074
> >>>       
> >> Wuerzburg, Germany,
> >>     
> >>> room B206
> >>> phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 
> >>> mailto:menth@informatik.uni-wuerzburg.de
> >>> http://www3.informatik.uni-wuerzburg.de/research/ngn
> >>>       
> >>
> >> _______________________________________________
> >> PCN mailing list
> >> PCN@ietf.org
> >> https://www1.ietf.org/mailman/listinfo/pcn
> >>
> >>     
> 
> --
> Dr. Michael Menth, Assistant Professor
> University of Wuerzburg, Institute of Computer Science Am 
> Hubland, D-97074 Wuerzburg, Germany, room B206
> phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 
> mailto:menth@informatik.uni-wuerzburg.de
> http://www3.informatik.uni-wuerzburg.de/research/ngn
> 


_______________________________________________
PCN mailing list
PCN@ietf.org
https://www1.ietf.org/mailman/listinfo/pcn