RE: [bmwg] WG Last Call: Removal of FIB Benchmarking Methodology from the BMWG Goals

sporetsky@quarrytech.com Wed, 21 January 2004 22:29 UTC

Received: from optimus.ietf.org ([132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA15223 for <bmwg-archive@lists.ietf.org>; Wed, 21 Jan 2004 17:29:28 -0500 (EST)
Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AjQqK-0000jU-Uy; Wed, 21 Jan 2004 17:29:00 -0500
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AjQq0-0000hn-Ud for bmwg@optimus.ietf.org; Wed, 21 Jan 2004 17:28:41 -0500
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA15120 for <bmwg@ietf.org>; Wed, 21 Jan 2004 17:28:37 -0500 (EST)
From: sporetsky@quarrytech.com
Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AjQpy-0000YZ-00 for bmwg@ietf.org; Wed, 21 Jan 2004 17:28:38 -0500
Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1AjQom-0000SD-00 for bmwg@ietf.org; Wed, 21 Jan 2004 17:27:26 -0500
Received: from email.quarrytech.com ([4.17.144.4] helo=qtech1.quarrytech.com) by ietf-mx with esmtp (Exim 4.12) id 1AjQoa-0000QC-00 for bmwg@ietf.org; Wed, 21 Jan 2004 17:27:12 -0500
Received: by email.quarrytech.com with Internet Mail Service (5.5.2653.19) id <XT41AWY2>; Wed, 21 Jan 2004 17:26:42 -0500
Message-ID: <496A8683261CD211BF6C0008C733261A0430D842@email.quarrytech.com>
To: sand@nortelnetworks.com, sporetsky@quarrytech.com, kdubray@juniper.net, bmwg@ietf.org
Cc: bwijnen@lucent.com, david.kessens@nokia.com
Subject: RE: [bmwg] WG Last Call: Removal of FIB Benchmarking Methodology from the BMWG Goals
Date: Wed, 21 Jan 2004 17:26:38 -0500
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C3E06D.A2922650"
X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org
X-Spam-Status: No, hits=0.5 required=5.0 tests=AWL,HTML_20_30,HTML_MESSAGE, NO_REAL_NAME autolearn=no version=2.60
Sender: bmwg-admin@ietf.org
Errors-To: bmwg-admin@ietf.org
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>

 <http://www.ietf.org/internet-drafts/draft-ietf-bmwg-conterm-05.txt> Allan,
 
The BMWG already has adopted the following work items for Route Convergence:
 
Terminology for Benchmarking BGP Device Convergence in the Control Plane
(66758 bytes)
 
<http://www.ietf.org/internet-drafts/draft-ietf-bmwg-ospfconv-intraarea-07.t
xt> Benchmarking Basic OPSF Single Router Control Plane Convergence (30710
bytes)
 
<http://www.ietf.org/internet-drafts/draft-ietf-bmwg-ospfconv-applicability-
03.txt> Benchmarking Applicability for Basic OSPF Convergence (21766 bytes)
 
<http://www.ietf.org/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-meth
-01.txt> Benchmarking Methodology for IGP Data Plane Route Convergence
(31086 bytes)
 
<http://www.ietf.org/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-term
-02.txt> Terminology for Benchmarking IGP Data Plane Route Convergence
(61350 bytes)
 
<http://www.ietf.org/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-app-
01.txt> Benchmarking Applicability for IGP Data Plane Route Convergence
(12280 bytes)
 
The intention of the FIB Performance work item was to benchmark FIB Scaling
and Forwarding Performance with the FIB scaled.
 
Scott



-----Original Message-----
From: Allan Sand [mailto:sand@nortelnetworks.com]
Sent: Wednesday, January 21, 2004 5:05 PM
To: 'sporetsky@quarrytech.com'; kdubray@juniper.net; bmwg@ietf.org
Cc: bwijnen@lucent.com; david.kessens@nokia.com
Subject: RE: [bmwg] WG Last Call: Removal of FIB Benchmarking Methodology
from the BMWG Goals



I'm new to this forum, so excuse me if I have breached protocol.  I would
like to add the following for consideration. 

Ideally a route update should have no impact on the forwarding plane except
as pertains to the prefix(es) being updated (added, withdrawn or modified).
The ideal behaviour is the router should continue to forward traffic to all
'other' prefixes. In practice, the behaviour of routers is far from ideal
and the affect of a route update (on the forwarding plane) is characterized
by the number of unavailable prefixes and the duration of the 'impact'.
Unavailable here means not being able to forward to a prefix. 

The factors affecting the number of unavailable prefixes and the duration of
that state can be grouped between those related to control plane
implementation and those impacted by forwarding plane implementation.
Control plane implementation focuses on control traffic throughput (i.e.
processing control messages that determine convergence time) under normal,
high and overload states (for control messages). Hence the control plane
networking algorithms include different implementations of hold-off timers
and rate controls and SPF speedup algorithms.  

The factors affecting the number of prefixes impacted (i.e. unavailable for
forwarding) and duration of impact is more of a forwarding plane
implementation byproduct; the main engineering issues being prefix lookup
speed and maximum prefix size.  In most forwarding plane implementations,
the number of prefixes impacted is determined by the type of prefix lookup
algorithm. One common algorithm is a Patricia Tree (PT), a form of trie data
base. Another common algorithm is prefix hashing.

Prefixes are listed in a PT as nodes and branches. A route update which
results in a branch type prefix being added to the FIB has no impact on
looking up other prefixes. In other words, a route update which adds a new
network prefix will have no impact on forwarding to other prefixes. However,
a route update which adds a node type prefix, may result in an inability to
forward to that node prefix and all its branches for the period of time it
takes to rearrange that particular node/branch hierarchy in the PT. 2nd
generation routers using PT come in at 20 prefixes/ms update rate.  

In addition, routers use clever techniques in the control plane to reduce
the number of prefixes that eventually need to be sent (updated) to the FIB.
One such technique is prefix incremental updates. With this technique only
changed prefixes are updated, rather than say a wholesale dump of the RIB.
Another technique is to use a 2 stage lookup scheme in the FIB. This
technique reduces the number of prefixes to 1, when N prefixes share a
common alternate next hop.

In summary, FIB benchmark tests should include a measure of the impact
(number of unavailable prefixes and duration) resulting form route updates.
The test cases (i.e. what prefixes end up getting changed as a result of the
route update) need to be carefully designed not to favour any one router
control and forwarding plane implementation. But rather should be designed
to mimic real network configurations and scenarios.  

-----Original Message----- 
From: sporetsky@quarrytech.com [ mailto:sporetsky@quarrytech.com
<mailto:sporetsky@quarrytech.com> ] 
Sent: Wednesday, January 21, 2004 1:53 PM 
To: kdubray@juniper.net; bmwg@ietf.org 
Cc: bwijnen@lucent.com; david.kessens@nokia.com 
Subject: RE: [bmwg] WG Last Call: Removal of FIB Benchmarking Methodology
from the BMWG Goals 


Kevin, 

I think a FIB Benchmarking standard to measure FIB scaling would be very
useful to industry.  This specific methodology  may have low interest
because it is rather narrow in test coverage, having only 3 test cases as

follow: 
1. Baseline Measurements.  
2. Maximum FIB Size Test 
3. FIB Size Impact On Packet Forwarding Performance 

Test 1 is less preferred in practice to the already defined Forwarding Rate
test and Test 3 has been obviated by the architecture of modern routers.
Test 2 is the one very useful to industry.  I recommend that a small team be
formed to determine if there are a number of interesting test cases around
Test 2 that could be developed in order to resurrect this work.

Scott 

-----Original Message----- 
From: Kevin Dubray [ mailto:kdubray@juniper.net <mailto:kdubray@juniper.net>
] 
Sent: Wednesday, January 21, 2004 11:54 AM 
To: bmwg@ietf.org 
Cc: Bert Wijnen; david.kessens 
Subject: [bmwg] WG Last Call: Removal of FIB Benchmarking Methodology from
the BMWG Goals 


This is an action item from the BMWG meeting at the 58th IETF. 

When originally undertaken, the FIB performance benchmarking effort was 
considered a critical, 
constituent component of the convergence benchmarking body of work. 

Subsequent WG support for the FIB benchmarking methodology document has 
been underwhelming. 
The last known state of the FIB Benchmarking Methodology draft can be 
found here: 

    http://www.watersprings.org/pub/id/draft-ietf-bmwg-fib-meth-01.txt
<http://www.watersprings.org/pub/id/draft-ietf-bmwg-fib-meth-01.txt>  

Commentary from its most recent last call can be found here: 
  
http://www1.ietf.org/mail-archive/working-groups/bmwg/current/msg00477.html
<http://www1.ietf.org/mail-archive/working-groups/bmwg/current/msg00477.html
>  

Please voice your opinion as to whether or not you feel it is critical 
for the BMWG to support 
this work.  Send your comments to this list or to the 
chairs.(acmorton@att.com, kdubray@juniper.net) 

If you would like serve as the editor for this effort, please indicate 
your desire to the 
chairs. 

This WG Last Call period will extend through 02 Feb 04.  At the end of 
the period, the chairs 
will make a recommendation to the ADs, based on the WG's input AND the 
ability to identify 
a document editor. 





_______________________________________________ 
bmwg mailing list 
bmwg@ietf.org 
https://www1.ietf.org/mailman/listinfo/bmwg
<https://www1.ietf.org/mailman/listinfo/bmwg>  

_______________________________________________ 
bmwg mailing list 
bmwg@ietf.org 
https://www1.ietf.org/mailman/listinfo/bmwg
<https://www1.ietf.org/mailman/listinfo/bmwg>