[bmwg] FW: Comment on meth-18.
"Kris Michielsen" <kmichiel@cisco.com> Mon, 20 July 2009 11:20 UTC
Return-Path: <kmichiel@cisco.com>
X-Original-To: bmwg@core3.amsl.com
Delivered-To: bmwg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 58F6C3A69E1 for <bmwg@core3.amsl.com>; Mon, 20 Jul 2009 04:20:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.37
X-Spam-Level:
X-Spam-Status: No, score=-1.37 tagged_above=-999 required=5 tests=[AWL=1.230, BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sXBDlMcyx7me for <bmwg@core3.amsl.com>; Mon, 20 Jul 2009 04:20:41 -0700 (PDT)
Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by core3.amsl.com (Postfix) with ESMTP id AACC43A6AA1 for <bmwg@ietf.org>; Mon, 20 Jul 2009 04:20:40 -0700 (PDT)
X-TACSUNS: Virus Scanned
Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id n6KBJsxh006834; Mon, 20 Jul 2009 13:19:54 +0200 (CEST)
Received: from kmichielwxp (dhcp-peg2-vl21-144-254-14-152.cisco.com [144.254.14.152]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id n6KBJrTl013312; Mon, 20 Jul 2009 13:19:53 +0200 (CEST)
From: Kris Michielsen <kmichiel@cisco.com>
To: bmwg@ietf.org, 'Al Morton' <acmorton@att.com>, "'McLendon, John'" <John.McLendon@spirent.com>
Date: Mon, 20 Jul 2009 13:19:48 +0200
Message-ID: <000a01ca092c$015fb160$980efe90@emea.cisco.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Mailer: Microsoft Office Outlook 11
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3350
Thread-Index: AcoFhX5d/L88OUFuStem1Mmgc3WxrgAEPfsgACVfQqAANqU5IACF9K8AAAGicRA=
Subject: [bmwg] FW: Comment on meth-18.
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Jul 2009 11:20:42 -0000
My previous (unfinished) email left inadvertently. Here's the completed version. == Hi John, My view on it is as follows. Let's take an example of an IGP adjacency loss. When looking at the Forwarding Rate over all routes, we derive the following graph of Fwd Rate vs time. At CEI the Tester stops rx/tx IGP hellos. This is a general case since it doesn't cause instantaneous traffic drop for all routes on the Convergence Event Instant (CEI). At time Ta traffic starts dropping for the first route Rta (observed on Preferred Egress Interface) and for this example traffic forwarding for Rta starts on Next-Best Egress Interface starts at time Ta'. Depending on the DUT implementation this can be anywere between First Route Convergence Instant and Convergence Recovery Instant (see also discussion in meth-18, Section 4). ^ Fwd | Rate |------------- ............ | \ . | \ . | \ . | \ . |.................-.-.-.-.-.-.---------------- +----+-------+---------------+-----------------> ^ ^ ^ ^ time T0 CEI Ta Ta' Preferred Egress Interface: --- Next-Best Egress Interface: ... This is also indicated in the graph below of the rx rate for route Rta only. ^ rx | rate |------------- ................. Rta | | . | | . |.............-.-.-.-.-.-.-.-.---------------- +----+-------+---------------+-----------------> ^ ^ ^ ^ time T0 CEI Ta Ta' Preferred Egress Interface: --- Next-Best Egress Interface: ... The goal is to derive the time between CEI and Ta' (convergence time of Rta). An obvious way to derive convergence time is to collect a timestamp at CEI and use that in the Rate-Derived Method. For the loss-derived methods, we need to measure the time between T0 and CEI and substract that from the duration of the loss period observed on the Next-Best Egress Interface only (loss-period duration on Next-best Egress: Ta'-T0). Another way is to start dropping packets at Convergence Event Instant. Then the convergence time is equal to the loss of connectivity period duration and we can observe traffic on all DUT egress interfaces to measure convergence time. In the graph below the "=" curve is the Fwd Rate on the Preferred Egress Interface as observed/derived by the Tester. ^ Fwd | Rate |===== ............ | | . | | . | | . | | . |.....=.=.=.=.=.=.=.=.=.=.=.=.================ +----+-------+---------------+-----------------> ^ ^ ^ ^ time T0 CEI Ta Ta' ^ rx | rate |===== ................. Rta | | . | | . |.....=.=.=.=.=.=.=.=.=.=.=.=.================ +----+-------+---------------+-----------------> ^ ^ ^ ^ time T0 CEI Ta Ta' Preferred Egress Interface as observed/derived by Tester: === Next-Best Egress Interface: ... >From the above observations one can derive convergence time (both rate-derived and loss-derived) without the need to collect a timestamp at CEI (and T0). The gain is probably larger for the loss-derived methods since there in general no timestamps need to be collected. The use of the word "drop" is mostly a matter of terminology, I think, but I'm open to other suggestions. "Drop" comes from my view that the Tester does not necessarily need to be a single device. One can add device(s) to the Tester to get to the needed functionality. In the example of an IGP adjacency failure, one can put an additional device "A" (e.g. L2 switch for GE) between the DUT Preferred Egress Interface and the test device. That way the physical link between "A" and test device can be brought down (Convergence Event) while the DUT Preferred Egress Interface will stay up. The Tester (more precisely device "A" of the Tester) will start "dropping" the packets at Convergence Event Instant. Your comments are very much appreciated. Thanks, Kris > -----Original Message----- > From: McLendon, John [mailto:John.McLendon@spirent.com] > Sent: 17 July 2009 20:34 > To: Kris Michielsen; Al Morton; bmwg@ietf.org > Subject: RE: [bmwg] Comment on meth-18. > > Hi Kris, > I'm still not sure what "discard" traffic means in terms of a > test device. > If the test device is monitoring both ports simultaneously > (and all of the testers I am knowledgeable about are capable > of this), then my understanding of the test scenario is that > the test device either detects/causes the switchover, > captures the time of the Convergence Event Instant onset, and > can then apply the equation once full convergence occurs. > > If this is correct, then nothing needs to be said about > traffic received by the tester on the Preferred Egress > Interface unless one wishes to measure the delta between the > onset of the Convergence Event Instant and the end of the > Convergence Event, i.e. one wants to measure the time between > first packet loss and all traffic stopping on the Preferred > Egress Interface, i.e. the Ta/Tb/Ta'/Tb'/Ta"/Tb" timestamps > in Figure 6. > In this case, one might want to state that the tester > collects stats on the both interfaces for the entire test, > altho' this is implicit in the equations and in Figure 6. > > I also have one other comment - the current measurement > period is reported in seconds. The test devices I am aware of > can capture the timestamps with millisecond accuracy > (presuming there is a minimum transmission rate of at least > two packets per millisecond for each route). One might want > to put in a MAY for this. > > Make sense? > J... > > -----Original Message----- > From: Kris Michielsen [mailto:kmichiel@cisco.com] > Sent: Friday, July 17, 2009 11:46 AM > To: McLendon, John; 'Al Morton'; bmwg@ietf.org > Subject: RE: [bmwg] Comment on meth-18. > > John, > > Thanks for reviewing the draft! > > What do you think of the following suggestion? > OLD: > For Convergence Events caused by the Tester, such as an IGP > cost change, the Tester may start to drop all traffic > received from the Preferred Egress Interface at the > Convergence Event Instant to achieve the same result. > > NEW: > For Convergence Events caused by the Tester, such as an IGP > cost change, the Tester may start to discard all traffic > received from the Preferred Egress Interface at the > Convergence Event Instant, or may be able to separately > observe packets received from the Preferred Egress Interface > prior to the Convergence Event Instant, to achieve the same result. > > Thanks, > Kris > > > -----Original Message----- > > From: bmwg-bounces@ietf.org [mailto:bmwg-bounces@ietf.org] > On Behalf > > Of McLendon, John > > Sent: 16 July 2009 00:00 > > To: Al Morton; bmwg@ietf.org > > Subject: [bmwg] Comment on meth-18. > > > > Hi all, > > I have a comment on the following statement in Sec 4, top > of pp. 9, in > > meth-18: > > > > "For Convergence Events caused by the Tester, such as an IGP cost > > change, the Tester may start to drop all traffic received from the > > Preferred Egress Interface at the Convergence Event Instant > to achieve > > the same result." > > > > The word "drop" implies that the test device forwards or drops > > traffic. > > In my experience, test devices neither forward or drop > traffic. They > > may generate or analyze traffic, but a test device dropping traffic > > doesn't seem helpful to the desired measurement. Many test > devices can > > bring a link down at the physical layer. > > > > J... > > > > > > > > -----Original Message----- > > From: bmwg-bounces@ietf.org [mailto:bmwg-bounces@ietf.org] > On Behalf > > Of Al Morton > > Sent: Wednesday, July 15, 2009 3:48 PM > > To: bmwg@ietf.org > > Subject: Re: [bmwg] WGLC: draft-ietf-bmwg-igp-dataplane drafts > > > > Comments on meth-18, all nits, > > Al (mostly as participant) > > > > At 01:44 PM 7/14/2009, Al Morton wrote: > > >...This message begins a Last call on the IGP-Dataplane > Convergence > > >Time Benchmarking drafts. > > > > > > >http://tools.ietf.org/html/draft-ietf-bmwg-igp-dataplane-conv-term-18 > > > >http://tools.ietf.org/html/draft-ietf-bmwg-igp-dataplane-conv-meth-18 > > > >http://tools.ietf.org/html/draft-ietf-bmwg-igp-dataplane-conv-app-17 > > > > > >The Last Call will end on July 31, 2009. > > > > > > Section 4, top of page 10, > > s/At least one condition need to /At least one condition needs to / > > > > > > 5.7. Measurement Accuracy > > ... When packet jitter is much less than the convergence > > time, it is a negligible source of error and therefor > it will be > > s/therefor/therefore/ > > > > 5.9 Tester Capabilities > > Add "Also see section 6 for method-specific capabilities." > > > > 6.1.3. Measurement Accuracy > > > > TBD > > (can't leave this TBD, if there's no obvious material, > delete section > > ?) > > > > > > Section 7, Reporting Format > > > > Maximum Packet Delay Threshold seconds > > this is called "Forwarding Delay Threshold" in the term doc, right? > > pick one for both... > > > > Section 9, same as terms, use the standard BMWG paragraphs... > > > > > > > > > > _______________________________________________ > > bmwg mailing list > > bmwg@ietf.org > > https://www.ietf.org/mailman/listinfo/bmwg > > > > <DIV><FONT size="1"> > > > > E-mail confidentiality. > > -------------------------------- > > This e-mail contains confidential and / or privileged information > > belonging to Spirent Communications plc, its affiliates and / or > > subsidiaries. If you are not the intended recipient, you are hereby > > notified that any disclosure, copying, distribution and / or the > > taking of any action based upon reliance on the contents of this > > transmission is strictly forbidden. If you have received > this message > > in error please notify the sender by return e-mail and > delete it from > > your system. If you require assistance, please contact our IT > > department at helpdesk@spirent.com. > > > > Spirent Communications plc > > Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 > 9XN, United > > Kingdom. > > Tel No. +44 (0) 1293 767676 > > Fax No. +44 (0) 1293 767677 > > > > Registered in England Number 470893 > > Registered at Northwood Park, Gatwick Road, Crawley, West > Sussex, RH10 > > 9XN, United Kingdom. > > > > Or if within the US, > > > > Spirent Communications, > > 26750 Agoura Road, Calabasas, CA, 91302, USA. > > Tel No. 1-818-676- 2300 > > > > </FONT></DIV> > > _______________________________________________ > > bmwg mailing list > > bmwg@ietf.org > > https://www.ietf.org/mailman/listinfo/bmwg > > > > > <DIV><FONT size="1"> > > E-mail confidentiality. > -------------------------------- > This e-mail contains confidential and / or privileged > information belonging to Spirent Communications plc, its > affiliates and / or subsidiaries. If you are not the intended > recipient, you are hereby notified that any disclosure, > copying, distribution and / or the taking of any action based > upon reliance on the contents of this transmission is > strictly forbidden. If you have received this message in > error please notify the sender by return e-mail and delete it > from your system. If you require assistance, please contact > our IT department at helpdesk@spirent.com. > > Spirent Communications plc, > Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, > United Kingdom. > Tel No. +44 (0) 1293 767676 > Fax No. +44 (0) 1293 767677 > > Registered in England Number 470893 > Registered at Northwood Park, Gatwick Road, Crawley, West > Sussex, RH10 9XN, United Kingdom. > > Or if within the US, > > Spirent Communications, > 26750 Agoura Road, Calabasas, CA, 91302, USA. > Tel No. 1-818-676- 2300 > > </FONT></DIV> >
- [bmwg] FW: Comment on meth-18. Kris Michielsen