RE: RE: [bmwg] BMWG Last Call: draft-ietf-bmwg-igp-dataplane-conv -*** -10

[[e-mail]]

Yes Al.  You bring up an important distinction that 
1. the Offered Load to the ingress interface of the DUT should equal the
maximum Throughput as defined in RFCs 1242 and 2544
AND
2. the Forwarding Rate as defined in RFC 2285 is measured at the egress
interfaces

Based upon our email thread and phone call I will make the following changes
for the -11 Terminology and Methodology documents->

TERMINOLOGY
FROM:
1. Introduction
.
.
.
   An example of Route Convergence as observed and measured from the 
   data plane is shown in Figure 1.  The graph in Figure 1 shows 
   Throughput versus Time.  Time 0 on the X-axis is on the far 
   right of the graph.  The components of the graph and metrics are 
   defined in the Term Definitions section.  

                           Convergence    Convergence 
                           Recovery         Event   
                           Instant         Instant      Time = 0sec

           Maximum            ^              ^             ^
           Throughput--> ------\    Packet   /---------------
                                \    Loss   /<----Convergence
              Convergence------->\         /      Event Transition

        Recovery Transition       \       /
                                   \_____/<------Maximum Packet Loss

 

        X-axis = Time
        Y-axis = Throughput                
                        Figure 1. Convergence Graph

CHANGE TO
   An example of Route Convergence as observed and measured from the 
   data plane is shown in Figure 1.  The graph in Figure 1 shows 
   Forwarding Rate versus Time.  Time 0 on the X-axis is on the far 
   right of the graph.  The Offered Load to the ingress interface of 
   the DUT SHOULD equal the measured maximum Throughput [5,6] of the DUT 
   and the Forwarding Rate [7] is measured at the egress interfaces 
   of the DUT.   The components of the graph and the metrics are defined 
   in the Term Definitions section.  

                          Convergence    Convergence 
                           Recovery         Event   
                           Instant         Instant  Time = 0sec

   Forwarding Rate =          ^              ^        ^     Offered Load =
        Offered Load --> ------\    Packet   /--------- <---Max Throughput
                                \    Loss   /<----Convergence
              Convergence------->\         /      Event Transition

        Recovery Transition       \       /
                                   \_____/<------Maximum Packet Loss

 

        X-axis = Time
        Y-axis = Forwarding Rate
                        Figure 1. Convergence Graph

FROM                
3.14 Packet Sampling Interval
        Definition:  
        The interval at which the tester (test equipment) polls to make 
        measurements for arriving packet flows.

        Discussion: 
        Metrics measured at the Packet Sampling Interval may include
        Throughput and Convergence Packet Loss.

CHANGE TO 
	. . . 
	  Discussion: 
        Metrics measured at the Packet Sampling Interval MUST include
        Forwarding Rate and Convergence Packet Loss.

ADD REFERENCES
1242 for Throughput
2544 for Throughput
2285 for Forwarding Rate

METHODOLOGY
FROM
3.2.7 Offered Load
   The offered Load MUST be the Throughput of the device as defined 
   in [5] and benchmarked in [6] at a fixed packet size.  
   Packet size is measured in bytes and includes the IP header and 
   payload.  The packet size is selectable and MUST be recorded.  
   The Throughput MUST be measured at the Preferred Egress Interface 
   and the Next-Best Egress Interface.  The duration of offered load 
   MUST be greater than the convergence time.  The destination 
   addresses for the offered load MUST be distributed such that all 
   routes are matched.  This enables Full Convergence [2] to be 
   observed.

CHANGE TO 
3.2.7 Offered Load
   . . . 
   The Forwarding Rate MUST be measured at the Preferred Egress Interface 
   and the Next-Best Egress Interface.  . . . 

ADD REFERENCE
2285 for Forwarding Rate


-----Original Message-----
From: Al Morton [mailto:[e-mail]]
Sent: Monday, April 10, 2006 3:44 PM
To: [e-mail]
Subject: Fwd: RE: [bmwg] BMWG Last Call:
draft-ietf-bmwg-igp-dataplane-conv-*** -10


BMWG,

I took a last look at the IGP Dataplane drafts today,
and had the short exchange with Scott, below.
Scott will respond with the proposed resolution
for your consideration.

I hope this will prompt others to take one last look...

Al


>Date: Mon, 10 Apr 2006 14:41:04 -0400
>To: [e-mail]
>From: Al Morton <[e-mail]>
>Subject: RE: [bmwg] BMWG Last Call: draft-ietf-bmwg-igp-dataplane-conv-***
-10
>
>Scott,
>
>I understand that a pre-condition is to (first find and then) offer
>the "Throughput" traffic level to the DUT. And also that Throughput
>is defined in RFC 2544 only, in fact, the procedure in Sections 23
>and 24 for Trials seem to be the details that encumber the Throughput
>benchmark and make it difficult to use when measuring rate-derived
>convergence, especially the queue bleeding-off time (2 seconds).
>
>So, what I thought was used here was a combination of Throughput
>and Forwarding Rate benchmarks.
>
>  - Throughput is needed to establish the "maximum" traffic load
>    and conduct the benchmark test for Convergence Time
>    (regardless whether Rate-Derived or Loss-Derived CT is measured).
>
>  - Forwarding Rate is needed to characterize the Rate-Derived
>    Convergence Time, specifically to determine which 100 ms
>    packet sampling intervals contain the "convergence event instant"
>    and "convergence recovery instant"  as in figure 1 of the terms draft.
>
>In other words, I think you can read a series of forwarding rate
>measurements taken at Throughput level, using 100 ms sampling:
>
>500  500  500                            500  500  500  ... 500 pkt/s
>                125                  200
>                    0  0  0  0  0  0
>and calculate the convergence time of this event to be 9 x 100ms = 900 ms
>
>I guess that my difficulty is that I cant see how to determine the
>"convergence event instant" and "convergence recovery instant"
>without making a measurement at egress, and getting a new measurement
>every 100ms (which Forwarding Rate can do).
>
>But as I said, I think a straight Throughput measurement works fine
>for Loss-derived convergence time.
>
>This may be a point that's hard to make by e-mail,
>feel free to call me if you want...
>
>Al
>
>At 12:35 PM 4/10/2006, [e-mail] wrote:
>>Hi Al,
>>
>>This work item does not define Forwarding Rate nor Throughput.  In order
to
>>make the Convergence Time measurement it is necessary to provide a known
>>Offered Load for which there is no packet loss.
>>Throughput is the measurement of this value obtained as specified in RFC
>>2544.  Given that value as defined and measured in another BMWG document,
>>the Loss-Derived Convergence Time can be measured using the IGP Data Plane
>>Covergence methodology.  Throughput is a better choice than Forwarding
Rate
>>since it is required that there be no packet loss.
>>
>>Does this help?
>>
>>Scott
>>
>>-----Original Message-----
>>From: Al Morton [mailto:[e-mail]]
>>Sent: Monday, April 10, 2006 12:04 PM
>>To: [e-mail]
>>Subject: Re: [bmwg] BMWG Last Call:
>>draft-ietf-bmwg-igp-dataplane-conv-***-10
>>
>>
>>At 11:44 AM 4/3/2006, Al Morton wrote:
>> >BMWG:
>> >
>> >A WG Last Call period for the Internet-Drafts titled:
>> >...
>> >will be open from 3 April through 10 April 2006.
>>
>>Scott, (off-list)
>>
>>I have been reviewing the drafts again, with an eye toward
>>filling-out the shepherding form and making a publication request.
>>(in fact, the paper copies showed-up this morning, thanks!)
>>
>>I'm struggling a bit with the change from Forwarding Rate
>>to Throughput.  At first I was concerned that this might have
>>been one of *my* previous comments and I had led you astray!
>>But I found that this comment dates back to the WGLC on version 06
>>(July '05), and Timmons made the comment, finding an inconsistency
>>between the term and meth drafts.  You concluded that the better
>>choice was Throughput, and implemented this change in version 09.
>>
>>Unfortunately, the last time the WG really read the drafts was version 08,
>>during the October 2005 WGLC, and I'm a bit concerned that this late
>>change might cause some problems in the future.  Perhaps you
>>can set me straight on this, and put my mind at ease.
>>
>>Let me briefly describe my concern:
>>
>>It doesn't seem possible to make the repeated Throughput measurements
>>necessary to collect the Rate-Derived Convergence Time.
>>You have to
>>
>>   - repeatedly measure packet rate at the egress interfaces to see when
the
>>     rate begins to drop and when it returns to the pre-convergence rate
>>     (throughput is a measure of offered load, either loss-less or not
>>      in each successive trial)
>>
>>   - make measurements in very short time intervals, like 100ms
>>     (Throughput Trials require you to wait 2 seconds afterward
>>     to bleed the queues, so you only get one Throughput measurement
>>      per trial, and whether it was a loss-less trial or not)
>>
>>At the same time, I can see how performing a single Throughput Trial
>>allows us to calculate the Loss-derived Convergence time.  One simply
>>determines the Frame loss count, and relates that count to time.
>>
>>So help me out, what dots am I failing to connect this morning?
>>
>>Al


_______________________________________________
bmwg mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/bmwg

_______________________________________________
bmwg mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/bmwg