[bmwg] FW: RE: Benchmarking Network-layer Traffic Control Mechanisms extension for artificial congestion

["Richard Watts \(riwatts\)" <[e-mail]>]

Hi Scott,
 
as per previous mail below please see proposed updates for discussion.
 
I have deliberately attempted to avoid introducing any new terminology
here, but we may have to include 'artificial forwarding congestion' to
the terminology draft as well. 
Hopefully the text in the updates below provides explanatory text for
the two following test topology diagrams
 
Discussion:
So the purpose is to provide an 'extension' to the existing '
Benchmarking Network-Layer Traffic Control Mechanisms draft' for
artificial congestion. The decision to be made is whether a separate
draft is required e.g. Benchmarking Network-layer Traffic Control
Mechanisms extension for artificial congestion or, are we able to
accommodate this extension within the existing draft. I would like to
propose/suggest that as the majority of the
concepts/terminology/methodology are the same that we accommodate this
within the existing draft.
 
'Artificial' congestion can be created on virtual/logical interfaces by
Traffic Control mechanisms, such that the Forwarding Capacity is limited
and typically less than the Forwarding Capacity of the actual interface.
Essentially the same test methodologies can be applied and the same
terminology (just different mechanism for creating congestion), but with
the consideration that the Output Vector will be based on the 'Limited'
Forwarding Capacity due to the applied Traffic Control Mechanisms
versus, the full Forwarding Capacity of the interface.
 
Updates to Existing Draft:
 
re: Section 3.1. Test Topologies
 
<To be added..>
 
Figure 3 shows the test topology for benchmarking performance when
'artificial' Forwarding Congestion does not exist on the egress link.
This topology is to be used when benchmarking the Undifferentiated
Response and the Traffic Control without 'artificial' Forwarding
Congestion.
 
'Artificial' Forwarding Congestion does not exist due to the fact that
the Offered Vector (Offer Load) does not exceed the 'limited' Forwarding
Capacity of the Traffic Control Mechanisms
 
Figure 4 shows the test topology for benchmarking performance when
'artificial' forwarding congestion does exist on the egress link. This
topology is to be used when benchmarking the Traffic Control with
'artificial' Forwarding Congestion
 
'Artificial' Forwarding Congestion is produced by an Offered Vector
(Offered Load) to an ingress interface on the DUT destined for a single
egress interface configured with traffic control mechanisms that limits
the Output Vector to a value ' less than ' full interface Output Vector.
 
 
 
        Expected                                 
        Vector                                       
           |                                            
           |                                         
           \/                                        
        ---------        Offered Vector (Limited) ---------
        |       |<--------------------------------|       |
        |       |                                 |       |
        |       |                                 |       |
        |  DUT  |                                 | Tester|
        |       |                                 |       |
        |       |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>|       |
        |       |        Output Vector (Limited)  |       |
        ---------                                 ---------
                      
           Figure 3. Test Topology for Benchmarking 
                     Without 'artificial' Forwarding Congestion
 
 
        Expected                                  
        Vector                                        
           |                                            
           |                                         
           \/                                        
        ---------       Offered Vector (Unlimited)---------
        |       |<--------------------------------|       |
        |       |                                 |       |
        |       |                                 |       |
        |  DUT  |                                 | Tester|
        |       |                                 |       |
        |       |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>|       |
        |       |         Output Vector (Limited) |       |
        ---------                                 ---------
                                             
               Figure 2. Test Topology for Benchmarking

                         With 'artificial' Forwarding Congestion
Re-use of existing and slight modifications to the Test Cases to be
added as below
4. Test Cases 
 
  4.4 Undifferentiated Response  
 
     Purpose:  
     To establish the baseline performance of the DUT.
 
     Procedure:
     1. Configure DUT with Expected Vector.
     2. Configure the Tester for the Offered Vector.  
        Number of DSCPs MUST equal 1 and the 
        RECOMMENDED DSCP value is 0 (Best Effort). 
        Use 1000 Flows identified by IP SA/DA.  All flows
        have the same DSCP value.
     3. Using the Test Topology in Figure 3, source the 
        Offered Load from the Tester to the DUT.
     4. Measure and record the Output Vector. 
     5. Maintain offered load for 10 minutes minimum 
        to observe possible variations in measurements.
     6. Repeat steps 2 through 5 with 10000 and 100000
        Flows.
 
     Expected Results:
     Forwarding Vector equals the Offered Load.  There 
     is no packet loss and no out-of-order packets.
 
 
  4.5 Traffic Control Baseline Performance
     Purpose:  
     To benchmark the Output Vectors for a Codepoint Set
     without 'artificial' Forwarding Congestion.
 
     Procedure:
     1. Configure DUT with Expected Vector for each DSCP in 
        the Codepoint Set.
     2. Configure the Tester for the Offered Vector.  
        Number of DSCPs MUST 2 or more. Any DSCP values can
        be used.  Use 1000 Flows identified by IP SA/DA
        and DSCP value.
     3. Using the Test Topology in Figure 3, source the 
        Offered Load from the Tester to the DUT.
     4. Measure and record the Output Vector for each DSCP
        in the Codepoint Set. 
     5. Maintain offered load for 10 minutes minimum 
        to observe possible variations in measurements.
     6. Repeat steps 2 through 5 with 10000 and 100000
        Flows.
     7. Increment number of DSCPs used and repeat steps 
        1 through 6.
 
     Expected Results:
     Forwarding Vector equals the Offered Load.  There is  
     no packet loss and no out-of-order packets.  Output 
     vectors match the Expected Vectors for each DSCP in 
     the Codepoint Set.
 
Poretsky                                              [Page 6] 
   
INTERNET-DRAFT       Methodology for Benchmarking         March 2006 
                 Network-layer Traffic Control Mechanisms 
 
  4.6 Traffic Control Performance with Forwarding Congestion
     Purpose:  
     To benchmark the Output Vectors for a Codepoint Set
     with 'artificial'Forwarding Congestion.
 
     Procedure:
     1. Configure DUT with Expected Vector for each DSCP in 
        the Codepoint Set.
     2. Configure the Tester for the Offered Vector.  
        Number of DSCPs MUST 2 or more. Any DSCP values can
        be used.  Use 1000 Flows identified by IP SA/DA
        and DSCP value.  The Offered Load MUST exceed the
        'limited'Forwarding Capacity of a single egress link by 25%.
     3. Using the Test Topology in Figure 4, source the 
        Offered Load from the Tester to the DUT.  The 
        ingress offered load MUST exceed 
        the 'limited' Forwarding Capacity of the egress link to 
        produce Forwarding Congestion.
     4. Measure and record the Output Vector for each DSCP
        in the Codepoint Set. 
     5. Maintain offered load for 10 minutes minimum 
        to observe possible variations in measurements.
     6. Repeat steps 2 through 5 with 10000 and 100000
        Flows.
     7. Increment offered load by 25% to 200% maximum.
     8. Increment number of DSCPs used and repeat steps 
        1 through 6.
 
     Expected Results:
     Forwarding Vector equals the Offered Load.  There is  
     no packet loss and no out-of-order packets.  Output 
     vectors match the Expected Vectors for each DSCP in 
     the Codepoint Set.
Scott, I am keen to hear your thoughts and please can you provide
comment on this proposal or make alternative suggestions so that we can
then make the appropriate changes following discussion, to the existing
draft ?
 
Kind Regards
 
Richard

________________________________

From: Richard Watts (riwatts) 
Sent: 05 May 2006 16:01
To: '[e-mail]'
Cc: [e-mail]
Subject: RE: Benchmarking Network-layer Traffic Control Mechanisms
extension for artificial congestion



Hi Scott

As we discussed and agreed I am keen to provide input and support to the
above mentioned draft with respect to making artificial congestion  an
extension to the existing benchmarking draft.

Please see my initial comments about the existing draft that I forwarded
a little while ago and I will also send out soon what I think the
wording may be for blending in the artificial congestion aspects of the
benchmarking, so that we might get some dialogue going on this topic.
The next ITEF in Montreal is not that long away now and I look forward
to meeting both you and the rest of the group then.

Look forward to your fedback

 

Regards

 

Richard

 

I would like to

Hi Scott

Apologies for the slight delay in getting back to you, time has been a
bit of a challenge, as always. However, please see below some 'cosmetic'
comments and queries regarding the existing benchmarking methodology
draft, which I hope are useful

Re: Section 3.1 Test Topologies

There seems to be a slight typo in the text i.e. ' Figure 2 shows the
Test Topology for benchmarking performance when Forwarding Congestion
does not exist on the egress link '. The 'not' needs to be removed to
align with Figure 2 heading.

Re: Section 3.2.3 c) under Offered Vector

' Packet size must be equal to or less than the interface MTU so that
there is no fragmentation' The 'must' needs to be in upper case.

Re: Section 3.2.5 Expected Vector

The last sentence 'Test cases may be repeated with variation to the
expected vector to produce a more benchmark results'. I take this to
mean vary the SLA requirements such as packet loss, jitter, forwarding
delay etc. If so is this actually required, I understand that the draft
uses the word 'may' so infers optional. But, if the DUT is tested to the
tightest SLAs and they are achieved, is there any mileage in testing to
achieve 'less tight' SLAs ?

Re: Step 2 in the procedure for both 4.2 and 4.3 should have 'be'
inserted between the 'MUST' and '2 or more'

Re: 'Expected Results' under section 4.3 states ' Forwarding Vector
equals the Offered Load. There is no packet loss and no out of order
packets. Output vectors match the Expected Vectors for each DSCP in the
Codepoint set' 

Should we not ensure consistency in terminology and change Forwarding
Vector to Output Vector as per bottom of Page 5 or, change Output Vector
on bottom of Page 5 to be consistent with Forwarding Vector in this
section ? Additionally, it states 'Forwarding Vector equals the Offered
Load'. Offered load, should be Expected Vector as this is the process
for the benchmarking of 'with' forwarding congestion ?

Not sure what your thoughts are, but I would not be inclined to state
anything about what the expected vector should be in the expected
results section, as this will vary depending upon what the target is for
the benchmark and how its configured on the DUT. So comments about no
packet loss may not be valid.

I am personally of the opinion that we can manipulate this draft to take
into account the artificial congestion, I think we just need to weave in
the appropriate wording so that the audience is aware that this
methodology applies also to artificial congestion. I think the concepts
and approach do not change, just because the mechanism for creating
congestion might differ.

If you are in agreement I will go ahead and try to make the appropriate
changes for your review, comment and input ?

I will also review the Terminology draft very shortly as well as
feedback any comments to yourself and the co-authors

Kind Regards

Richard

-----Original Message-----

From: Richard Watts (riwatts) 

Sent: 24 March 2006 10:57

To: [e-mail]; Gunter Van de Velde (gvandeve)

Cc: [e-mail]; [e-mail]; Richard Watts (riwatts)

Subject: RE: Benchmarking Network-layer Traffic Control Mechanisms
extension for artificial congestion

 

Hi Scott

Many thanks for your invite to co-author the current methodology draft
of which I will gladly accept.

I also agree with your approach with how to potentially move forward
with the methodology document(s). It would be easier I guess if we could
leverage the existing methodology document rather than, having to create
a new/separate one.

My initial thoughts are that we should be able to use the existing
methodology as we are still creating congestion (just artificially)
through the use of shapers and the like on virtual links, that said,
with HQF architectures, we have tiered levels of congestion management,
without the need to create artificial congestion through shaping.

Will start looking at the terminology and methodology documents to see
how best we might address this.

Once again, many thanks for your cordial invitation and your acceptance
to co-author, should we need to generate a separate methodology
document.

Kind Regards

Richard

-----Original Message-----

From: [e-mail] [mailto:[e-mail]
<mailto:[e-mail]> ] 

Sent: 22 March 2006 17:15

To: Gunter Van de Velde (gvandeve)

Cc: Richard Watts (riwatts); [e-mail]; [e-mail]

Subject: RE: Benchmarking Network-layer Traffic Control Mechanisms
extension for artificial congestion

Gunter,

Hello. It was a pleasure to meet you yesterday. Great work on IPv6! I am

looking forward to the author team's further work on it. 

The current Network-Layer Taffic Control methodology addresses
benchmarking

of egress QoS mechanisms, without naming specific mechanisms or

implementations. Yesterday's BMWG meeting raised the need for the

Network-Layer Taffic Control work item to have methodologies that
addressed

classification/shaping and application of DiffServ to virtual links.
First

we will need to look at how classification/shaping and application of

DiffServ to virtual links can be addressed in the current methodology

document. If we determine that these require separate methodology

documents, then it was agreed that these methodologies can be addressed
as

separate documents as part of the current Network-Layer Taffic Control
work

item using the existing Terminology document. If you agree with this

approach then I would be happy to participate as co-author for either of

these methology drafts, if we determine the documents are needed.
Likewise,

I would like to invite you or your colleague to join as co-author on the

current methodology draft.

Thanks!

Scott

-----Original Message-----

From: Gunter Van de Velde (gvandeve) [mailto:[e-mail]
<mailto:[e-mail]> ]

Sent: Wednesday, March 22, 2006 11:55 AM

To: [e-mail]

Cc: [e-mail]; [e-mail]; [e-mail]

Subject: Benchmarking Network-layer Traffic Control Mechanisms extension

for artificial congestion

 

Hi Scott,

Many thanks yesterday for your presentation and insights in the

Benchmarking test methodology for Network-Layer Control Mechanisms.

As mentioned during the BMWG meeting, a congestion scenario seen often

is that of artificial congestion caused by a diffserv traffic shaping 

function.

This is as you know commonly seen at the boundary of an network to
condition

the traffic for the right parameters (whatever these parameters actually

are).

I would like to pick up the task to be involved with this work, and
would

like

to invite you to be one of the co-authors to advice and share your

experience

in the benchmarking area. Please let me know if you are interested in
this 

contributing

role? I would like to introduce Richard Watts who is based in UK and is
an 

expert in QoS deployment (he leads a QoS expert team in Europe). Richard


offered to take the

lead editor role for this piece of work. This would mean that if you
accept

co-authorship we will be with the three to start the work.

Would you or Al have any recommended next steps in mind so that

we can present first draft results at IETF66?

My believe is that this work should use the
draft-ietf-bmwg-dsmterm-12.txt

and

draft-ietf-bmwg-dsmmeth-01.txt as foundation and complement these two

documents with two new drafts. Consequence is that the existing drafts

will have to be included as 'normative reference' which sounds logical

and acceptable to me.

The first question is now on how to proceed? Should we initially only

prepare a new terminology document for IETF66 or should we do in
addition

the methodology draft immediately in parallel?

Any suggestions and advice is welcome,

Kind Regards,

G/

_______________________________________________
bmwg mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/bmwg