[bmwg] Draft minutes of BMWG at IETF-57

[Al Morton <[e-mail]>]

BMWG,

Please look these draft minutes over for clarity
and any questions, etc.  Comments to co-chairs or
to the list by July 25th, please.

Thanks,
Al/Kevin


Benchmarking Methodology WG (bmwg)

Tuesday, July 15, 2003, 1300-1400 and 1415-1515
=======================================

CHAIRS Kevin Dubray <[e-mail]>
           Al Morton <[e-mail]>

Reported by Al Morton, based on the information generously
compiled by Sue Hares as official note-taker.

About 66 people attended one or more of the BMWG sessions.

1.   Working Group Status (Morton)
Status of BMWG I-Ds at close of IETF-57

AD/IESG Review
<draft-ietf-bmwg-conterm-05.txt>, revised to reflect IESG input, WG LC soon.

I-D Last Call
<draft-ietf-bmwg-fib-meth-01.txt>,  Call ended 3/14.
<draft-ietf-bmwg-dsmterm-07.txt>, Call ended 7/15 with comment.
<draft-ietf-bmwg-mcastm-13.txt>, Call ended 7/5, to ADs soon
<draft-ietf-bmwg-ospfconv-term-05.txt>, Call ended 7/1 with comments.
<draft-ietf-bmwg-ospfconv-intraarea-04.txt>, Call ended 7/1
<draft-ietf-bmwg-ospfconv-applicability-03.txt> Call ended 7/1

I-Ds
<draft-ietf-bmwg-dsmmeth-00.txt>, coming soon
<draft-ietf-bmwg-ipsec-term-01.txt>(draft 07/2003, need readers)
<draft-ietf-bmwg-benchres-term-03.txt>, back in WG
<draft-ietf-bmwg-acc-bench-term-00.txt> New 06/2003
<draft-ietf-bmwg-igp-dataplane-conv-term-00.txt> New 06/2003
<draft-ietf-bmwg-igp-dataplane-conv-meth-00.txt> New 06/2003
<draft-ietf-bmwg-igp-dataplane-conv-app-00.txt> New 06/2003


2.   Discussion on OSPF Convergence Benchmarking WG Last
      Call comments. Last call is over, but a few comments need
      discussion.  Identification of remaining issues.  (Morton)

http//www.ietf.org/internet-drafts/draft-ietf-bmwg-ospfconv-term-04.txt
http//www.ietf.org/internet-drafts/draft-ietf-bmwg-ospfconv-intraarea-05.txt
http//www.ietf.org/internet-drafts/draft-ietf-bmwg-ospfconv-applicability-03.txt

There was one remaining comment from the Last Call ending July 1,
asking to remove definitions of "Internal Measurements" and "External 
Measurements".
Following a short discussion, the commentor (S.Proetsky) agreed that the
definitions should remain, but that the terms most frequently used in
BMWG drafts and discussion are "White Box" and "Black Box" measurements.
The term names should be changed to add the "Boxes" as synonyms
and any resulting changes to the definition text. This potential resolution
will be discussed with authors and on the list.


3.  Benchmarking Network-layer Traffic Control Mechanisms
     Terminology -- (J. Perser et al.)

     http//www.ietf.org/internet-drafts/draft-ietf-bmwg-dsmterm-07.txt

Jerry Perser communicated the latest changes and offered one last
chance to comment during WG Last Call. There was one comment on the
"Channel Capacity" term and its definition from Tony DeLaRosa.
This comment was resolved during a face-face meeting later in the week.

     Also, Future Draft on Benchmarking Traffic Control Mechanisms
     Methodology --

Scott Poretsky gave a brief introduction and identified areas for comment
and contribution during development.
The purpose statement is to provide methodologies to benchmark devices
"capable of delivering the specific packet forwarding treatment indicated by
the DS field value" [RFC2474].  The Benchmark is a Device’s ability to meet
configured PHBs (the Expected Vector), not on specific QoS mechanisms. This 
makes
it possible to compare performance of routers implementing different 
mechanisms.
A short discussion revealed that measurement time scale will be important
for EF evaluation.

4.  Terminology for Benchmarking BGP convergence in the Control Plane
http//www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-conterm-05.txt

This draft was recently revised, and there will be
a short WG Last Call before returning it to the ADs.


5.  IPsec Device Benchmarking Terminology I-D --  (M. Kaeo, et al.)

     http//www.ietf.org/internet-drafts/draft-ietf-bmwg-ipsec-term-01.txt

The text of IPsec Terminology was completed in this version (01).
Merike asked BMWG if there were any terms missing, any problems
with Definitions, and if the group believed this draft was ready for
WG Last Call. There were several readers: one offering an endorsement,
and two questions on the scope (NAT devices and IKEv2 are out of scope,
however NAT-Transversal is included in the scope of this effort).
The current draft defined IMIX (Internet Mix for traffic synthesis),
but did not include the reference that Michele Bustos provided on the list.
There was a comment suggesting that this is a good definition to include,
with the caveat that no one mix of packet sizes represents Internet usage.
There will be a revision to make these points clear, and then a WG Last Call.


6.  IGP Data plane convergence benchmark I-Ds --   (S.Poretsky)

http//www.ietf.org/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-term-00.txt
http//www.ietf.org/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-meth-00.txt
http//www.ietf.org/internet-drafts/draft-poretsky-igp-convergence-app-00.txt

Scott Poretsky gave a brief introduction of the new work item.
All three drafts are progressing with comments.
There were several bmwg-list suggestions for terminology changes,
and they were illustrated in the viewgraphs. Discussion of the methodology
identified the need for a note to cover forwarding rate change
after convergence, and another suggested specific reporting
formats that remove the effect of delay timers. There was also a discussion
of measurement sampling rate and its affect on the accuracy of loss-based
convergence time measurements when convergence approaches the sampling 
interval.


2nd Session
Tuesday, July 15, 2003, 1415-1515
=======================================

7.  Resource Reservation Terminology benchmark I-Ds --  (Fehér Gábor Gume)

http//www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-benchres-term-03.txt

Description of changes based on AD review, primarily to remove Diffserv
material, sync-up with current NSIS drafts, and remove non-standard examples
(e.g., Boomerang).There were still a few places where comments could be more
thoroughly adopted, and the draft will be revised again shortly.
Gábor made an appeal for more readership and review on BMWG-list.


8.  Terminology for Benchmarking Core Router Accelerated Life Testing.
     (S. Poretsky et al.).

http//www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-acc-bench-term-00.txt

Comments asked for examples of Management Plane Failures, and identified 
problems
with allowing complete flexibility in test configurations (results comparisons
will likely be confounded by differences). The level of readership and
review is not yet clear for this draft.


9.  Proposed Milestone Revisions -- (Morton)

The review of milestones revealed a growing list, and a need to
achieve "Done" on a few more before adding new work.
Editors and list members should comment on this proposal (on the list).

    Done  First I-D on IPsec Device Benchmarking Terminology
    Aug 03  Methodology for IP Multicast Benchmarking to AD Review.
    Nov 03  Resource Reservation Benchmarking Terminology to AD Review
    Aug 03  Net Traffic Control Benchmarking Terminology to AD Review
    Dec 03  Methodology for FIB related Router Performance Benchmarking to 
AD review.
    Dec 03  EGP Convergence Benchmarking Terminology to AD Review
    Mar 04  Resource Reservation Benchmarking Methodology to AD Review
    Dec 03  Basic BGP Convergence Benchmarking Methodology to AD Review.
    Dec 03  Net Traffic Control Benchmarking Methodology to AD Review.
    Dec 03  IPsec Device Benchmarking Terminology to AD Review

    Dec 03   AD review on IGP/Data-Plane Terminology I-D
    Mar 04   AD review on IGP/Data-Plane Methodology and Applicability I-Ds

    Nov 03   AD review on Router Accelerated Test Terminology I-D
    Jul 04   AD review on Router Accelerated Test Methodology and 
Applicability I-Ds

10. New work proposal Automatic Protection Switching Benchmark Terminology
     (T.Kimura, J.Perser)
http//www.ietf.org/internet-drafts/draft-kimura-protection-term-01.txt

     Related Individual Draft on MPLS Protection Benchmarking Methodology
     (S.Poretsky, et al.)

http//www.ietf.org/internet-drafts/draft-poretsky-mpls-protection-meth-00.txt

The Goal has been articulated as follows:

     The objective of this effort is to produce a terminology and
     methodology set of drafts that specifies the performance
     benchmarking sub-IP layer resiliency and protection technologies.
     There is a common terminology draft and multiple methodology drafts
     for the technologies.  The methodology drafts will include (but not
     limited to) Automatic Protection Switching (APS) for SONET/SDH,
     Fast Reroute for Multi-Protocol Label Switching (MPLS), and
     Resilient Packet Ring (RPR) standardized in IEEE.

Discussion revealed that the previous direction to have a common
terminology draft for this work will be stretched by the plethora
of Layer 2 terms, and so this issue must be worked further to see if a
common terminology is possible. The best way forward may be to link
the technology-specific terms to the appropriate IP layer terms.
Recovery time is key concept: accurate measurement takes all 5 recovery types
into account: Lost packets, Induced delays, Duplicate packets,
Out-of-order packets, and Errored packets. Clearly, measuring packet loss
alone to assess Recovery Time will not be sufficient in some cases.
The general assumption here is that the protection recovery times will
be sufficiently fast to avoid triggering IGP reconvergence,
in response to Craig White's inquiry on this interaction.
Craig also commented that other transport components interact with
these protection mechanisms, and Tony DeLaRosa added that a
System Under Test may be needed to accommodate this point.
Matt Mathis suggested using general terms for the paths,
such as primary and secondary (and tertiary), and that we should
invent some term for the triggered failure, like "induced failure."
Also, a generic IP Protection Methodology may be
useful to benchmark unspecified protection mechanisms. The discussion
concluded that the proposal requires more work before review by the WG.


_______________________________________________
bmwg mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/bmwg