[ippm] Draft minutes of the Washington IPPM meeting

["Henk Uijterwaal (RIPE NCC)" <[e-mail]>]

Dear All,

Apologies for the lateness of these minutes.  Below is the first and
probably final draft of the minutes of the IPPM meeting in Washington last
month. If you have comments, please send them to Matt and myself before
16:00 ET today.

Henk

- - - - -



IP Performance Metrics WG (ippm)

Tuesday, November 9 at 0900-1130
=================================

The meeting was chaired by Henk Uijterwaal and Matt Zekauskas. Al Morton
acted as scribe with additional notes by Matt; raw notes were edited into
these minutes by the chairs.

Agenda:

1. Review of Agenda and Status of Milestones
2. One-Way Active Measurements Protocol
3. Reordering drafts
4. Implementation reports for RFC2678-2681.
5. Multiparty Communications Parameters and Metrics.
6. Status of the Link Bandwidth Draft.
7. Discussion on an open network performance testing protocol.



1. Review of Agenda and Status of Drafts and Milestones

Henk opened the meeting with a review of the agenda.  No one had
any suggestions for further changes.  He then went on to review
working group status.  The link bandwidth draft was due in August;
although there is no draft to date there has been progress, and
Phil Chimento will report on current thinking later.  There are
four drafts in progress.  The MIB draft did not show sufficient
interest on the mailing list, so it will be dropped from the charter;
it can, however, go on as an individual draft so the work done to date
is not lost.  Two new work items have been proposed: multi-to-multi
measurements and a way to store traceroutes.  A personal draft
on multiparty communication parameters and metrics has been created
and announced to the list; no draft has been created for traceroutes.


2. One-Way Active Measurements Protocol
   -- Stanislav Shalunov
   http://www.ietf.org/internet-drafts/draft-ietf-ippm-owdp-11.txt

A summary of all the changes is in the slides.  There was a substantial
wire-format change to robustly differentiate between packets that were
sent and not received and packets that were never sent (if, for example,
a measurement host had internal sending delays with a relatively small
timeout, such that packets would have been sent too late to be
recieved.  The ability to start in the past was removed (as the packets
are just not sent anyway); there is a new section on timing of packets
as sent that clarifies and makes explicit some implications that existed
in the old text.  The security section has been expanded substantially
to clarify requirements and explain design decisions.  Return codes
in the "ACCEPT field" have been expanded; there was some discussion
of what it means to be an internal error versus the catchall failure;
internal errors are completely unexpect - something wrong in the underlying
implementation.  There is now a MUST to remember the list of skipped packets
(which can happen for a number of practical reasons).

Stanislav ended by noting that we still need a security review;
Steve Bellovin asked two people to review; one could not, and the
other has not yet completed his review.  Jeff Dunn from SI international
offered to provide a review, and the chairs will push on the IESG folks
again.



3. Reordering drafts
   -- Al Morton
   http://www.ietf.org/internet-drafts/draft-ietf-ippm-reordering-07.txt

Al Morton reported on the status of the reordering draft.  There were
a number of substantive comments from last call on the mailing list.
See slides for the proposed changes based on the last call comments.
Al will revise the draft, and we will last call the draft again.
There were no comments on the proposed changes from the floor.  Al
asked if he could get an additional person to read and comment on
the draft; Phil Chimento offered to do so.



4. Implementation reports for RFC2678-2681.

Henk once again called for implementation reports.  We've received
three so far.  Keynam Hedayat from Brix offered an implementation
report; Al Morton said he would try to do one as well.  Henk asked
that any reports be received by mid-December; this would allow the
chairs to compile them over the holidays.



5. Multiparty Communications Parameters and Metrics.
   draft-liang-multiparty-para-00.txt

The author has asked us to discuss the draft and see if it should
be picked up as a working group item.  This is a personal draft
that was announced on the mailing list, but there has been no comment
so far.  Roughly, this draft concerns the QoS requirements of multiparty
communication services, and places them in terms of measurement parameters.
It tries to derive a set of parameter metrics from the existing one-way
metrics in IPPM.  Henk asked if anyone in the audience had read the draft,
no one had.  Therefore the chairs did not ask if it should be a working
group item.  Matt noted that this draft is reminicent of Emile Stephan's
spatial metrics draft.  Emile said he would read and comment.




6. Status of the Link Bandwidth Draft.
   --Phil Chimento

Mark Allman pointed Phil in the direction of a workshop held at CAIDA
last December, and he started with that to take a survey of the current
tools and techniques.  (See the slides.)  There were many tools with
three different base techniques, and these techniques (and tools) all
measure different things.  Phil listed a bunch of difficulties
in creating a metric.

Phil asked what we should focus on in the draft.  We could do a
critical analysis of current tools; provide definitions; start
at first principles and create definitions, required metrics, and
statistical analyses.

Matt mentioned that the original intent was to simply attack the definitions.
Jeff Dunn supported a rigorous approach; note requirments, which
techniques were good, which were questionable, and a statistical
analysis.  He felt that there was questionable reporting and inferences
drawn in what he sees; and he would like to see "solid math" and calculus
of probability applied to this area to see what is going on.  He also
noted that MPLS is another thing that should be added to Phil's difficulty
list -- it could add overhead.

Matt Mathis noted that there was a general problem with this class of
metrics, because operational simply measure utilization with SNMP.
He asked if any ISPs are doing this... or would it just be used to
check an ISP's service.  He also noted that SLA generation based on
metrics was not an area the IESG deemed the group should pursue.
Cynthia Martin said the DoD uses this to assure they get the bandwidth
contracted for.

Joeseph Issac said the draft should have definitions and overview of
tools, but detalied statistics is probably too much given the state of the
art. If we want a document describing statistical analysis, it should be
separate.  Someone that worked for MCI said that this was a good proposal;
we should have definitions and not approach the methodology. What matters
is the time measurements are taken, and traffic patterns, these
considerations should be documented.

Roman Krzanowski from Verizon said the definition is an open issue; and
thinks this draft represents something that is needed.  Phil asked which
of the things the different tools measured would be most useful (link
capacity, availble capacity, bottleneck/tight link...); Roman said it
depends on the part of the network you were interested in, but he would
like to have reliable measurements of capacity.  He would especially like
agreed upon metrics in this area.  Henk said that in discussion with ISPS
he finds end-to-end parameters are the most important; ISPs know link
parameters, but they don't know end-to-end parameters. Roman has attended
the IDQ workshop and available BW was of interest there. Summary will be
taken to the list.

Emile Stephan noted that we have discussed active techniques, but passive
techniques are also needed. He said it was important to note what could be
measured and with what accuracy.  If one operator is sending to another
you want to know how the measurement was performed, using what methodology
and which accuracy.  Roman said he concurred; we have methodology and
technology but we do not have agreement.

7. Discussion on an open network performance testing protocol.
   --Roman Krzanowski  Verizon Technologies

Roman works on architecture and SLA/monitoring, and he is faced with
multiple classes/technologies.  Despite all the definitions and dedicated
capapbilities, there are no standards for conducting measurements.  The
cost for maintenance of measurement devices far exceeds the cost of the device.
Scaling is huge at the customer edge - 1000's of points.

What Roman proposes (desires) is a protocol that is implemented in the
deployed hardware for latencey, packet loss and jitter.  This protocol
must be open, so it can be implemented by multiple manufacturers.
He also sees the need the measurement designs to be standardized.
Outline for protocol requirements (see slides): accuracy <1ms,
support the IPPM RFCs, etc.

Al Morton noted the list of metrics focused on round-trip, which is
practical.  Roman said yes, because it is easy.  You can get accurate
time everywhere, but it is not free.  Jeff Dunn made a number of observations
related to timing and what was possible, including that a system must
sample appropriately for the accuracy desired, and these conditions
should go into the archtecture.   Roman said he was interested
in the practicality so that we can get something fundimental and basic
in everywhere.  Jeff also noted you need special treatment of measurement
traffic to get accuracy, but that impacts other traffic.
Matt Mathis thought we were looking for the holy grail of measurement:
you want to measure everything, you want to pay nothing.  He said Roman
should look at the IPMP draft that has been discussed here previously,
and the reprot from the IRTF measurement group on that protocol.
ICMP does work well because it does not get preferred access to resources.
IPMP has diverged into two camps, and input from ISPs might help.
The IRTF review proposed making this simpler.  Just find a way to
record a timestamp, and a cookie that the ISP can understand.
Can infer everything you are looking for except loss rate from this
data.  The person from MCI felt that it was very expensive to get core
line cards to do the time stamps.  It was also important to be sure
you define where the edges of what you measure are.  Roman will follow up.
Keynam from Brix said we shouldn't argue, but be open to all solutions
in measurement, both hardware and software.  He wanted a clarification
if we are looking at a protocol for testing, or a protocol for reporting?
Roman said that if you are doing testing, must agree on some way of
collecting data, but that is secondary to agreeing on testing methodology
and protocol.
Al Morton said that how the providers make measurements is another point
where we need to agree, and we need comparable measurements to do some
inter-domain work. We had an AS draft in IPPM that attempted to narrow
the field.  That might be a good first step toward what you want.

Roman was going to follow up after reading the IPMP report posted to the
IMRG and IPPM list, as well as the OWAMP drafts.

With that the meeting used the alloted time, and the meeting was closed.


------------------------------------------------------------------------------
Henk Uijterwaal                           Email: henk.uijterwaal(at)ripe.net
RIPE Network Coordination Centre          http://www.amsterdamned.org/~henk
P.O.Box 10096          Singel 258         Phone: [phone]
1001 EB Amsterdam      1016 AB Amsterdam  Fax: [phone]
The Netherlands        The Netherlands    Mobile: [phone]
------------------------------------------------------------------------------

Look here junior, don't you be so happy.
And for Heaven's sake, don't you be so sad.                 (Tom Verlaine)

_______________________________________________
ippm mailing list
[e-mail] 
https://www1.ietf.org/mailman/listinfo/ippm