RE: [ippm] Advancement of metrics specifications on the IETF StandardsTrack

["Geib, Ruediger" <[e-mail]>]

Henk, all,

bradner metrictest proposes a method to compare two implementations on a statistical basis and provides criteria to determine, whether their results have "similar" statistical properties. The draft however doesn't define a method to determine which implementation is correct if both measure "different" statistical properties for the same basic event. I'm not sure, whether that is an issue. May be it is fair then to say, that both are not compatible with the RFC (if not, how to decide, which is the failed one)? 
What is to be done, if four implementations are available, say a, b, c and d. Assume a and b as well as c and d to fulfill the bradner metric test in the shown order of pairs, but not a and c/d (b and c/d respectively). Will all be stamped "compatible to RFC..." then?

I think the following conditions should be met:

- repeat a test with a single implementation and prove that this single 
  implementation produces n results for n repeated measurements of the same 
  reference object with similar statistical properties in itself.

- provide a deterministic measurement of the metric.

- a better statistical comparism between two implementations and a 
  comparism with an absolute measurement of the same event.

The deterministic measurement requires:
- the loss statistics of a congested interface
- passive monitors for delay parameters.

The time interval of the events to be measured in comparism with the probing interval plays a role too. You can't measure a 2 second event with probing intervals of 4 seconds. 

I've collected limited practical experience on packet loss measurements (one ippm like implementation compared to loss counters in a commercial backbone).

Regards,

Ruediger



|-----Original Message-----
|From: Henk Uijterwaal [mailto:[e-mail]]
|Sent: Tuesday, July 31, 2007 8:43 AM
|To: IETF IPPM WG
|Subject: [ippm] Advancement of metrics specifications on the IETF
|StandardsTrack
|
|
|IPPM group,
|
|As you all know, the IPPM group has interested in moving its metrics
|(RFCs 2678-2681) to a standard for a long time.   This requires that
|there are 2 or more independent and interoperable 
|implementations of the
|various metrics.  While these implementations exists, there is no
|known process to determine whether they are interoperable.
|
|A possible process was proposed a long time ago in draft
|
|   Advancement of metrics specifications on the IETF Standards Track
|   draft-bradner-metricstest-02.txt
|
|This draft has recently been resubmitted.
|
|The authors (and we)'d like to see some review of the 
|suggested process.
|So, we'd like to ask you to read and comment on this draft.
|
|Matt & Henk
|
|---------------------------------------------------------------
|---------------
|Henk Uijterwaal                           Email: 
|henk.uijterwaal(at)ripe.net
|RIPE Network Coordination Centre          
|http://www.amsterdamned.org/~henk
|P.O.Box 10096          
|Singel 258         Phone: [phone]
|1001 EB Amsterdam      1016 AB Amsterdam  Fax: [phone]
|The Netherlands        The Netherlands    Mobile: [phone]
|---------------------------------------------------------------
|---------------
|
|# Lawyer: "Now sir, I'm sure you are an intelligent and honest man--"
|# Witness: "Thank you. If I weren't under oath, I'd return the 
|compliment."
|
|
|_______________________________________________
|ippm mailing list
|[e-mail] 
|https://www1.ietf.org/mailman/listinfo/ippm
|

_______________________________________________
ippm mailing list
[e-mail] 
https://www1.ietf.org/mailman/listinfo/ippm