Re: [ippm] Advancement of metrics specifications on the IETF StandardsTrack

[Lars Eggert <[e-mail]>]

CC'ing the authors of draft-bradner-metricstest; not sure if they are  
subscribed to IPPM.

I'm happy to see discussion on this; as I said in the meeting,  
"statistical equivalence" of results from different metrics  
implementations and methods to determine it are probably the main  
unresolved issue of the document.

Lars

On 2007-8-21, at 9:56, ext Geib, Ruediger wrote:

> Henk, all,
>
> bradner metrictest proposes a method to compare two implementations  
> on a statistical basis and provides criteria to determine, whether  
> their results have "similar" statistical properties. The draft  
> however doesn't define a method to determine which implementation  
> is correct if both measure "different" statistical properties for  
> the same basic event. I'm not sure, whether that is an issue. May  
> be it is fair then to say, that both are not compatible with the  
> RFC (if not, how to decide, which is the failed one)?
> What is to be done, if four implementations are available, say a,  
> b, c and d. Assume a and b as well as c and d to fulfill the  
> bradner metric test in the shown order of pairs, but not a and c/d  
> (b and c/d respectively). Will all be stamped "compatible to  
> RFC..." then?
>
> I think the following conditions should be met:
>
> - repeat a test with a single implementation and prove that this  
> single
>   implementation produces n results for n repeated measurements of  
> the same
>   reference object with similar statistical properties in itself.
>
> - provide a deterministic measurement of the metric.
>
> - a better statistical comparism between two implementations and a
>   comparism with an absolute measurement of the same event.
>
> The deterministic measurement requires:
> - the loss statistics of a congested interface
> - passive monitors for delay parameters.
>
> The time interval of the events to be measured in comparism with  
> the probing interval plays a role too. You can't measure a 2 second  
> event with probing intervals of 4 seconds.
>
> I've collected limited practical experience on packet loss  
> measurements (one ippm like implementation compared to loss  
> counters in a commercial backbone).
>
> Regards,
>
> Ruediger
>
>
>
> |-----Original Message-----
> |From: Henk Uijterwaal [mailto:[e-mail]]
> |Sent: Tuesday, July 31, 2007 8:43 AM
> |To: IETF IPPM WG
> |Subject: [ippm] Advancement of metrics specifications on the IETF
> |StandardsTrack
> |
> |
> |IPPM group,
> |
> |As you all know, the IPPM group has interested in moving its metrics
> |(RFCs 2678-2681) to a standard for a long time.   This requires that
> |there are 2 or more independent and interoperable
> |implementations of the
> |various metrics.  While these implementations exists, there is no
> |known process to determine whether they are interoperable.
> |
> |A possible process was proposed a long time ago in draft
> |
> |   Advancement of metrics specifications on the IETF Standards Track
> |   draft-bradner-metricstest-02.txt
> |
> |This draft has recently been resubmitted.
> |
> |The authors (and we)'d like to see some review of the
> |suggested process.
> |So, we'd like to ask you to read and comment on this draft.
> |
> |Matt & Henk
> |
> |---------------------------------------------------------------
> |---------------
> |Henk Uijterwaal                           Email:
> |henk.uijterwaal(at)ripe.net
> |RIPE Network Coordination Centre
> |http://www.amsterdamned.org/~henk
> |P.O.Box 10096
> |Singel 258         Phone: [phone]
> |1001 EB Amsterdam      1016 AB Amsterdam  Fax: [phone]
> |The Netherlands        The Netherlands    Mobile: [phone]
> |---------------------------------------------------------------
> |---------------
> |
> |# Lawyer: "Now sir, I'm sure you are an intelligent and honest man--"
> |# Witness: "Thank you. If I weren't under oath, I'd return the
> |compliment."
> |
> |
> |_______________________________________________
> |ippm mailing list
> |[e-mail]
> |https://www1.ietf.org/mailman/listinfo/ippm
> |
>
> _______________________________________________
> ippm mailing list
> [e-mail]
> https://www1.ietf.org/mailman/listinfo/ippm

_______________________________________________
ippm mailing list
[e-mail] 
https://www1.ietf.org/mailman/listinfo/ippm