Re: [ippm] Preliminary measurement comparison of "Working Latency" metrics

["MORTON JR., AL" <[e-mail]>]

Hi again RPM friends and IPPM'ers,

As promised, I repeated the tests shared last week, this time using both the verbose (-v) and sequential (-s) dwn/up test options of networkQuality. I followed Sebastian's calculations as well.

Working Latency & Capacity Summary

Net Qual                           UDPST                        Ookla
DnCap     RPM    DelLD  DelMin     DnCap    RTTmin   RTTrange   DnCap    Ping(no load)
885       916 m  66ms   8ms        970      28       0-20       940      8
888      1355 h  44ms   8ms        966      28       0-23       940      8
891      1109 h  54ms   8ms        968      27       0-19       940      9
887      1141 h  53ms   11ms       966      27       0-18       937      7
884      1151 h  52ms   9ms        968      28       0-20       937      9

With the sequential test option, I noticed that networkQuality achieved nearly the maximum capacity reported almost immediately at the start of a test. However, the reported capacities are low by about 60Mbps, especially when compared to the Ookla TCP measurements.

The loaded delay (DelLD) is similar to the UDPST RTTmin + (the high end of the RTTrange), for example 54ms compared to (27+19=46). Most of the networkQuality RPM measurements were categorized as "High". There doesn't seem to be much buffering in the downstream direction.



> -----Original Message-----
> From: ippm <[e-mail]> On Behalf Of MORTON JR., AL
> Sent: Monday, October 24, 2022 6:36 PM
> To: [e-mail]
> Subject: [ippm] Preliminary measurement comparison of "Working Latency"
> metrics
> 
> 
> Hi RPM friends and IPPM'ers,
> 
> I was wondering what a comparison of some of the "working latency" metrics
> would look like, so I ran some tests using a service on DOCSIS 3.1, with the
> downlink provisioned for 1Gbps.
> 
> I intended to run apple's networkQuality, UDPST (RFC9097), and Ookla Speedtest
> with as similar connectivity as possible (but we know that the traffic will
> diverge to different servers and we can't change that aspect).
> 
> Here's a quick summary of yesterday's results:
> 
> Working Latency & Capacity Summary
> 
> Net Qual                UDPST                        Ookla
> DnCap     RPM           DnCap    RTTmin   RTTVarRnge DnCap    Ping(no load)
> 878       62            970      28       0-19       941      6
> 891       92            970      27       0-20       940      7
> 891       120           966      28       0-22       937      9
> 890       112           970      28       0-21       940      8
> 903       70            970      28       0-16       935      9
> 
> Note: all RPM values were categorized as Low.
> 
> networkQuality downstream capacities are always on the low side compared to
> others. We would expect about 940Mbps for TCP, and that's mostly what Ookla
> achieved. I think that a longer test duration might be needed to achieve the
> actual 1Gbps capacity with networkQuality; intermediate values observed were
> certainly headed in the right direction. (I recently upgraded to Monterey 12.6
> on my MacBook, so should have the latest version.)
> 
> Also, as Sebastian Moeller's message to the list reminded me, I should have
> run the tests with the -v option to help with comparisons. I'll repeat this
> test when I can make time.
> 
> The UDPST measurements of RTTmin (minimum RTT observed during the test) and
> the range of variation above the minimum (RTTVarRnge) add-up to very
> reasonable responsiveness IMO, so I'm not clear why RPM graded this access and
> path as "Low". The UDPST server I'm using is in NJ, and I'm in Chicago
> conducting tests, so the minimum 28ms is typical. UDPST measurements were run
> on an Ubuntu VM in my MacBook.
> 
> The big disappointment was that the Ookla desktop app I updated over the
> weekend did not include the new responsiveness metric! I included the ping
> results anyway, and it was clearly using a server in the nearby area.
> 
> So, I have some more work to do, but I hope this is interesting-enough to
> start some comparison discussions, and bring-out some suggestions.
> 
> happy testing all,
> Al
> 
> 
> 
> 
> _______________________________________________
> ippm mailing list
> [e-mail]
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm__;!!Bhd
> T!hd5MvMQw5eiICQbsfoNaZBUS38yP4YIodBvz1kV5VsX_cGIugVnz5iIkNqi6fRfIQzWef_xKqg4$