[ippm] Preliminary measurement comparison of "Working Latency" metrics

["MORTON JR., AL" <[e-mail]>]

Hi RPM friends and IPPM'ers,

I was wondering what a comparison of some of the "working latency" metrics would look like, so I ran some tests using a service on DOCSIS 3.1, with the downlink provisioned for 1Gbps.

I intended to run apple's networkQuality, UDPST (RFC9097), and Ookla Speedtest with as similar connectivity as possible (but we know that the traffic will diverge to different servers and we can't change that aspect).

Here's a quick summary of yesterday's results:

Working Latency & Capacity Summary

Net Qual                UDPST                        Ookla
DnCap     RPM           DnCap    RTTmin   RTTVarRnge DnCap    Ping(no load)
878       62            970      28       0-19       941      6
891       92            970      27       0-20       940      7
891       120           966      28       0-22       937      9
890       112           970      28       0-21       940      8
903       70            970      28       0-16       935      9

Note: all RPM values were categorized as Low.

networkQuality downstream capacities are always on the low side compared to others. We would expect about 940Mbps for TCP, and that's mostly what Ookla achieved. I think that a longer test duration might be needed to achieve the actual 1Gbps capacity with networkQuality; intermediate values observed were certainly headed in the right direction. (I recently upgraded to Monterey 12.6 on my MacBook, so should have the latest version.)

Also, as Sebastian Moeller's message to the list reminded me, I should have run the tests with the -v option to help with comparisons. I'll repeat this test when I can make time.

The UDPST measurements of RTTmin (minimum RTT observed during the test) and the range of variation above the minimum (RTTVarRnge) add-up to very reasonable responsiveness IMO, so I'm not clear why RPM graded this access and path as "Low". The UDPST server I'm using is in NJ, and I'm in Chicago conducting tests, so the minimum 28ms is typical. UDPST measurements were run on an Ubuntu VM in my MacBook.

The big disappointment was that the Ookla desktop app I updated over the weekend did not include the new responsiveness metric! I included the ping results anyway, and it was clearly using a server in the nearby area.

So, I have some more work to do, but I hope this is interesting-enough to start some comparison discussions, and bring-out some suggestions.

happy testing all,
Al