Re: [ippm] I-D Action: draft-ietf-ippm-capacity-metric-method-00.txt

["MORTON, ALFRED C (AL)" <[e-mail]>]

Hi Rüdiger,

Thanks for your ideas on additional sources of random loss
that we might encounter during Maximum IP-Layer Capacity testing.

We can add some of your considerations directly into the draft.
In general, the recommendation is to use test endpoints as close to the 
measured link as possible (but this is not always possible).

*In the last 2 weeks*, discussions of our Hackathon testing (and follow-up
testing, to be shared at IETF-107) have occurred during the meetings
of two other SDOs seeking Harmonized metrics and methods with IETF:
ETSI Technical Committee STQ MOBILE and ITU-T Study Group 12 (Q17).
Selected points follow:

More on conditions encountered in testing:

Some transmission technologies have multiple methods of operation that 
may be activated when channel conditions degrade or improve, and these 
transmission methods may determine the Maximum IP-Layer Capacity. 
Examples include line-of-sight microwave modulator constellations, 
or cellular modem technologies where the changes may be initiated 
by a user moving from one coverage area to another. Operation in the 
different transmission methods may be observed over time, but the modes 
of Maximum IP-Layer Capacity will not be activated deterministically 
as with the "turbo mode" described as previously described in Section 6.6.

There is also new interest to apply the Maximum IP-Layer Capacity method
during cellular drive testing. This will surely produce forms of 
measurement path variation not yet seen, but the problems seen so far
have all been grappled-with successfully. We will be developing
a test plan with interested parties.

Reporting Formats:

Finally, the authors have also discussed measurement reporting and 
results interpretation at the ETSI and SG12 Q17 meetings. We have listed 
items to provide context to Singleton Capacity measurements:

- timestamp (especially the time when the maximum was observed in dtn)
- source and destination (by IP or other meaningful ID)
- (inner) parameters of the test case (Section 4)
- outer parameters, such as "done in motion" or other factors belonging 
  to the context of the measurement
- result (to take care of cases where the process was somehow interrupted 
  or the attempt failed)
- a field where unusual circumstances could be documented, and another 
  one for "ignore/mask out" purposes in further processing

Thoughts on these and other aspects of reporting formats are welcome.

Thanks for your consideration of these topics, we will update the 
draft again shortly.

Al
(for the co-authors)


> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]]
> Sent: Tuesday, February 4, 2020 3:31 AM
> To: MORTON, ALFRED C (AL) <[e-mail]>
> Cc: [e-mail]
> Subject: AW: [ippm] I-D Action: draft-ietf-ippm-capacity-metric-method-
> 00.txt
> 
> Hi Al,
> 
> Conditions which might occur while measuring are
> 
> - congestion of an interconnection or backbone interface. I don't expect that to
>   be prevalent, but it might happen. If the congested interface has a higher
>   bandwidth then the measurement sender, then the measurement packets
>   won't arrive as a burst at the bottleneck. Further, Random Early Detection
>   may be deployed. In both cases
>   * packet loss is present, which is likely to occur independent of the sender rate.
>   * If the queue or congestion respectively is persistent, a small delay variation is
>      present, which is independent of the sender rate too. It is bigger than the
>      delay variation of a non congested interface.
> 
> The drop pattern caused by a congested backbone or interconnection
> interface is likely not causing a drop pattern which easy to characterize.
> A measurement of the access bandwidth is deteriorating to an estimate, as
> single losses and queuing at an access may be hard to discern from the
> queue and drops caused by backbone or interconnection links. If the access
> bandwidth is rate controlled by a policer, this will even be harder, as in
> that case no queue will build up at the access.
> 
> - the presence of middleboxes might result in transport layer or application specific
>   rate limitations. A variety of conditions may arise, ranging from complete blocking
>   of a measurement flow to access bandwidth measurements being significantly higher
>   than the rates achieved by flows of other protocols or applications, if the latter are
>   rate controlled by a middlebox while the measurement flow isn't.
> 
> It may also be a good idea to foresee ECN support by the measurement
> system. A commodity ECN deployment may not be available by now, but
> standardization efforts of ECN based network and transport behavior have
> grown in the last decade.
> 
> This list isn't exhaustive and others are invited to append or discuss the
> above scenarios. Hints on "how to detect" and "how to separate impact" of
> effects impairing capacity measurements are welcome (especially those easy
> to implement :) .
> 
> Regards,
> 
> Ruediger
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: ippm <[e-mail]> Im Auftrag von MORTON, ALFRED C (AL)
> Gesendet: Donnerstag, 30. Januar 2020 16:14
> An: [e-mail]
> Betreff: Re: [ippm] I-D Action: draft-ietf-ippm-capacity-metric-method-
> 00.txt
> 
> Hi IPPM,
> 
> Thanks to everyone who supported adoption of our draft.
> This version simply changes the filename.
> 
> At the IETF-106 Hackathon, we encountered some new conditions in our
> testing with the prototype implementation:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_IETF-
> 2DHackathon_ietf106-2Dproject-2Dpresentations_blob_master_106-2Dacm-
> 2DHackathon-2DMeasurements.pdf&d=DwIFAw&c=LFYZ-
> o9_HUMeMTSQicvjIg&r=_6cen3Hn-e_hOm0BhY7aIpA58dd19Z9qGQsr8-
> 6zYMI&m=LId4VFVBccULCDxpj9IhTYeQDcZSVtpUSc4r5kWT1nI&s=LwM10qGdCo3Dh9ljl7pS
> 0e2bEeZp056vnNWbSwziG74&e=
> 
> Load Adjustment:
> Search Algorithm uses Loss and delay variation feedback
> 
> Observed:
> Random loss throughout some tests, with insignificant delay variation
> 
> Possible revisions to Search Algorithm, etc. under these conditions:
> 	Add a variable Loss threshold for conditions encountered
> 	Continue "large" rate increases when loss-only is encountered
> 	May need longer test durations under Loss-only Conditions
> 
> We'd like to discuss these findings further with the WG, and run some more
> tests to answer questions and investigate further.
> 
> regards,
> Al, for the co-authors
>