Re: [ippm] Magnus Westerlund's Discuss on draft-ietf-ippm-capacity-metric-method-06: (with DISCUSS)

[[e-mail]]

Hi Magnus,

Before going to discuss details some questions:

Would it help to insert text that person initiating test is required to parametrize an access bandwidth to be validated? 
I'm asking, as the intent of the draft might not be to capture a completely unknown access bandwidth (but I think it doesn't say so explicitely yet).

Do some of your comments address a concern that a test creates to much congestion during one feedback interval FT?
That may stem from the idea, that we've exactly reached access bandwidth AB during FT and badly congest the access during FT+1 (I don't deny that this is possible).

If that's your concern, would an approach to limit the maximum congestion created before FT+1 is reached being bound to a limit Buffer_max[ms]@AB[Bit/s] based on the last captured congestion free bandwidth AB during FT?
As an example, Buffer_max = 0,05 s if AB < 1 Gbit/s (smaller Buffer_max values may be applicable at higher rates). This would result in a formula determining the sending behaviour during st, rather than a table. 

Regards,

Rüdiger



-----Ursprüngliche Nachricht-----
Von: Magnus Westerlund <[e-mail]> 
Gesendet: Donnerstag, 11. März 2021 12:16
An: [e-mail]; Geib, Rüdiger <[e-mail]>
Cc: [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]
Betreff: RE: Magnus Westerlund's Discuss on draft-ietf-ippm-capacity-metric-method-06: (with DISCUSS)

Hi,

I have reviewed the text changes in context. 

First of all I like to be clear that these changes do need review from the rest of the WG and a confirming consensus call as they are substantial. 

Section 2.

I think the scope text is still fairly open. Yes it is clear that the load algorithm is only intended for measurements. However, the usage is not particular limited, especially in regards to my main concern of edge to central nodes across multiple AS in the Internet like the different TCP speed tests often are deployed. The security consideration requirements are good for a number of reasons but put no limitations on that aspect. So I would prefer a more explicit statement here. I think an important aspect here is that any ISP seeing issues from these measurements should know who to talk to. I don't know how to best formulate that. 


Section 8.1:

So I am trying to understand the implication of the load algorithm at higher rates and how recommendations works out in relation to the definition of the rate. 

So the document says: 

    Each rate is defined as
   datagrams of size ss, sent as a burst of count cc, each time interval
   tt (default for tt is 1ms, a likely system tick-interval)

So I think this definition is fine for lower rate as the number of packets in each 1 ms burst is fairly small and the buffer it hits will likely be relatively large compared to the increase in load. However at higher rates like beyond 10 GBPS where 1 GBPS steps are recommended. So transmitting at bursts every 1 ms intervals means that one are transmitting 833 packets each burst at 10 GBP rate of 1500 bytes size, so likely even higher for more moderate 1200 byte size packets. That is almost 1,3 mb of data. So where pacing may be quite good at lower bit-rates < 1Gbps I wonder if it starts breaking down at higher rates, which appears to in the region where buffers becomes more shallow due to the cost of having large buffers and where good pacing reduces the need for buffering. I would also note that the reaction time for the control can be 1RTT + 50 ms which thus the increase in offered load for a step size becomes 10s of MB during an regulation period.

As the load algorithm hasn't been tested beyond 10Gbps and it appears that the numbers can start to become more problematic at these speeds, wouldn't it be better to say that this is not intended beyond 10 Gbps. 

On the table I have the following comments:

   +--------------+-------------+--------------+-----------------------+
   | Parameter    | Default     | Tested Range | Expected Safe Range   |
   |              |             | or values    | (not entirely tested, |
   |              |             |              | other values NOT      |
   |              |             |              | RECOMMENDED)          |
   +--------------+-------------+--------------+-----------------------+
   | FT, feedback | 50ms        | 20ms, 100ms  | 5ms <= FT <= 250ms    |
   | time         |             |              | Larger values may     |
   | interval     |             |              | slow the rate         |
   |              |             |              | increase and fail to  |
   |              |             |              | find the max          |
   +--------------+-------------+--------------+-----------------------+

I would note that a FT of 5 ms will have the potential to result in significant fluxtuations in some systems like mobile systems as the scheduler time is actually likely to be longer than 5 ms.

   +--------------+-------------+--------------+-----------------------+
   | Feedback     | L*FT, L=10  | L=100 with   | 0.5sec <= L*FT <=     |
   | message      | (500ms)     | FT=50ms      | 30sec Upper limit for |
   | timeout      |             | (5sec)       | very unreliable test  |
   | (stop test)  |             |              | paths only            |
   +--------------+-------------+--------------+-----------------------+

Even the default means that one looses 10 feedback packets in a row. That is a lot and shows that one have a serious interruption on the return path. Already loosing 3 feedback packets in a row indicates that one have significant outages if this is lost. 

Secondly, this is formulated only based on  intervals of FT. For startup the RTT is relevant factor. So I think there are several factors here for the timeout that maybe need to be teased apart? So initially one offers a very low load and one may not have a good measurement on base RTT. Thus, time to first feedback is okay to be fairly large and 500 ms is likely okay but quite longer than expected for an access to local internet exchange measurement. However, when one scale up the rate I think these values are way to long as the total amount of traffic sent without feedback becomes quite significant. Receiving no feedback for more than 10 reporting intervals are already way to long. And to state that 30 seconds would be an acceptable value I can't support even for a measurement tool.

The definition of what "Feedback message timeout" and "Load packet Timeout" is not defined. I assume that Feedback message timeout is the time without receiving any feedback messages after starting a measurement. Is the load packet timeout the time the receiver is waiting before using signalling channel to end the measurement without receiving any packets, or for the sender to receive feedback that says that no packets have been received? The roles here are not clear.

Sending packets for several seconds without seeing any result appears problematic and allowing values beyond several seconds looks broken. 

   +--------------+-------------+--------------+-----------------------+
   | table index  | 0.5Mbps     | 0.5Mbps      | when testing <=10Gbps |
   | 0            |             |              |                       |
   +--------------+-------------+--------------+-----------------------+
   | table index  | 1Mbps       | 1Mbps        | when testing <=10Gbps |
   | 1            |             |              |                       |
   +--------------+-------------+--------------+-----------------------+

Why is this value not relevant when testing beyond 10 Gbps, the ramp up time becomes to long with these values or?


   | ss, UDP      | none        | <=1222       | Recommend max at      |
   | payload      |             |              | largest value that    |
   | size, bytes  |             |              | avoids fragmentation  |
   +--------------+-------------+--------------+-----------------------+

So isn't there a mismatch between the metric and the load algorithm values here? With the rate definition in Section 8.1 being defined as based on "ss" that UDP payload bytes, rather than IP packet sizes that are used? 

I understand that one want to ensure that one measure using a size that actually works in the path. However, I think one should be warned that one might run into packet rate limitations rather than byte limits if one would use too small. 
`
   +--------------+-------------+--------------+-----------------------+
   | cc, burst    | none        | 1 - 100      | same as tested        |
   | count        |             |              |                       |
   +--------------+-------------+--------------+-----------------------+

So the cc value is dependent on target rate and the value of ss and tt. So should it be included in this table? Especially as 100 is not sufficient for multi-gigabit speeds with a tt of 1 ms. 

   +--------------+-------------+--------------+-----------------------+
   | low delay    | 30ms        | 5ms, 30ms    | same as tested        |
   | range        |             |              |                       |
   | threshold    |             |              |                       |
   +--------------+-------------+--------------+-----------------------+

So I think this value is highly dependent on several aspects and maybe should get more discussion. First for a measurement campaign it is relevant what one consider as the target additional latency that is acceptable when finding capacity. Secondly, the jitter in the network technology. For WIFI,  mobile and DOCIS a to low value may be shorter than the scheduling latencies that might occur. It is also a question about how precise the implementation are capable of measuring per packet latency variances. 

   +--------------+-------------+--------------+-----------------------+
   | high delay   | 90ms        | 10ms, 90ms   | same as tested        |
   | range        |             |              |                       |
   | threshold    |             |              |                       |
   +--------------+-------------+--------------+-----------------------+

Also here I wished there was a bit more discussion. So this value clearly must be above expected jitter for the network technology. It also needs to be sufficient large to represent a fair amount of queue to avoid measurement errors. I assume that if one would chose a value larger than available buffer depth one would drive the network into packet loss. And as long as there are some room between low delay range threshold and the actual delay causing loss or this higher one has a chance to regulate to that rate. 

   +--------------+-------------+--------------+-----------------------+
   | sequence     | 0           | 0, 100       | same as tested        |
   | error        |             |              |                       |
   | threshold    |             |              |                       |
   +--------------+-------------+--------------+-----------------------+

What is this value really? 

   +--------------+-------------+--------------+-----------------------+
   | consecutive  | 2           | 2            | Use values >1 to      |
   | errored      |             |              | avoid misinterpreting |
   | status       |             |              | transient loss        |
   | report       |             |              |                       |
   | threshold    |             |              |                       |
   +--------------+-------------+--------------+-----------------------+

Also here I am uncertain what is the criteria here?

   +--------------+-------------+--------------+-----------------------+
   | Fast mode    | 30          | 3 * Fast     | same as tested        |
   | decrease, in |             | mode         |                       |
   | table index  |             | increase     |                       |
   | steps        |             |              |                       |
   +--------------+-------------+--------------+-----------------------+

So is the recommended value 30 or 3*Fast mode increase? Should they be proportional or not?

The last entry appears to be a summary fact of the parameterization, and is it relevant?


What is the goal here in relation to push other congestion controlled traffic out of the way? It appears that it is likely to cause delay based congestion to be pushed out of the way. I am more uncertain how it interacts with loss based ones, as depending on situation it appears that it could avoid going into the loss regim. 

My conclusion is that some aspect of this do appear more clarifications on what they are and further assumptions on how the load algorithm will be deployed spelled out so that its function is more controlled. 

Cheers

Magnus





> -----Original Message-----
> From: MORTON, ALFRED C (AL) <[e-mail]>
> Sent: den 8 mars 2021 18:33
> To: [e-mail]; Magnus Westerlund
> <[e-mail]>
> Cc: [e-mail]; [e-mail]; draft-ietf-ippm-capacity-
> [e-mail]; [e-mail]; [e-mail];
> [e-mail]
> Subject: RE: Magnus Westerlund's Discuss on draft-ietf-ippm-capacity-
> metric-method-06: (with DISCUSS)
> 
> Hi all,
> 
> The working text was just submitted as version 07. [0] We can continue
> discussions from this point.
> 
> Thanks again for all the IESG reviews!
> 
> Al (for the co-authors)
> 
> [0] https://datatracker.ietf.org/doc/html/draft-ietf-ippm-capacity-metric-
> method
> 
> 
> > -----Original Message-----
> > From: [e-mail] [mailto:[e-mail]]
> > Sent: Monday, March 8, 2021 3:19 AM
> > To: MORTON, ALFRED C (AL) <[e-mail]>;
> > [e-mail]
> > Cc: [e-mail]; [e-mail]; draft-ietf-ippm-capacity-
> > [e-mail]; [e-mail]; [e-mail];
> > [e-mail]
> > Subject: AW: Magnus Westerlund's Discuss on draft-ietf-ippm-capacity-
> > metric-method-06: (with DISCUSS)
> >
> > Hi Al, hi Magnus,
> >
> > To make progress within the time left, it's likely best to leave
> > things as they are. That produces a standard and a benchmark. A cross
> > domain deployment may be tested against that benchmark to gain more
> > experience on operational aspects.
> >
> > Regards,
> >
> > Ruediger
> >
> > -----Ursprüngliche Nachricht-----
> > Von: MORTON, ALFRED C (AL) <[e-mail]>
> > Gesendet: Freitag, 5. März 2021 14:22
> > An: Magnus Westerlund <[e-mail]>; Geib,
> Rüdiger
> > <[e-mail]>
> > Cc: [e-mail]; [e-mail]; draft-ietf-ippm-capacity-
> > [e-mail]; [e-mail]; [e-mail];
> > [e-mail]
> > Betreff: RE: Magnus Westerlund's Discuss on draft-ietf-ippm-capacity-
> > metric-method-06: (with DISCUSS)
> >
> > Hi Magnus and Rüdiger,
> >
> > The working text on the "applicability" limits that we added to the
> > scope section *included an earlier agreement* to add the first
> > sentence below and the bullet, so now the whole paragraph reads:
> >
> >    The primary application of the metric and method of measurement
> >    described here is the same as in Section 2 of [RFC7479] where:
> >
> >    o  The access portion of the network is the focus of this problem
> >       statement.  The user typically subscribes to a service with
> >       bidirectional access partly described by rates in bits per second.
> >
> >    In addition, the use of the load adjustment algorithm described in
> >    section 8.1 has the following additional applicability limitations:
> >
> >    - MUST only be used in the application of diagnostic and operations
> >    measurements as described in this memo
> >
> >    - MUST only be used in circumstances consistent with Section 10,
> >    Security Considerations
> >
> > We can make further edits to balance your comments but let's start
> > here, and sorry for not including the whole paragraph last night - it
> > seems that I should have.
> >
> > Al
> >
> > > -----Original Message-----
> > > From: Magnus Westerlund [mailto:[e-mail]]
> > > Sent: Friday, March 5, 2021 8:06 AM
> > > To: [e-mail]
> > > Cc: [e-mail]; [e-mail]; draft-ietf-ippm-capacity-
> > > [e-mail]; [e-mail]; [e-mail];
> > > [e-mail]; MORTON, ALFRED C (AL) <[e-mail]>
> > > Subject: RE: Magnus Westerlund's Discuss on
> > > draft-ietf-ippm-capacity-
> > > metric-method-06: (with DISCUSS)
> > >
> > > Hi,
> > >
> > > Ruediger, if you can find a formulation that covers that national
> > > test case I am likely fine with it. If the involved parties know who
> > > to discuss issues with and can get them addressed I am not worried.
> > > I am worried where someone deploys a couple of servers, like the
> > > current TCP speed tests and users run it totally by themselves.
> > >
> > > And to be clear I think with more experience with large scale
> > > deployment of the algorithm and more experiments beyond the
> intended
> > > deployment model this should be possible to update the specs to
> > > remove this type of limitation.
> > >
> > > Cheers
> > >
> > > Magnus
> > >
> > > > -----Original Message-----
> > > > From: [e-mail] <[e-mail]>
> > > > Sent: den 5 mars 2021 12:13
> > > > To: Magnus Westerlund <[e-mail]>
> > > > Cc: [e-mail]; [e-mail];
> > > > draft-ietf-ippm-capacity- [e-mail];
> > > > [e-mail]; [e-mail]; [e-mail];
> > > > [e-mail]
> > > > Subject: AW: Magnus Westerlund's Discuss on
> > > > draft-ietf-ippm-capacity-
> > > > metric-method-06: (with DISCUSS)
> > > >
> > > > <snip>
> > > >
> > > > I think this is a bit to unclear in regards to the limitation of
> > scope.
> > > My
> > > > worry are internet wide measurements. Reading what exists in the
> > > currently
> > > > available draft in the Section 10, there are no limitation here
> > > described to
> > > > only use this algorithm only across controlled networks.
> > > > Especially as bullet
> > > > 5 in Section 10 relies on the load algorithm what is currently
> > > > written
> > > is
> > > > fine
> > > > across the whole internet as long as sender and receiver are okay
> > > > with
> > > the
> > > > measurement which is not what I at least thought we agreed.
> > > >
> > > > Can you please be explicit that the load algorithm is limited to
> > > > use
> > > across
> > > > networks path that are controlled or managed and not intended for
> > > Internet
> > > > wide usage.
> > > >
> > > >  - MUST only be used as part of measurements within managed
> > > > networks, and not across general Internet.
> > > >
> > > > [RG] That's tough, as regulators are interested in this test and
> > > regulators
> > > > aren't part of a domain. So they might resort to TCP speed tests,
> > > > being
> > > less
> > > > accurate and precise and not standardised (note that penalties are
> > > discussed
> > > > to be linked to results). I'd be interested in finding agreement
> > > > to have metric and method standardized for at least nations under
> > > > authority of a single regulator. No idea how exactly. Would a
> > > > bound on RTT/in some countries multiple instances and some
> > > > additional parametrization information prior
> > > to
> > > > start, e.g., a contracted access bandwidth, help?
> > > >
> > > > [RG] I'm off for the weekend (I'm out for a lasting solution, not
> > > > necessarily a speedy one) - regards,
> > > >
> > > > Ruediger