[iccrg] draft-briscoe-iccrg-prague-congestion-control: CE-marked bytes or packets?

[Neal Cardwell <[e-mail]>]

Re:
  https://datatracker.ietf.org/doc/html/draft-briscoe-iccrg-prague-congestion-control-01

and the passages:

"2.3.2.  Moving Average of ECN Feedback
...it measures the fraction, frac, of ACKed bytes that carried ECN
feedback over the previous round trip. ...

2.4.3.  Additive Increase and ECN Feedback
...a Prague CC applies additive increase irrespective of its CWR
state, but only for bytes that have been ACK'd without ECN feedback.
...  This approach reduces additive increase as the marking
probability increases..."

I was curious about the design choice to specify that the algorithm
reacts to the fraction of *bytes* that have been CE-marked instead of
the fraction of *packets*. IMHO it would be useful for the document to
outline the motivation.

Apologies if I have missed this in previous e-mail discussions or
presentations. I may well have. :-)

I can imagine a number of potential reasons why it could be
advantageous to react to the fraction of packets CE-marked rather than
the fraction of bytes CE-marked:

(1) AFAICT byte counters distort the path's ECN marking probability
more than using packet counters. For example, suppose we have a round
trip with 100 packets sent at roughly uniform intervals across the
round trip time:

o  99 packets of 1 byte each, all CE-marked
o 1 packet of 1000 bytes that was not CE-marked

Then the byte-based Prague "frac" ("the fraction, frac, of ACKed bytes
that carried ECN feedback over the previous round trip") is:

  99 bytes / 1099 bytes ~= .09

Whereas the fraction of ACKed packets that carried ECN feedback is:

   99 packets / 100 packet = .99

So in this toy example there is a >10x difference in the CE "frac"
signal depending on whether bytes or packets are counted.

And given that these packets were spaced uniformly across the round
trip, 99% of the time the bottleneck had excess queuing. This 99%
number is well reflected in a packet-based "frac", but seems to imply
that the byte-based "frac" approach dramatically underestimates the
probability that a packet will encounter excessive queuing, aka the
packet CE marking probability.

The Prague draft in section 1 mentions:

" The Prague CC is a particular instance of a scalable congestion control. ...
For a scalable congestion control B=1, so its response function takes
the form cwnd = K/p. ...
p:  Steady-state probability of drop or marking"

So Prague is defined as a scalable congestion control, which has a
response function that is a function of the probability of ECN
marking. But AFAICT the "frac" mentioned in the Prague spec is a
byte-weighted number, and by contrast the fraction of *packets*
CE-marked is a much better estimate of the probability of a packet
being CE-marked (which is my interpretation of the somewhat ambiguous
"probability of drop or marking").

(2) The current Linux TCP reference implementation of TCP Prague does
not actually use bytes; it uses packets. Likewise, DCTCP and BBRv2 use
packets rather than bytes. So AFAIK the real-world deployment
experience with shallow-threshold ECN thus far is almost entirely with
packet-based algorithms rather than byte-based algorithms. It seems
risky to specify Prague with a byte-based approach that has not been
tested, especially given that the byte-based and packet-based
algorithms can measure massively different signals in some cases (see
(1) above).

(3) AFAIK byte counters are not available when relying on the AccECN
ACE field if there is ACK loss, since the CE marks counted in the ACE
field cannot be properly matched against the size of segments that
were already ACKed and freed. So in environments where only the ACE
field is available then this would imply that TCP Prague cannot be
used (since Prague is specified only in bytes). This would seem to
significantly limit the utility of the ACE field and/or byte-based
Prague, in such scenarios. If Prague were defined in terms of packets
then it seems that perhaps it could be more likely to be useful in
paths that only support the ACE field and strip out the AccECN option?


In summary, if byte counting is considered preferable, IMHO it would
be good to document in this draft why this is so, change the Linux TCP
Prague code to use the byte-based approach, and then for the
definition of "p" in the draft to specify that it means the
probability that a payload "byte" is CE marked rather than leaving the
bytes/packets distinction ambiguous.

best regards,
neal