Re: [iccrg] Updates to draft-briscoe-iccrg-prague-congestion-control-03

[Bob Briscoe <[e-mail]>]

Marten,

On 31/10/2023 10:50, Marten Seemann wrote:
> I read the draft and I'm trying to figure out how I'd implement Prague 
> in my QUIC stack. There are a couple of things I've noticed:
>
> 1.
>
>     Section 2.3.2: It’s unclear to me when exactly /alpha/ is updated.
>     I assume that once I receive the first ACK, I save the timestamp.
>     When I receive a new ACK, there are two code paths: if it’s
>     received within one /rtt_virt/, just accumulate the counters used
>     to calculate /frac/. If it’s received after /rtt_virt/, update
>     /alpha /according to the equation given in this section, reset the
>     counters for /frac/ and save the timestamp as the beginning of the
>     next /rtt_virt/ epoch. However, this would mean that the /alpha/
>     value used for multiplicative decrease (section 2.4.2) would
>     always be slightly outdated, which seems suboptimal for an
>     immediate response to a growing queue. Is there a better way?
>
[BB] Indeed. It's actually lot worse than "slightly outdated". As it 
says at the end of the section you refer to:

    However, another approach is being investigated because these
    per-RTT updates introduce 1--2 rounds of delay into the congestion
    response on top of the inherent round of feedback delay (see Section
    3.1.3
    <https://datatracker.ietf.org/doc/html/draft-briscoe-iccrg-prague-congestion-control-03#pracc_faster_response>
    in the section on variants and future work).


Section 3.1.3 goes on to summarize the brief tech report I posted on 
arXiv that explains and defines an alternative approach that removes all 
this lag, cited as [PerAckEWMA]:
Removing the Clock Machinery Lag from DCTCP/Prague 
<https://arxiv.org/abs/2101.07727>

Joakim Misund was evaluating it about a year ago, when he decided to get 
a proper job ;) instead of doing his PhD with me. I am just getting 
round to working on it myself again. But if you wanted to try it 
yourself as wel, we could certainly compare notes.

> 2.
>
>     Section 2.3.3: QUIC uses both packet- and time-threshold loss
>     detection (see sections 6.1.1 and 6.1.2 of RFC 9002). I’m not sure
>     what exactly the recommendation of this draft is.
>
[BB] I hadn't appreciated that QUIC deems there's been a loss if 
/either/ of these conditions is met [RFC9002; §6.1]:

    The packet was sent kPacketThreshold packets before an acknowledged
    packet (Section 6.1.1
    <https://datatracker.ietf.org/doc/html/rfc9002#packet-threshold>),
    or it was sent long enough in the past (Section 6.1.2
    <https://datatracker.ietf.org/doc/html/rfc9002#time-threshold>).


Whereas TCP RACK [RFC8985], only uses DupThresh at the start of a flow 
(when the RTT is likely to be inaccurate) until a decent reordering 
window is established (if reordering is detected at all):

    if some reordering has been observed, then RACK does not trigger
    fast recovery based on DupThresh.

Assuming the QUIC RFC is not just badly written, it doesn't seem right. 
Once a flow has got going, a time threshold is generally considered more 
robust than a packet-threshold. So, even if the packet-threshold is 
adapted in parallel to the time-threshold, using the logical OR of them 
both will often override the time-threshold with the less robust packet 
threshold.

> 2.
>
>     [cont] It would certainly be possible to turn off packet-threshold
>     loss detection, and rely on time-threshold altogether. Is that
>     what QUIC implementations should do?
>

[BB] Well, that's certainly what I thought RACK was meant to do.
This might end up requiring an erratum to RFC9002.

> 3.
>
>     Section 2.4.2: Is the suppression of further decreases after one
>     ECN-triggered decrease for one /srtt/, or is it one /rtt_virt/?
>     Reading section 2.4.4 it sounds like it’s /rtt_virt/, but this
>     could probably be clarified in this section.
>

[BB] Yes. I will check that the Linux code does that though.

We should have caught that mistake in the draft recently when we went 
through the draft checking all the places where it had said 'RTT' before 
we introduced rtt_virt. I've added this to my list of edits to make for 
the next rev.


> 4.
>
>     Section 2.4.3: The QUIC ACK frame acknowledges (multiple) ranges
>     of packets at the same time, together with cumulative ECN counts.
>     It’s therefore not possible to tell which packet was ECN-marked.
>     This means that a QUIC stack will be able to determine
>     /acked_sacked/, but not /ece_delta/. Is it valid to approximate it
>     by assuming that all packets had the same average size? Either
>     way, this is pretty awkward to fit into the pseudo-code given in
>     appendix B.5 of RFC 9002.
>

[BB] Yes. It has to be, given the current QUIC protocol.

Despite having the opportunity to fully integrate ECN into the design of 
QUIC, it seems it was still a bit of an afterthought (I shouldn't 
complain, 'cos I volunteered to help with adding ECN, but then didn't 
get involved 'cos of other pressing work at the time).


> 5.
>
>     Section 2.4.3: Similarly, what's the correct order to process an
>     ACK that reports an ECN marking: For example, an ACK might
>     acknowledge 20 new packets, and report one ECN marking. I think
>     the correct order would be applying the additive increase for 19
>     packets first, and then applying the multiplicative decrease
>     afterwards. This is because receiving a CE-marked packet would
>     elicit an immediate ACK frame from a QUIC receiver (RFC 9000,
>     section 13.2.1). The draft should probably be explicit about this.
>

[BB] Good point.
I agree with your logic, and I've added this to the list of edits too.
However, I think I'll word it as a SHOULD, 'cos it makes sense when CE 
triggers feedback, but the implementer might have better info in some 
scenarios.

> 6.
>
>     Section 2.4.4: I'm struggling to follow how exactly cwnd is
>     supposed to change for small RTTs. Most important from an
>     implementation perspective: section 2.4.3 says that
>     /ai_per_rtt/ will have a different value for small RTTs. It would
>     be helpful if section 2.4.4 would contain an equation for
>     /ai_per_rtt/.
>

[BB] I think the equation you want is already in §2.4.4, altho I agree 
that the text could be clearer, because it states rules you might think 
are right then explains why they're wrong, rather than saying what is 
right first, then explaining why:

    Therefore, the increase in cwnd per packet has to be (1/M^2) * (1/cwnd).

This gives the increase per packet, not per RTT, but that's what is 
needed in an implementation isn't it? Or am I misunderstanding why you 
want the increase per RTT in particular?

Thanks for all these useful comments and questions.



Bob

>
>
> On Sat, 14 Oct 2023 at 19:45, Bob Briscoe 
> <ietf=[e-mail]> wrote:
>
>     iccrg,
>
>     We've just posted an update to prague-congestion-control.
>     Links to diffs are quoted below.
>     The main technical changes:
>
>       * the Apple implementation falls back to CUBIC behaviour on loss
>         (both the reduction and the subsequent increase). Currently
>         the Linux implementation still falls back to Reno on loss, but
>         that is being changed.
>       * how the Apple implementation over QUIC behaves when the path
>         or the remote peer fails to support ECN properly
>       * the items already discussed on this list in response to Neal's
>         review, some of which were editorial, but others were
>         technical, e.g.
>           o pseudocode for removing integer rounding bias
>           o clarifying the RTT-independence approach
>
>     Cheers
>
>
>     Bob & co-authors
>
>
>
>     -------- Forwarded Message --------
>     Subject: 	New Version Notification for
>     draft-briscoe-iccrg-prague-congestion-control-03.txt
>     Date: 	Sat, 14 Oct 2023 05:07:58 -0700
>     From: 	[e-mail]
>     To: 	Bob Briscoe <[e-mail]>
>     <mailto:[e-mail]>, Koen De Schepper
>     <[e-mail]> <mailto:[e-mail]>,
>     Olivier Tilmans <[e-mail]>
>     <mailto:[e-mail]>, Vidhi Goel
>     <[e-mail]> <mailto:[e-mail]>
>
>
>
>     A new version of Internet-Draft
>     draft-briscoe-iccrg-prague-congestion-control-03.txt has been
>     successfully
>     submitted by Bob Briscoe and posted to the
>     IETF repository.
>
>     Name: draft-briscoe-iccrg-prague-congestion-control
>     Revision: 03
>     Title: Prague Congestion Control
>     Date: 2023-10-14
>     Group: Individual Submission
>     Pages: 34
>     URL:
>     https://www.ietf.org/archive/id/draft-briscoe-iccrg-prague-congestion-control-03.txt
>     Status:
>     https://datatracker.ietf.org/doc/draft-briscoe-iccrg-prague-congestion-control/
>     HTML:
>     https://www.ietf.org/archive/id/draft-briscoe-iccrg-prague-congestion-control-03.html
>     HTMLized:
>     https://datatracker.ietf.org/doc/html/draft-briscoe-iccrg-prague-congestion-control
>     Diff:
>     https://author-tools.ietf.org/iddiff?url2=draft-briscoe-iccrg-prague-congestion-control-03
>
>     Abstract:
>
>     This specification defines the Prague congestion control scheme,
>     which is derived from DCTCP and adapted for Internet traffic by
>     implementing the Prague L4S requirements. Over paths with L4S
>     support at the bottleneck, it adapts the DCTCP mechanisms to achieve
>     consistently low latency and full throughput. It is defined
>     independently of any particular transport protocol or operating
>     system, but notes are added that highlight issues specific to certain
>     transports and OSs. It is mainly based on experience with the
>     reference Linux implementation of TCP Prague and the Apple
>     implementation over QUIC, but it includes experience from other
>     implementations where available.
>
>     The implementation does not satisfy all the Prague requirements (yet)
>     and the IETF might decide that certain requirements need to be
>     relaxed as an outcome of the process of trying to satisfy them all.
>     Future plans that have typically only been implemented as proof-of-
>     concept code are outlined in a separate section.
>
>
>
>     The IETF Secretariat
>
>
>     _______________________________________________
>     iccrg mailing list
>     [e-mail]
>     https://www.irtf.org/mailman/listinfo/iccrg
>

-- 
________________________________________________________________
Bob Briscoehttp://bobbriscoe.net/