Re: [tsvwg] ECN encapsulation draft - proposed resolution

["De Schepper, Koen (Nokia - BE/Antwerp)" <[e-mail]>]

>> So you have not in fact read the post with the full explanation in it, in which the relevant formulae are in fact discussed. 

I saw some equations, but missing the relevant factor here: mss being the packet size. If a flow's rate depends on that packet size, then typically the byte or bit rate r is proportional to mss. Then the other aspect is the probability p measurement. Is that expressed in a per byte probability or per packet probability (which is the same if the sender always send equal packet sizes, but not if there is a certain distribution of packet sizes). So this depends heavily on the exact specific implementation and traffic payload.

>> The steady state of DCTCP is a constant number of marks per RTT.  That's what you get when you cancel the measurement of probability against the number of segments per RTT that the measured probability is applied to.  

DCTCP collects the total ACKed number of "bytes" received in an RTT and the CE-ACKed number of "bytes" received in an RTT. Their ratio is used to update alpha, the reduction factor. So if that ratio changes after decapsulation, you change the rate of DCTCP.
Depending on how this window is used at the sender (if it depends on the detected path MSS or to a reference 1500 bytes MSS), the sender is self-sensitive to path MSS or not. It boils down on if the window controls the MSS packet rate or byte rate. In Linux the window is kept in packets, so it is sensitive to path detected MSS.

>> You really could simplify DCTCP's response to "each CE mark removes half a segment from the cwnd", thereby bypassing a lot of unnecessarily complicated fixed-point maths.  I can walk you through *exactly* how the DCTCP response equations simplify if you need to hear it.

I've tried this before (5 years ago by now?). If you reduce the window by half a packet every mark, it isn't DCTCP anymore. It becomes highly unstable. You need the smoothing. You should try to implement it and run it on a step. Run alone, it becomes Reno, and in a mix of flows, it creates nice patterns 😉.

>> I'm going to stop you there, and remind you that the predominant deployment of ECN AQMs today is a time-domain algorithm, namely Codel.

Then let me remind you too that like we discussed before, it is not the AQM that determines how the CC responds. Any AQM is potentially time-domain, if it applies a deterministic probability (even PI2 can) as it in that case applies a mark for every 1/p-ed packet. The real question is if your codel implementation marks differently based on the packet size of the packet itself, and the intermediate non-marked packets?
So Codel is counting in time between packets marks (but adapts this to the level of congestion independent of what mix of packet sizes there are in between), and then marks a single packet independent of its size. If the time to hit the packet at a specific (even random) time would depend on it's size, then CoDel would mark depending on the size. But I understood that Codel just marks the next packet that arrives after the time-out is passed, so again Codel marks size independent. So bottom line is, that any marker (AQM) can be reduced to a random or deterministic process that marks packets independent or dependent on some properties. It seems you assume Codel does mark packets based on its size (since you claim small packets are marked less than big packets). Even if this would be the case, it seems to be the not recommended case (see RFC7141) and not even solve the purpose you intent (equal throughput, as it needs a squared size proportion probability.

Following RFC7141 the packet marking should be agnostic to the packet size, so encapsulation and decapsulation can also. They should keep the packet marking/drop probability equal.

This results in these 2 methods being a very simple (preferred) and a bit more complex (not even needed) method for decapsulation:
- only mark if the first byte of the inner (reassembled) packet is marked (ignore marks on any other bytes/segments)
- apply a random probability on the inner packet based on the ratio of marked bytes in the inner packet and the inner packet size

Both might "throw away marks" or "create marks" as the count of marked packets will change if the inner to outer size is different, and the packet probability needs to be kept the same statistically.

>> I think it'd be helpful for some *other* WG members to weigh in on this topic for a change.  Does anyone else have an opinion they'd like to share?

Seems all of this has been discussed before and the consensus has been written in RFC7141, which I fully agree with and is still very applicable for L4S going forward. Any bias in the network on packet size would need to work differently for Classic and L4S, so this draft made the right recommendations!

Koen.

-----Original Message-----
From: Jonathan Morton <[e-mail]> 
Sent: Monday, June 14, 2021 6:45 PM
To: De Schepper, Koen (Nokia - BE/Antwerp) <[e-mail]>
Cc: Bob Briscoe <[e-mail]>; [e-mail]; Markku Kojo <[e-mail]>
Subject: Re: [tsvwg] ECN encapsulation draft - proposed resolution

> On 14 Jun, 2021, at 7:04 pm, De Schepper, Koen (Nokia - BE/Antwerp) <[e-mail]> wrote:
> 
>>> Which is what I did in a previous post, using Dimensional Analysis. 

> Dimensional analysis is what you can use to check the validity of equations and further derivations from them. I didn't see any equation in your explanation. 

So you have not in fact read the post with the full explanation in it, in which the relevant formulae are in fact discussed.  Do I have to quote it in full for you to bother to find and read it?

> But I understand your point about the interval being constant for Classic CCs. The Congestion Controls determine if a "packet/byte probability per RTT" or an "RTT intervals between mark/drops" is used:

> - DCTCP-like CCs use the marked byte (or packet) ratio per RTT (or other time interval), but marks are expected during every RTT (or other time interval)

The steady state of DCTCP is a constant number of marks per RTT.  That's what you get when you cancel the measurement of probability against the number of segments per RTT that the measured probability is applied to.  You really could simplify DCTCP's response to "each CE mark removes half a segment from the cwnd", thereby bypassing a lot of unnecessarily complicated fixed-point maths.  I can walk you through *exactly* how the DCTCP response equations simplify if you need to hear it.

At any rate, as the estimated path throughput changes, so does the required interval in packets or bytes between marks, but the required interval in *time* between marks is constant as long as RTT doesn't change (and on a given path, it shouldn't change much).  Hence the dimension that DCTCP is sensitive to is marks/sec, and an infrastructure that preserves that dimension in the congestion signal is desired.

> - Classic CCs trigger reduction on occurrence of any number of bytes/packets dropped/marked per RTT (whether it is a packet of only 1 byte that is marked in that RTT or all packets and bytes marked in that RTT, the result is the same), and need a number of RTTs to recover from the reduction. If the flow is split in small packets and those are marked with the same random prob, they get smaller RTT intervals and a lower rate.

Yes, because when the packets are smaller there are more of them per second and fewer bytes between them.  Keeping a constant packet interval between marks therefore distorts the byte interval and the time interval between marks in the same direction.  Since the byte-seconds per mark dimension is what Reno and CUBIC are sensitive to, this also distorts their response.

> There were recommendations for AQMs to marking packet based on their size, to overcome this unfairness, but this is not often applied I understood.

I'm going to stop you there, and remind you that the predominant deployment of ECN AQMs today is a time-domain algorithm, namely Codel.

There are a number of other AQMs out there that may (or may not) behave as you describe, but are deployed without ECN support and thus can only *drop* packets.  The handling of dropped fragments is unambiguous, since the affected packets cannot be reassembled and must therefore be dropped in whole.  But you are keen to state that dropping and ECN should be different, so let's start by applying marks in the correct way when ECN is supported.

Since DualQ-Coupled and its PI2 algorithm is a new development, you should be in a prime position to make it operate, at least approximately, in either the time-interval or byte-interval domains rather than the packet-interval domain.  As I have already pointed out, this simplifies the requirements on downstream devices considerably.

> Probably better is to solve this again in the end system: by only triggering a congestion event if 1MSS of data is marked. This solves both cases: So it needs to collect at least 1500 bytes of marked packets  which come from either a series of 32 marks in a row (a large 1500b outer packet was marked and all 32 46b inner packets too) or it was NOT encapsulated in a tunnel, and an AQM is used that marks the small packets directly or it was encapsulated and a decapsulation works as a normal AQM and marks the packets with a probability (and in both cases marks 32 times more packets). Assuming the small packets are separately ACKed the 32 factor can be "undone" in the sender by only triggering after 1500 bytes were ACKed CE.

That sounds extremely complicated, compared to just Doing It Right in the first place.

I think it'd be helpful for some *other* WG members to weigh in on this topic for a change.  Does anyone else have an opinion they'd like to share?

 - Jonathan Morton