Re: [tcpm] Seeking WG opinions on ACKing ACKs with good cause (was: Possible error in accurate-ecn)

[Bob Briscoe <[e-mail]>]

RIchard, Martin, 2 brief responses inline tagged [BB]...

On 13/03/2021 10:29, Scheffenegger, Richard wrote:
> Hi Martin,
>
> > To me, the real complexity here is marking ACKs in the first place,
> > which forces you to do a bit of accounting about the number of acks
> > you've sent.
>
> I don't quite follow; 

[BB] I think Martin is not disagreeing with your assessment. He was just 
pointing out that the knock-on effects of ACKing pure ACKs originally 
stems from allowing pure ACKs to be ECN capable in the first place. He's 
not saying that's bad. Just it's the root cause of the new phenomenon 
we're seeing here.

> Given that both AccECN endpoints are also using
> ECN++, all control packets (ACKs without any data) can be sent as ECT.
> In order for these packet to use the same Q as data packets, it is
> beneficial to the sender of those, set ECT on them even.
>
> The network is then free to set a CE mark - should the AQM deem that
> necessary - on these control packets.
>
> On the reflector, the CE.packet counter is simply incremented for each
> incoming packet with the CE codepoint - no special treatment is
> necessary for that aspect to work.
>
> Thus the same rules apply to CE-marked data segments, as to any control
> packets received.
>
> > to include "does not increase the ACE counter" as another criterion.
> > Thus TCP treats this as it would a pure window update, which is
> > functionally similar. This should solve any spurious retransmission
> issues.
>
> Unfortnuately, a true duplicate ack can just as well become CE-marked -
> and IMHO this is a more probably scenario, than the misinterpretation of
> a flight of CE-carrying ACKs to trigger a spurious retransmission.

[BB] I was going to say roughly the same as Richard in response to 
Martin here.

Just because some apparent DupACKs with an increased CE count are not 
DupACKs, does not imply that a true DupACK cannot have increase CE count.

That's why we need other tests, like lack of SACK when negotiated, or 
timestamp evidence.


Bob

>
> As Bob's slide shows, the risk only exists under certain pathological
> conditions:
>
> a) the window from A->B must have been at least 3*n*(1/mark probability
> for pure ACKs) in size
> b) right afterward, the data direction has to change, with B becoming
> the sender
> c) B must have no other means (like SACK, Timestamp) to differentiate,
> if the incoming ACKs (carrying new CE infomation, but not advancing
> snd.una) have been triggered by a gap/reordering in the data segements
> delivered to A
>
> So, we effectively talk about a non-SACK, non-Timestamp, but AccECN and
> ECN++ enabled flow, possibly encountering a pathological network
> situation (sudden high CE marking probabilty), swapping the data
> directions at that specific moment also.
>
> What would AccECN loose by NOT sending ACKs carrying new CE-state back:
>
> o) no timely update of the (currently quiescent) endpoint about changed
> network congestion state - likely to influence the cwnd before the next
> transmission and NOT bursting into a congested network (with all the
> dire effects - loss, RTT/RTO delay for loss recovery, ...)
> o) more complex implementation (special case for CE-marks on ACKs
> without data)
> o) Sender can not assume to get full information about the networks
> congestion state unless data is transmitted
> o) Delayed (untimely) CC reaction once data is being transmitted, if the
> ACE counter wrap detection triggers (or complete removal of this
> conservative CC functionality) [the ACK for new data will have the new
> CE.packet counter state, indicating that some time between the prior
> transmit from B, and now, there was network congestion. But no
> infomation when it exactly happend.
>
>
> Besides: As pure ACKs may be lost at any moment, the above effects can
> impact an AccECN sender anyway, even if the receiver is properly
> reflecting, in a timely manner, most up-to-date network congestion
> state. But by removing the (simple) Ack-for-new-CE-marks capability from
> the receiver, we forego any chance to improve the status quo in the 
> future.
>
> If an implementer does want to make the choice of deliberately
> surpressing these ACKs, an AccECN sender has to deal with it anyway.
>
> However, at the "cost" of a typically very minor increase in pure ACKs
> sent 1/(n*1/(ACK-CE-mark probability)), AccECN can allow new CC to
> remain up-to-date while no data transmissions are currently happening.
>
>
> Richard
>
> Am 12.03.2021 um 22:37 schrieb Martin Duke:
>> Hi Bob,
>>
>> Although I don't think the difference between these alternatives is all
>> that large, I believe I would go with (B) -- allow acks of acks at a
>> rate less than 100%. I suppose that in some corner cases it will prevent
>> drops out of a shallow queue, which isn't a big deal, but why not do it?
>> To me, the real complexity here is marking ACKs in the first place,
>> which forces you to do a bit of accounting about the number of acks
>> you've sent.
>>
>> It would be good to also add language to extend the RFC5681 definition
>> of "Duplicate ACK"
>>
>> "
>>
>>   DUPLICATE ACKNOWLEDGMENT: An acknowledgment is considered a
>>        "duplicate" in the following algorithms when (a) the receiver of
>>        the ACK has outstanding data, (b) the incoming acknowledgment
>>        carries no data, (c) the SYN and FIN bits are both off, (d) the
>>        acknowledgment number is equal to the greatest acknowledgment
>>        received on the given connection (TCP.UNA from [RFC793 
>> <https://www.rfc-editor.org/rfc/rfc793>]) and (e)
>>        the advertised window in the incoming acknowledgment equals the
>>        advertised window in the last incoming acknowledgment.
>>
>>        Alternatively, a TCP that utilizes selective acknowledgments
>>        (SACKs) [RFC2018,RFC2883 
>> <https://www.rfc-editor.org/rfc/rfc2883>] can leverage the SACK 
>> information to
>>        determine when an incoming ACK is a "duplicate" (e.g., if the ACK
>>        contains previously unknown SACK information).
>>
>> "
>>
>> to include "does not increase the ACE counter" as another criterion.
>> Thus TCP treats this as it would a pure window update, which is
>> functionally similar. This should solve any spurious retransmission 
>> issues.
>>
>> On Fri, Mar 12, 2021 at 2:54 AM Bob Briscoe <[e-mail]
>> <mailto:[e-mail]>> wrote:
>>
>>     Yoshi, tcpm list,
>>
>>     As promised we set up a small design team on the single issue of
>>     occasionally ACKing pure ACKs if they carry new info (ECN marking at
>>     the IP layer). The design team has two solutions and everyone would
>>     be prepared to accept either, but preferences differ. So we're
>>     seeking wider opinions (more opinions obviously won't narrow the
>>     choices, but at least the WG can then make a decision informed by
>>     those who care).
>>
>>     To understand the question, you can either read the email below. Or
>>     the 3 slides for the tcpm meeting are briefer, and include pictures:
>> https://datatracker.ietf.org/meeting/110/materials/slides-110-tcpm-draft-ietf-tcpm-accurate-ecn-00#page=6
>> <https://datatracker.ietf.org/meeting/110/materials/slides-110-tcpm-draft-ietf-tcpm-accurate-ecn-00#page=6>
>>
>>     Here's the current draft text in question (see here for context
>> https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn-14#section-3.2.2.5
>> <https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn-14#section-3.2.2.5>
>>     ):
>>
>>
>>                 3.2.2.5.1 Data Receiver Safety Procedures
>>
>>         An AccECN Data Receiver:
>>
>>         o  SHOULD immediately send an ACK whenever a data packet 
>> marked CE
>>            arrives after the previous packet was not CE.
>>
>>         o  MUST immediately send an ACK once 'n' CE marks have 
>> arrived since
>>            the previous ACK, where 'n' SHOULD be 2 and MUST be in the 
>> range 2
>>            to 6 inclusive.
>>
>>     Only the second bullet is in question. Here is the proposed diff for
>>     each alternative, then we explain:
>>
>>     Alternative A
>>
>>         o  MUST immediately send an ACK once 'n' CE marks have 
>> arrived since
>>     -     the previous ACK,
>>     +     the previous ACK and there is outstanding data to acknowledge,
>>            where 'n' SHOULD be 2 and MUST be in the range 2 to 6 
>> inclusive.
>>
>>
>>     Alternative B
>>
>>         o  MUST immediately send an ACK once 'n' CE marks have 
>> arrived since
>>     -     the previous ACK, where 'n' SHOULD be 2 and MUST be in the 
>> range 2
>>     +     the previous ACK, where 'n' SHOULD be 3 and MUST be in the 
>> range 3
>>            to 6 inclusive.
>>
>>
>>     Extra guidance text would be required in each case too (see the 
>> end).
>>
>>     Background:
>>     AccECN is a change to the TCP wire protocol that requires the packet
>>     count of congestion feedback to include any congestion experienced
>>     (CE) arriving on Pure ACKs (amongst other things). AccECN doesn't
>>     require Pure ACKs to be ECN-capable, but allows for them to be.
>>     Similarly, AccECN doesn't require any congestion response to CE on
>>     pure ACKs, but having the feedback information there allows a
>>     response to be added with a one-ended update, if desired/necessary.
>>     Basically the data receiver is a 'dumb reflector'.
>>
>>     The above two bullets were designed to ensure that an ACK is
>>     triggered a) on the first sign of congestion, and b) frequently
>>     enough for the count of CE markings to be fed back using the 3-bit
>>     ACE field before it wraps, even if an occasional pure ACK is lost.
>>
>>     We then realized that the wording could require an ACK to be
>>     triggered in response to a CE-marked pure ACK. The circumstance when
>>     this could occur would be when peer X sends a volley of data to Y
>>     then stops, and the path back from Y to X is congested (probably by
>>     other flow(s) so that many of the ACKs are CE-marked. The second
>>     bullet above under alternative (B) would require X to ACK every
>>     'n-th' CE-marked pure ACK. However, if Y immediately started sending
>>     a volley of data to X, Y could misinterpret those ACKs (of ACKs)
>>     from X as DupACKs.
>>
>>     There are two ways to deal with this:
>>     A) Some of us prefer to completely prevent ACKs on pure ACKs, on the
>>     basis that they do not want to risk sometimes generating more 
>> ACKs today
>>     B) Others want to ensure that these rules will cause pure ACKs to be
>>     ACKed when the amount of CE on the ACKs merits it. But sparingly and
>>     strongly damping any ACK ping-pong.
>>
>>     There are complexity arguments on both sides.
>>
>>     B more complex:
>>          extra (non-mandatory) 'if' condition for lack of SACK options
>>     on a pure ACK (to decide it's not a DupACK).
>>     B less complex:
>>          consistent handling of CE marking whether on pure ACKs or data
>>     (which would probably remove an 'if' condition).
>>
>>     A more complex:
>>          CE markings on a string of pure ACKs can build up without
>>     feeding them back, until released by a data packet (if ever).
>>          More code at the other end to deal with the resulting risk of
>>     many wraps of the ACE field (or ignore?).
>>     A less complex:
>>          less different from current TCP.
>>
>>     Extra guidance text would be necessary in either case.
>>
>>     * Alt A) would need text on handling the risk of many ACE wraps
>>          (to be written).
>>
>>     * Alt B) would need something like the following changes:
>>
>>          For the avoidance of doubt, the above change-triggered ACK
>>     mechanism
>>          is deliberately worded to solely apply to data packets, and to
>>     ignore
>>          the arrival of a control packet with no payload, because it is
>>     -  important that TCP does not acknowledge pure ACKs.  The change-
>>     +  important that TCP does not acknowledge pure ACKs which convey no
>>     new
>>     +  state information to the sender. The change-
>>          triggered ACK approach can lead to some additional ACKs but it
>>     feeds
>>          back the timing and the order in which ECN marks are 
>> received with
>>          minimal additional complexity.  If only CE marks are
>>     infrequent, or
>>          there are multiple marks in a row, the additional load will be
>>     low.
>>          Other marking patterns could increase the load significantly.
>>     +
>>     +  Providing feedback on the congestion state of the return channel
>>     +  after a sender has ceased transmitting more data helps inform the
>>     +  clients TCP congestion controller about the state of the return
>>     path.
>>     +  Should the role of data sender and receiver subsequently 
>> change, the
>>     +  new sender has more up to date knowledge of the network state,
>>     +  preventing transmissions of inappropriate size at that moment.
>>
>>          Even though the first bullet is stated as a "SHOULD", it is
>>     important
>>          for a transition to immediately trigger an ACK if at all
>>     possible, so
>>          that the Data Sender can rely on change-triggered ACKs to 
>> detect
>>          queue growth as soon as possible, e.g. at the start of a flow.
>>     This
>>          requirement can only be relaxed if certain offload hardware 
>> needed
>>          for high performance cannot support change-triggered ACKs
>>     (although
>>          high performance protocols such as DCTCP already 
>> successfully use
>>          change-triggered ACKs).  One possible compromise would be 
>> for the
>>          receiver to heuristically detect whether the sender is in
>>     slow-start,
>>          then to implement change-triggered ACKs while the sender is in
>>     slow-
>>          start, and offload otherwise.
>>
>>     +   The second bullet creates a possible case where an AccECN
>>     implementation
>>     +   could sometimes ACK pure ACKs, which in turn might be 
>> mistaken for
>>     +   duplicate ACKs (in scenarios where TCP peers take turns to send
>>     +   sets of data packets). To prevent spurious transmissions in such
>>     +   circumstances, if SACK has been negotiated, an implementation 
>> could
>>     +   optionally assume that an ACK is not a Duplicate ACK if it has
>>     no SACK option,
>>     +   which would indicate it was an ACK of an  ACK. Alternatively it
>>     could use
>>     +   timestamp options to rule out DupACKs.
>>
>>
>>
>>
>>     Bob
>>
>>     --
>> ________________________________________________________________
>>     Bob Briscoehttp://bobbriscoe.net/ <http://bobbriscoe.net/>
>>
>>     _______________________________________________
>>     tcpm mailing list
>>     [e-mail] <mailto:[e-mail]>
>>     https://www.ietf.org/mailman/listinfo/tcpm
>>     <https://www.ietf.org/mailman/listinfo/tcpm>
>>
>>
>> _______________________________________________
>> tcpm mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/tcpm
>>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/