Re: [tcpm] I-D Action: draft-ietf-tcpm-accurate-ecn-16.txt

[Bob Briscoe <[e-mail]>]

Richard,

Indeed, I recently discovered I had missed an item that I added to my 
ToDo list in Mar 2021, based on a conversation between Gorry and Mirja 
where they both made the same point - that fall-back behaviour after 
discovering ECN mangling ought to be non-normative in this draft, 
because it's beyond the scope of the new feedback mechanism:
https://mailarchive.ietf.org/arch/msg/tcpm/00URm1uwptgrAzKq96T6KSh8dQ4/

We do need to say the following normatively in "Testing for Mangling of 
the IP/ECN Field" because they are relevant to feedback, the Data Sender:
* remains in AccECN mode
* MUST continue to feed back any ECN markings on arriving packets.

BTW, Vidhi, the text you quoted was from §2 on "Protocol Overview and 
Rationale". That was meant to be an example of why it is useful for the 
handshake to give feedback on exactly which IP-ECN codepoint arrived. It 
wasn't meant to be a full spec. So I'll make it clearer that it's only 
an example, rather than try to make it comprehensive, which isn't 
appropriate at that point in the doc.

Nonetheless, your points are correct, but if they are dealt with anyway, 
they should be dealt with in §3.2.2.3. "Testing for Mangling of the 
IP/ECN Field" where these scenarios are already covered (except for the 
congestion response, which was silently left out of scope):
https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#section-3.2.2.3 


We could just say that the questions of whether to set ECT for 
subsequent packets, and whether to respond to feedback of CE markings 
are beyond the scope of the draft. I think we should, but I think we 
should also give some non-normative advice on what to do. It can only be 
interim advice, not normative requirements. Both because it's out of 
scope, and because the best behaviour and the alternatives will depend 
on deployment experience. So they shouldn't be written normatively in a 
Proposed Standard yet.

We will do what the WG wants, but assuming it agrees to including 
non-normative advice, following from your (Richard's) list of cases, 
here's a full matrix. I don't propose this matrix should go in the 
AccECN draft - it's just for this email, to check every case is covered.


	Feedback in SYN/ACK of SYN's IP-ECN
IP-ECN on SYN 	Not-ECT 	ECT(0) 	ECT(1) 	CE
Not ECT 	√ 	NR 	NR 	NR
ECT(0) 	ND 	√ 	√ 	√
ECT(1) 	ND 	CU
	√ 	√
CE 	n/a 	n/a 	n/a 	n/a


Key:
√       Valid response
NR    Non-normative advice: Set *N*ot-ECT for subsequent packets, but 
continue to *R*espond to CE feedback
ND    Non-normative advice: Set *N*ot-ECT for subsequent packets, and 
*D*isable response to CE feedback
CU    Currently unused transition (not listed as invalid in the draft)
n/a    not applicable in the AccECN spec, in which there is never a case 
where the sender sets CE when sending a packet.

In the draft we don't need to mention any cases where CE is sent (but a 
draft on fall-back probably would, because the sender might set CE to 
test the path). We can cover the other cases with two groups of invalid 
transitions: those where the SYN (or SYN-ACK) arriving at the receiver 
appears to have been
i) ECN-capable
ii) Not-ECN-capable.

I propose to also say that details (like what 'subsequent packets' 
implies about how long this behaviour continues) are for future study.

I'll try to write some text and propose it to this list. Perhaps it 
should be in a new appendix?


Bob

On 05/02/2022 12:01, Scheffenegger, Richard wrote:
> Vidhi,
>
> Thanks for reading the draft very closely!
>
> Well, in the non-ECT SYN case, where the SYNACK indicates an ECT (ect0,
> ect1, ce) is the obvious - and fully semantically correct - use case of
> that possible use.
>
> For the set of three codepoints (ECT0, ECT1, CE) the only other possible
> valid detection scenario would be if a SYN+CE is sent (deliberately,
> althought one can argue that a sender should not originate CE-marked
> packets), but a SYNACK without CE is returned (full or partial
> bleaching). At least under the current RFC3168 definition of IP ECN
> codepoints.
>
> allowed transitions:
> nonECT -> nonECT
>
> ECT0 -> ECT0
> ECT0 -> ECT1
> ECT0 -> CE
>
> ECT1 -> ECT1
> ECT1 -> ECT0 (possibly)
> ECT1 -> CE
>
> CE -> CE
>
>
> Thinking about this aspect more, I am now unsure, if the higher level
> functions, such as detection of IP ECN codepoint mangling, or validating
> across multiple RTTs, should be in this document - or not rather be in a
> separate draft, in order to facilitate independent implementation and
> improvement of mechanisms in that space.
>
> However, the above table of allowed IP ECN transistions may be updated
> independently of AccECN - and performing extensive checks as part of a
> signal protocol ossifies this IMHO.
>
> As a middle ground, perhaps having these descriptions in a non-normative
> section, clearly stating these to be examples of what an impemented
> could choose to do?
>
> Richard
>
> Am 05.02.2022 um 02:14 schrieb Vidhi Goel:
>> I like your response for all the points. I can review the diff for the
>> proposed changes, if you’d like before committing them to the draft.
>>
>> Sorry, I want to add a new comment for this text,
>>
>> If a TCP client has set the SYN to Not-ECT, but
>>     receives feedback that the IP-ECN field on the SYN arrived with a
>>     different codepoint, it can detect such middlebox interference and
>>     send Not-ECT for the rest of the connection.
>>
>>
>> This statement holds good for Not-ECT but doesn’t if lets say the SYN
>> was ECT0 or ECT1 as those could be marked CE and still be valid. Should
>> we add a statement for ECT marked SYN as well?
>>
>> Thanks,
>> Vidhi
>>
>>> On Feb 4, 2022, at 5:44 AM, Bob Briscoe <[e-mail]
>>> <mailto:[e-mail]>> wrote:
>>>
>>> Vidhi,
>>>
>>> I've increased the table of contents to tocdepth = "4" which might
>>> help find the relevant sections better, because a lot of the meat in
>>> this draft is under sections 3.2.2 (ACE) and 3.2.3 (Option).
>>>
>>> I've responded below, but I'd prefer it if you would say these things
>>> on the list, so we have a better justification for changing the draft.
>>> Reviewing a draft that has just be re-posted shouldn't imply anything
>>> about whether Apple is implementing the draft or not.
>>>
>>> (BTW, regarding the proposed change to the initial value of r.e1b, I
>>> was going to send that  to the list once co-authors agreed to it. But
>>> it would be nice if some of the edits were not self-generated.)
>>>
>>> See [BB] inline for responses (which I'll repeat if you send to the 
>>> list).
>>>
>>> On 04/02/2022 07:37, Vidhi Goel wrote:
>>>> Hello Bob and authors,
>>>>
>>>> I have read the draft a few times and will send my extensive review
>>>> at a later point. But right now, I want to ask some critical things
>>>> that an implementation could benefit from.
>>>>
>>>> The draft says,
>>>> /If a TCP client has set the SYN to Not-ECT, but/
>>>> /   receives feedback that the IP-ECN field on the SYN arrived with a/
>>>> /   different codepoint, it can detect such middlebox interference 
>>>> and/
>>>> /   send Not-ECT for the rest of the connection/
>>>>
>>>> 1. On the forward path from client to server, the client will revert
>>>> to Not-ECT when it sees for example, that it sent a Non-ECT SYN but
>>>> received ACE encoding other than 0 1 0. What does the client do in
>>>> the last ACK of the 3WHS -
>>>>  a. does it stay in AccECN mode and still send AccECN encoding based
>>>> on the IP code point of SYN-ACK (Table 4)? This would mean that
>>>> client won’t participate in ECN on the sender half of the connection
>>>> and only provide AccECN feedback to the server as a receiver.
>>>>  b.  does it disable AccECN mode and set ACE= 0 0 0 so that a server
>>>> in AccECN mode can disable ECN based on Table4?
>>>>
>>>> While writing this, I realized that the intention is probably a.
>>>> Could you confirm? Also, when the sender sets Not-ECT in its data
>>>> packets, it should also disable acting upon any ACE feedback as we
>>>> could still receive false ACE feedback from the server if the
>>>> network, lets say, changed 00 (not ECT) to 01(ECT1). If you agree, we
>>>> should add some text around this. Based on the current text, the
>>>> sender will always respond to the ACE feedback even if it sends 
>>>> Not-ECT.
>>>>
>>>> TBH, this is a complicated scenario, where sender said to the network
>>>> - I don’t trust you so I can’t use ECN. Feel free to drop my packets.
>>>> And the network mangles the IP to ECT1 and then set CE (when
>>>> congested) which would be feedback’ed from the receiver. Now, the
>>>> packet wasn’t dropped which it should have been. So, is it better to
>>>> just ignore this feedback because sender doesn’t trust the network or
>>>> just act on it and reduce cwnd in order to reduce congestion in the
>>>> network somewhere.
>>>
>>> [BB] Your points are all good ones.
>>> I'll address this last complex scenario first, because it has
>>> implications for the earlier questions. First I'll define some
>>> terminology:
>>>
>>> Simple mangling scenario:
>>>
>>>   * some network function, e.g. broken Diffserv ToS-byte remarking,
>>>     illegally remarks a Not-ECT  SYN to CE.
>>>
>>> Your complex mangling scenario, repeated here:
>>>
>>>   * some network function, e.g. broken Diffserv ToS-byte remarking,
>>>     illegally remarks a Not-ECT SYN to ECT0 or ECT1,
>>>   * then congestion at a subsequent bottleneck is marking some 
>>> packets CE
>>>
>>> Then potentially the client could tailor its behaviour after sending a
>>> Not-ECT SYN, If the SYN/ACK feedback is
>>>
>>>  1. ECT: disable sending ECT, but continue responding to ECN feedback
>>>  2. CE, disable sending ECT, and disable response to ECN feedback
>>>
>>> However, this is uncertain, because CE feedback on the SYN/ACK could
>>> indicate either the simple or the complex mangling scenario.
>>>
>>> A simpler alternative would be to always continue responding to ECN
>>> feedback. Rationale:
>>>
>>>   * Whether case #1 or #2, assume that mangling of the SYN might have
>>>     been Not-ECT to ECT, even if the feedback off the SYN is CE.
>>>   * Then as the connection progresses, if /all/ feedback is CE,
>>>     there's already a recommendation to fall-back to disabling
>>>     congestion response.
>>>
>>> If we do this, I think we ought to say "SHOULD continue to respond to
>>> ECN feedback", not "MUST".
>>>
>>> And we'll need to put this all to the WG.
>>>
>>> 1.a) Now back to the beginning of your point. The text you quote is
>>> from §2.5 which is in the non-normative "Overview and Rationale"
>>> section (§2).
>>> You really need the normative text from:
>>>
>>>
>>>           3.2.2.3.
>>> <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#section-3.2.2.3>Testing
>>>           for Mangling of the IP/ECN Field
>>> <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#name-testing-for-mangling-of-the>
>>>
>>>
>>> where it says (end of 1st para):
>>>
>>>     " ...for the remainder of the connection the client MUST NOT send
>>>     ECN-capable packets, but it MUST continue to feed back any ECN
>>>     markings on arriving packets."
>>>
>>> So, you're right, the answer is (a).
>>> It can't be (b), because, once in AccECN mode, 000 on the ACE field
>>> just becomes a counter value, and no longer negotiates the feedback 
>>> mode.
>>>
>>> 1.b) As per the above, let's conservatively assume complex mangling.
>>> So, at the end of the first para quoted above I suggest we add:
>>>
>>>     " ...for the remainder of the connection the client MUST NOT send
>>>     ECN-capable packets *but**it**SHOULD continue to respond to ECN
>>>     feedback even though **it **is no longer sending ECN-capable
>>>     packets (see reasoning below)**. T**he client**MUST remain in the
>>>     same feedback mode and***it MUST continue to feed back any ECN
>>>     markings on arriving packets."
>>>
>>> I'll do the same for next para about the server.
>>> I'll work out some text for the reasoning, but I won't give it here.
>>>
>>> There are two other cases where it disables sending ECT, which don't
>>> say whether it continues to respond to congestion:
>>>
>>>   * "Testing for Mangling" section, penultimate para,  where it's
>>>     receiving solid CE:
>>>     "Once a Data Sender has entered AccECN mode it SHOULD check
>>>     whether all feedback received for the first three or four rounds
>>>     indicated that every packet it sent was CE-marked. If so, for the
>>>     remainder of the connection, the Data Sender SHOULD NOT send
>>>     ECN-capable packets*and it SHOULD NOT respond to ECN feedback*,
>>>     but *it MUST remain in the same feedback mode and *it MUST
>>>     continue to feed back any ECN markings on arriving packets*(in its
>>>     role as Data Receiver)*."
>>>   * Next section "Zeroing of the ACE Field", 2nd para:
>>>     "If the value of this ACE field is zero (0b000), the Data Sender
>>>     disables sending ECN-capable packets for the remainder of the
>>>     half-connection by setting the IP/ECN field in all subsequent
>>>     packets to Not-ECT, *but**it**SHOULD continue to respond to ECN
>>>     feedback even though **it **is no longer sending ECN-capable
>>>     packets.* *It MUST also remain in the same feedback mode and it
>>>     MUST continue to feed back any ECN markings on arriving packets
>>>     (in its role as Data Receiver).*"
>>>
>>> I'll also add reasoning (in the zeroing section, it already says that
>>> ACE=0b000 is not an unambiguous indication of mangling).
>>>
>>> And this prompts me to edit the bullet in an earlier section about the
>>> obligation of a sender to respond to congestion feedback:
>>>
>>>
>>>         3.1.5.
>>> <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#section-3.1.5>Implications
>>>         of AccECN Mode
>>> <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#name-implications-of-accecn-mode>
>>>
>>>
>>>     "It is still obliged to respond appropriately to AccECN feedback
>>>     that indicates there were ECN marks on *ECN-capable *packets it
>>>     had previously sent."
>>>
>>> And I'll add a bullet:
>>>
>>>     "*If the sender chooses not to send ECN-capable packets (e.g.
>>>     because path traversal of the ECN field is suspect), it can ignore
>>>     any **ECN **feedback about those packets if it is certain that it
>>>     cannot be valid (see Section 3.2.2, which gives normative
>>>     requirements for certain specific cases).*"
>>>
>>> I've said 'can' rather than 'MUST' because it's hard to cover all
>>> cases, e.g. single packets without ECT when it's not clear whether the
>>> feedback covered another packet that was ECT.
>>>
>>> How does all this sound?
>>>
>>>> 2. On the reverse path from server to client, if a server sends a
>>>> Not-ECT SYN-ACK and receives ACE handshake encoding on last ACK other
>>>> than 0 1 0, there is no text like above that says server should send
>>>> Not-ECT for the rest of the connection (or at least I didn’t find
>>>> it). I think the server should also do same as client as the two
>>>> paths could be different. One could make it more complicated by
>>>> saying, if both client and server decide to not use ECN on their
>>>> corresponding sender half, then ECN should be disabled but let’s talk
>>>> about that later.
>>>
>>> [BB] It's in the second para of:
>>>
>>>
>>>           3.2.2.3.
>>> <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#section-3.2.2.3>Testing
>>>           for Mangling of the IP/ECN Field
>>> <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#name-testing-for-mangling-of-the>
>>>
>>>
>>>> 3. In general, not setting the ECT (ECT0 or ECT1) code point on an
>>>> outgoing packet is different from supporting AccECN right as in the
>>>> host can still provide AccECN feedback on the receive path.
>>>
>>> [BB] Yup.
>>>
>>> Do you think any further explanation is needed in the above sections?
>>>
>>>
>>> Bob
>>>
>>>> ….
>>>>
>>>> To be continued if more questions come to my mind.
>>>>
>>>>
>>>> Thanks,
>>>> Vidhi
>>>>
>>>>> On Feb 3, 2022, at 7:24 AM, Bob Briscoe <[e-mail]> wrote:
>>>>>
>>>>> tcpm folks,
>>>>>
>>>>> This rev to accurate-ecn is the first of two. The second will
>>>>> hopefully follow on its heels in the next couple of days.
>>>>>
>>>>> Diffs between this first rev (-16) and -15:
>>>>>
>>>>> 1.  switches round two fairly large sections, so I've deferred other
>>>>> changes to a second rev so the diffs won't be masked by the switch
>>>>> round of sections.
>>>>> Suggested by Ilpo to match the order in which the tests in these
>>>>> sections will be executed:
>>>>> * Test for mangling the IP-ECN field (now 3.2.2.3),
>>>>> * Then test for zeroing the ACE field (now 3.2.2.4).
>>>>>
>>>>> 2. Ilpo suggested some clarifications in "3.2.3.2.5. Consistency
>>>>> between AccECN Feedback Fields", which is about the receiver of
>>>>> feedback ensuring consistency between the mandatory 3-bit ACE field
>>>>> and the optional 24-bit counters. In brief (paraphrasing) it
>>>>> previously only said "MUST consider both fields", when it is now
>>>>> clearer that it actually meant "MUST reconcile both fields", so that
>>>>> there is always a consistent baseline for subsequent ACKs.
>>>>>
>>>>> 3. A minor point is added in an appendix about the details that the
>>>>> pseudocode doesn't cover.
>>>>>
>>>>>
>>>>>
>>>>> Bob
>>>>>
>>>>> PS. #2 & #3 were added to the XML ages ago (Jul '21), so you will
>>>>> have seen them in the HTML. However, prob due to my clumsiness, the
>>>>> posted TXT didn't include them whereas the posted XML did (ironic
>>>>> for a section about consistency). In turn, inconsistency was only
>>>>> possible because I am having to manually post the TXT for this
>>>>> draft, due to an unresolved issue with v3 RFC XML tables.
>>>>>
>>>>>
>>>>> On 03/02/2022 14:43, [e-mail] wrote:
>>>>>> A New Internet-Draft is available from the on-line Internet-Drafts
>>>>>> directories.
>>>>>> This draft is a work item of the TCP Maintenance and Minor
>>>>>> Extensions WG of the IETF.
>>>>>>
>>>>>>         Title           : More Accurate ECN Feedback in TCP
>>>>>>         Authors         : Bob Briscoe
>>>>>>                           Mirja Kühlewind
>>>>>>                           Richard Scheffenegger
>>>>>> Filename        : draft-ietf-tcpm-accurate-ecn-16.txt
>>>>>> Pages           : 60
>>>>>> Date            : 2022-02-03
>>>>>>
>>>>>> Abstract:
>>>>>>    Explicit Congestion Notification (ECN) is a mechanism where 
>>>>>> network
>>>>>>    nodes can mark IP packets instead of dropping them to indicate
>>>>>>    incipient congestion to the end-points.  Receivers with an ECN-
>>>>>>    capable transport protocol feed back this information to the 
>>>>>> sender.
>>>>>>    ECN was originally specified for TCP in such a way that only one
>>>>>>    feedback signal can be transmitted per Round-Trip Time (RTT).
>>>>>>  Recent
>>>>>>    new TCP mechanisms like Congestion Exposure (ConEx), Data 
>>>>>> Center TCP
>>>>>>    (DCTCP) or Low Latency Low Loss Scalable Throughput (L4S) need 
>>>>>> more
>>>>>>    accurate ECN feedback information whenever more than one 
>>>>>> marking is
>>>>>>    received in one RTT.  This document specifies a scheme to provide
>>>>>>    more than one feedback signal per RTT in the TCP header. 
>>>>>>  Given TCP
>>>>>>    header space is scarce, it allocates a reserved header bit
>>>>>> previously
>>>>>>    assigned to the ECN-Nonce.  It also overloads the two existing 
>>>>>> ECN
>>>>>>    flags in the TCP header.  The resulting extra space is 
>>>>>> exploited to
>>>>>>    feed back the IP-ECN field received during the 3-way handshake as
>>>>>>    well.  Supplementary feedback information can optionally be 
>>>>>> provided
>>>>>>    in a new TCP option, which is never used on the TCP SYN.  The
>>>>>>    document also specifies the treatment of this updated TCP wire
>>>>>>    protocol by middleboxes.
>>>>>>
>>>>>>
>>>>>> The IETF datatracker status page for this draft is:
>>>>>> https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ecn/
>>>>>>
>>>>>> There is also an htmlized version available at:
>>>>>> https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-16 
>>>>>>
>>>>>>
>>>>>> A diff from the previous version is available at:
>>>>>> https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-accurate-ecn-16
>>>>>>
>>>>>>
>>>>>> Internet-Drafts are also available by rsync at rsync.ietf.org
>>>>>> <http://rsync.ietf.org>::internet-drafts
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> tcpm mailing list
>>>>>> [e-mail]
>>>>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>>>
>>>>> -- 
>>>>> ________________________________________________________________
>>>>> Bob Briscoe http://bobbriscoe.net/
>>>>>
>>>>> _______________________________________________
>>>>> tcpm mailing list
>>>>> [e-mail]
>>>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>>
>>>
>>> -- 
>>> ________________________________________________________________
>>> Bob Briscoehttp://bobbriscoe.net/
>>
>>
>> _______________________________________________
>> tcpm mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/tcpm

-- 
________________________________________________________________
Bob Briscoehttp://bobbriscoe.net/