Re: [tcpm] New Version Notificationfordraft-ietf-tcpm-accurate-ecn-11.txt

[Bob Briscoe <[e-mail]>]

Ilpo,

On 11/03/2020 12:37, Ilpo Järvinen wrote:
> Hi all,
>
> Here's the option format with the extra octet:
>
>      0                   1                   2                   3
>      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>     |  Kind = TBD1  |  Length = 11  | Reserved  |Cnt|   EE0B field  |
>     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>     | EE0B (cont'd)                 |           ECEB field          |
>     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>     |               |                  EE1B field                   |
>     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
> Cnt indicates the counter (IP-ECN field value) from the last received
> packet. The receiver incremented this counter with this ACK. The counter
> value might have not increased if the payload length was zero but
> a subsequent packet would increase the same counter if IP-ECN field
> remains the same.
>
> The octet containing Cnt MUST be included when any byte counter is
> included.
I don't think this needs to be a MUST at all. The most important thing 
is to include the Cnt bits when there are no other counters (the case in 
the sentences below).
> It MAY be left out otherwise.
I think you are referring to an empty option on the SYN/ACK here (to 
declare that the server is sending options). I can't think of any other 
use for an empty option.

> If an AccECN option without bytes
> counters is increased but Cnt value is included and changed, the sender
> switches to the indicated counter (but cannot sync counters which may be
> be somewhat off in case of ACK losses).
This last sentence is the most important part that you hadn't stated before.
     (I assume there's a typo: s/increased/included/.)
But you haven't said whether it's MUST or SHOULD (except the previous 
sentence says that it MAY be left out).
It cannot be a MUST (in case there isn't any option space on a 
particular packet).
If we were to go in this direction, this sentence surely ought be a 
"SHOULD" with the same strength as the "SHOULD" for Change-Triggered ACKs.


>
> If the Option is sent in a change-triggered ACK, the Cnt field will
> differ from the previously sent value. For other ACKs the option, Cnt is
> the same as previously. As a new AccECN options should be sent in
> the next change-triggered ACK, the IP-ECN field edges are captured.
>
> Now, even if there were any number of ACK losses prior to receiving
> such an AccECN option, the sender can immediately sync with the receiver's
> state
I think you only mean the receiver's state relating to which ACK counter 
is currently changing. The state of the counters can still be wrong, due 
to previous ACK losses.

> and knows what counter the receiver will be increasing, no need to
> guess that based on any byte counter delta (on the next ACK if it
> doesn't contain AccECN option).
>
> The main point with Cnt is to help the sender to do more accurate
> estimation in the cases where ACKs without AccECN option arrive.
> As such estimation needs only run on ACKs, the Cnt information does
> not require sending extra ACKs at any point of time. In fact, the
> change-triggered ACKs requirement could even be relaxed with it
> because together the byte counters and Cnt sync the AccECN state
> such that knowing the exact point of the IP-ECN field edge is no
> longer as necessary as earlier.
I believe you are saying that the meaning of Cnt on an acceptable ACK is 
twofold:
1) For the past:
     * If the byte counter indicated by Cnt is absent from this option, 
consider the new bytes acked by this packet to have incremented that 
counter.
     * If the byte counter indicated by Cnt is present on this option 
??? (Does it have any other use for the past?)
2) For the future:
     * If there is no AccECN Option on a subsequent acceptable ACK that 
acknowledges new data, consider the new bytes acked by that packet to 
have incremented the byte counter indicated by the last seen value of Cnt.

In the presence of ACK loss:
* Meaning#1 is much more useful.
* Meaning #2 is less useful, because if ACKs are being thinned or lost, 
the value of Cnt most recently received by the Data Sender won't 
necessarily be the one most recently sent by the Data Receiver.

Assessment:
* Meaning #2: To put it ironically, this scheme tries to solve the 
problem of losing ACKs that say which counter will change in future, by 
introducing a field on an ACK that says which counter will change in 
future. But the ACK carrying this new field is just as likely to be lost 
as the old way.

* Meaning #1: Then the question has to be, if Cnt is most useful on a 
packet without the associated counter (meaning#1), then why not include 
that counter? IOW, Cnt consumes 3B to save 5B (or sometimes to save 8B) 
(including the overhead of the option kind and length). And note that 
Cnt is still ambiguous, whereas including the actual counter is unambiguous.

I guess the idea of Cnt arose from reading all the SHOULDs about when to 
send an AccECN Option as if they might not be adhered to. So this 
additional 1B field is really only solving a problem where there is not 
space for 5B or 8B, but there is space for 3B.

>
> If Cnt is zero, the receiver got a packet with non-ECT and no counter
> is increasing as a result of the ACK or subsequent ACKs until next
> the AccECN option arrival.
More precisely, I think you mean that all the new bytes ACK'd by this 
packet were Not-ECT.
That's useful. Without Cnt this is only really possible by sending an 
option with all three counters then sending one later with the counters 
unchanged.

>
> Since the byte counter indicated with Cnt field is typically increasing,
> it would be possible to make the Cnt value dual purpose and use it to
> indicate which counter appears first in the Option (if there are any
> counters in the first place). This would partically resolve Michael's
> concern about the field ordering. Other counters that have changed since
> the receiver sent last AccECN option should be included after the
> first field (in some order, specifying the order of the remaining
> fields is beyond this Cnt proposal).
As you say, we don't have to do this.
I'm yet to be convinced of the value of allowing the field order to 
change on each packet, just because we want the field order to be able 
to be changed for each connection.


Bob

>
> --
>   i.
>
>
> On Tue, 10 Mar 2020, Bob Briscoe wrote:
>
>> Ilpo,
>>
>> Can you please describe the idea more fully? I cannot understand it from the
>> description, although I thought I understood it when you explained it to me
>> (off list).
>>
>> Here's my understanding of the problem, for the benefit of the list (Ilpo
>> and I have been discussing this off list, and I don't think the list will
>> understand without this):
>>
>> * This is all in the context of a Receiver that is sending AccECN TCP
>> Options. It doesn't have to send these options at all but, if it does, they
>> complement and help disambiguate the 3-bit ACE counter in the main TCP
>> header (which is always used).
>> * Further context: Even if a receiver is sending the AccECN Option, it can
>> choose to sometimes send ACKs without an AccECN Option, and the sender can
>> then assume that all the bytes ACK'd had the same ECN marking (perhaps
>> wrongly if there have been ACK losses).
>> * The proposed rules for when to use the AccECN Option are here:
>> https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn-11#section-3.2.3.3
>>
>>
>> The first rule is:
>>
>>     Change-Triggered ACKs:  If an arriving packet increments a different
>>        byte counter to that incremented by the previous packet, the Data
>>        Receiver SHOULD immediately send an ACK with an AccECN Option,
>>        without waiting for the next delayed ACK [...].
>>
>>        Even though this bullet is stated as a "SHOULD", it is important
>>        for a transition to immediately trigger an ACK if at all possible,
>>        ...
>> Originally Ilpo thought this made the sender implementation too difficult,
>> even in very common delayed ACK cases, e.g.:
>>
>>      ECT1
>> ACK#1 (ECT1 counter)
>>      ECT1
>>      CE
>> ACK#2 (ECT1 & CE counters)
>>      ECT1
>> ACK#3 (ECT1 counter)
>>      ECT1
>>      ECT1
>> ACK#4 (no AccECN Option)
>>      ECT1
>>      ECT1
>> ACK#5 (no AccECN Option)
>> ...
>>
>> In ACK#2, both ECT1 and CE counters have incremented, so how does the sender
>> know which counter triggered this ACK? it needs to know this to guess which
>> counter to increment if it gets an ACK without an AccECN Option.
>> My answer: the sender can infer that CE triggered the second ACK shown,
>> because in the previous ACK, only ECT1 was incrementing.
>>
>> Ilpo was concerned that, in the case where ACK#3 is lost or thinned, the
>> sender could assume from the subsequent lack of any AccECN Options that more
>> and more CE was arriving, when actually more and more ECT1 was arriving.
>>
>> I agreed, but my argument was that there are more "SHOULD" rules for the
>> receiver, such as:
>>
>>        *  It SHOULD always include an AccECN Option if the r.ceb counter
>>           is incrementing and it MAY include an AccECN Option if r.ec0b
>>           or r.ec1b is incrementing
>> So, certainly it is possible that there will be ambiguous occasions when
>> there is no AccECN Option. However, the idea is not to omit an AccECN option
>> at all unless there is no space (e.g. because some other option takes
>> priority). So these cases where assumptions have to be made are not intended
>> to be routine - the rules are only written as SHOULDs in order to give the
>> receiver flexibility if it needs the space to send other options. it is not
>> expected to break all the SHOULD rules all the time.
>>
>> Nonetheless, it did make sense to add this rule, at Ilpo's suggestion:
>>
>>        *  It SHOULD include a counter that has continued to increment on
>>           the next scheduled ACK following a change-triggered ACK;
>> For example, ACks #3 and #5 below are due to this rule:
>>
>>      ECT1
>> ACK#1 (ECT1 counter)
>>      ECT1
>>      CE
>> ACK#2 (ECT1 & CE counters)
>>      CE
>>      CE
>> ACK#3 (CE counter)
>>      ECT1
>> ACK#4 (ECT1 counter)
>>      ECT1
>>      ECT1
>> ACK#5 (ECT1 counter)
>>      ECT1
>>      ECT1
>> ACK#6 (no AccECN Option)
>> ...
>>
>>
>> But I pointed out that, of course, this doesn't fully solve the problem,
>> because this second AccECN Option could be lost, just as easily as the first
>> (obviously a lower probability that they are both lost, but this could be
>> common with heavy ACK thinning).
>>
>> It was at this point that Ilpo suggested that a 2-bit field in a 1-byte
>> option could specify which was the last counter to have changed.
>>
>> However, now I re-read both Ilpo's offlist and the onlist emails, I'm not
>> sure I understand the rules of the scheme, whereas I thought I did.
>>
>>
>> Ilpo, can you write a really precise and full spec of your proposal, stating
>> all assumptions?
>> Are ACKs without any AccECN Option allowed? Or are you saying this 1 byte
>> option MUST always be included?
>>
>>
>>
>> Bob
>>
>> On 10/03/2020 06:38, Scharf, Michael wrote:
>>
>>        Yep, if one byte was added, there would probably be remaining
>>        bits left.
>>
>>         
>>
>>        RFC 5218 Section 2.2.1: „Protocols that are extensible are more
>>        likely to be wildly successful in terms of being used for
>>        purposes outside their original design.“
>>
>>         
>>
>>        Michael
>>
>>         
>>
>>        Von: Ilpo Järvinen
>>        Gesendet: Montag, 9. März 2020 23:32
>>        An: Scharf, Michael
>>        Cc: Bob Briscoe; tcpm IETF list; Richard Scheffenegger; Mirja
>>        Kuehlewind
>>        Betreff: Re: [tcpm] New Version Notification
>>        fordraft-ietf-tcpm-accurate-ecn-11.txt
>>
>>   
>>
>> Hi Michael,
>>
>>   
>>
>> I agree with out  that it's somewhat painful to deal with the marker
>> bit
>>
>> at the beginning for the option field order.
>>
>>   
>>
>> Another challenge I've come across is to correctly reproduce the
>>
>> Accurate ECN state at the sender which would benefit from having
>>
>> a few extra bits available.
>>
>>   
>>
>> It currently requires guessing at every option arrival what the
>> receiver
>>
>> is doing next (what counter it will be increasing next) which is not
>> very
>>
>> robust to ACK losses. On change-triggered ACK, two byte counters are
>>
>> increasing rather than one. ACK losses will easily confuse approach
>> that
>>
>> just assumes back-and-forth transitions. Thus, I suggested sending one
>>
>> extra option after the change-triggered ACK to allow sender to see
>>
>> unambiguous counter delta between the change-triggered ACK and that
>>
>> next ACK (it is now in -11).
>>
>>   
>>
>> To allow the sender to immediately sync with the receiver on any
>>
>> AccECN option arrival, it would make sense to encode that information
>>
>> into the option itself. It would need 2 extra bits into the option
>>
>> so adding that one byte there would help. ...It might be possible to
>>
>> combine it to field order and define that the first encoded field is
>>
>> the currently increasing counter (the order of the remaining two
>> fields
>>
>> still need to be addressed somehow).
>>
>>   
>>
>> --
>>
>>   i.
>>
>>   
>>
>> On Mon, 9 Mar 2020, Michael Scharf wrote:
>>
>>   
>>
>>> With chair hat off, I really wonder if the solution to encode the
>> two
>>
>>> different orders in the TCP Option is an example for good and robust
>>> protocol engineering.
>>>   
>>> For instance, the current design makes it hard to decode the field
>> in a
>>
>>> monitoring tool (such as Wireshark). Also, as far as I understand,
>> it does
>>
>>> not allow to switch the encoding during a connection, which limits
>>> flexibility. We almost certainly do not understand *now* all future
>> use
>>
>>> cases of this Standard.
>>>   
>>> Unless I miss something, there would be several other solutions:
>>>   
>>> First, IMHO, we have enough TCP option codepoints left to spend two
>>> codepoints if there is a good reason for doing so. As compared to
>> the
>>
>>> current design proposal in -10/-11, spending two different option
>> kinds
>>
>>> would look to me like *much* better protocol engineering.
>>>   
>>> Second, if the TCPM community insists in only one option kind
>> codepoint for
>>
>>> whatever reason, IMHO one could add one „sub-type“ byte to the
>> option. The
>>
>>> TCP Option field has to be multiples of 4 byte, i.e., if a segment
>> only
>>
>>> contains a 11 byte AccECN TCP option, an additional NOP TCP option
>> is needed
>>
>>> for padding, no? So, what downside have 12 bytes as compared to 11
>> bytes?
>>
>>> For the shorter variants, the overhead of a „sub-type“ field
>> increases, but
>>
>>> it may still be within reasonable limits. What do I miss?
>>>   
>>> Third, one could use different lengths for the different orders,
>> e.g.,
>>
>>> lenths 5/8/11 for type 0 and 6/9/12 for type 12. Is this not
>> possible?
>>
>>>   
>>> In all these cases, the resulting protocol looks simpler and more
>> robust to
>>
>>> me. What prevents us from using the KISS principle?
>>>   
>>> Michael
>>>   
>>>   
>>>   
>>> Von: Bob Briscoe
>>> Gesendet: Freitag, 6. März 2020 04:34
>>> An: tcpm IETF list
>>> Cc: Richard Scheffenegger; Mirja Kuehlewind
>>> Betreff: Re: [tcpm] New Version Notification for
>>> draft-ietf-tcpm-accurate-ecn-11.txt
>>>   
>>> tcpm,
>>> You will have seen draft-10 then draft-11 in quick succession, as
>> already
>>
>>> explained.
>>> The diffs from draft-09 to -10 were those that had built up since
>> Jul'19.
>>
>>> The diffs from draft-10 to -11 were solely those for the change from
>> EXP
>>
>>> track to STD track.
>>> Draft-10 doesn't seem to display in the list of links to each
>> version, but
>>
>>> you can manually write the URL.
>>> The main technical changes in draft-10 were numerous - many will be
>>> recognized from list discussion since Jul'19.
>>> Particular thanks to Ilpo Järvinen who identified many niggles (and
>> their
>>
>>> solutions) while writing and testing a full Linux implementation
>> (based on
>>
>>> Olivier Tilmans's, in turn based on Mirja's).
>>>    *  Allowed 2 different orders of the fields in the AccECN Option
>>>    *  Reflect IP-ECN field of SYN/ACK only on ACK of SYN/ACK, not also
>> on
>>
>>>       first data packet
>>>        +  greatly simplifies implementation, esp with TFO.
>>>        +  repeating on first data packet was for reliable delivery,
>> which is
>>
>>>           now achieved with ACE counter (see next bullet)
>>>    *  Increment the ACE counter if CE on SYN/ACK (but still not if CE
>> on SYN)
>>
>>>        +  Reliable delivery of feedback of CE on SYN/ACK
>>>    *  Redefine 'first packet' as first to arrive, not first in
>> sequence in 2
>>
>>>       cases:
>>>        +  Handshake reflection on the ACK of the SYN/ACK
>>>        +  In the test for zeroing of ACE
>>>        +  Reason: greatly simplifies implementation
>>>    *  if ACE could have wrapped more than once, SHOULD assume “safest
>> likely
>>
>>>       case”
>>>       not "conservatively assume" it did cycle
>>>        +  Reason: avoid unnecessary hit on performance
>>>    *  More robustness (with flexibility) in rules for when to include
>> an
>>
>>>       AccECN Option
>>>        +  Change-triggered AccECN Option as SHOULD, not MUST
>>>        +  SHOULD follow change-triggered AccECN Option with another
>> (removes
>>
>>>           ambiguity if ACK thinning or loss)
>>>        +  when same counter continues to increment, SHOULD
>> consistently
>>
>>>           include it every n ACKs
>>>        +  Made rule about precedence of SACK conditional (max 2 SACK
>> blocks)
>>
>>>        +  MAY exclude counters that have not changed for the whole
>> connection
>>
>>>    *  Allowed an AccECN server not to implement RFC3168 ECN (all
>> clients still
>>
>>>       have to)
>>>    *  Precluded mixed capability negotiation from either end
>>>        +  reduces freedom to choose SYN & SYN/ACK fall-back strategies
>>>        +  to prevent cases where each end's outcome after handshake
>> could be
>>
>>>           inconsistent (in reordering corner-cases)
>>>    *  Reserved the codepoint combination used by the historic nonce
>> case
>>
>>>    *  Merged in a number of points from RFC3168 that we hadn't covered
>>>        +  (a whole new subsection about obligations to do with ECN)
>>>    *  Explicit about checking "acceptable packets"
>>>        +  before counting their ECN markings or before counting the
>> ECN
>>
>>>           feedback they carry
>>>    *  Required retransmitted Fallback SYN to use same ISN
>>>        +  allows servers to detect ECN downgrade SYN attacks
>>>    *  Handled corner cases like In-window SYN during TIME-WAIT
>>> Bob
>>> On 06/03/2020 02:24, [e-mail] wrote:
>>>   
>>> A new version of I-D, draft-ietf-tcpm-accurate-ecn-11.txt
>>> has been successfully submitted by Bob Briscoe and posted to the
>>> IETF repository.
>>>   
>>> Name:            draft-ietf-tcpm-accurate-ecn
>>> Revision: 11
>>> Title:           More Accurate ECN Feedback in TCP
>>> Document date:   2020-03-05
>>> Group:           tcpm
>>> Pages:           58
>>> URL:
>> https://www.ietf.org/internet-drafts/draft-ietf-tcpm-accurat
>>
>>> e-ecn-11.txt
>>> Status:
>> https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ec
>>
>>> n/
>>> Htmlized:
>> https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn-11
>>
>>> Htmlized:
>> https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accura
>>
>>> te-ecn
>>> Diff:
>> https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-accurate-e
>>
>>> cn-11
>>>   
>>> Abstract:
>>>     Explicit Congestion Notification (ECN) is a mechanism where
>> network
>>
>>>     nodes can mark IP packets instead of dropping them to indicate
>>>     incipient congestion to the end-points.  Receivers with an ECN-
>>>     capable transport protocol feed back this information to the
>> sender.
>>
>>>     ECN is specified for TCP in such a way that only one feedback
>> signal
>>
>>>     can be transmitted per Round-Trip Time (RTT).  Recent new TCP
>>>     mechanisms like Congestion Exposure (ConEx), Data Center TCP
>> (DCTCP)
>>
>>>     or Low Latency Low Loss Scalable Throughput (L4S) need more
>> accurate
>>
>>>     ECN feedback information whenever more than one marking is
>> received
>>
>>>     in one RTT.  This document specifies a scheme to provide more
>> than
>>
>>>     one feedback signal per RTT in the TCP header.  Given TCP header
>>>     space is scarce, it allocates a reserved header bit, that was
>>>     previously used for the ECN-Nonce which has now been declared
>>>     historic.  It also overloads the two existing ECN flags in the
>> TCP
>>
>>>     header.  The resulting extra space is exploited to feed back the
>> IP-
>>
>>>     ECN field received during the 3-way handshake as well.
>> Supplementary
>>
>>>     feedback information can optionally be provided in a new TCP
>> option,
>>
>>>     which is never used on the TCP SYN.
>>>   
>>>         
>>                                                                      
>>
>>>        
>>>   
>>>   
>>> Please note that it may take a couple of minutes from the time of
>> submission
>>
>>> until the htmlized version and diff are available at tools.ietf.org.
>>>   
>>> The IETF Secretariat
>>>   
>>>   
>>>   
>>   
>>
>>
> >

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/