Re: [tcpm] New Version Notificationfordraft-ietf-tcpm-accurate-ecn-11.txt

[Ilpo Järvinen <[e-mail]>]

Hi all,

Here's the option format with the extra octet:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Kind = TBD1  |  Length = 11  | Reserved  |Cnt|   EE0B field  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | EE0B (cont'd)                 |           ECEB field          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               |                  EE1B field                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Cnt indicates the counter (IP-ECN field value) from the last received
packet. The receiver incremented this counter with this ACK. The counter
value might have not increased if the payload length was zero but
a subsequent packet would increase the same counter if IP-ECN field
remains the same.

The octet containing Cnt MUST be included when any byte counter is
included. It MAY be left out otherwise. If an AccECN option without bytes 
counters is increased but Cnt value is included and changed, the sender 
switches to the indicated counter (but cannot sync counters which may be 
be somewhat off in case of ACK losses).

If the Option is sent in a change-triggered ACK, the Cnt field will
differ from the previously sent value. For other ACKs the option, Cnt is 
the same as previously. As a new AccECN options should be sent in
the next change-triggered ACK, the IP-ECN field edges are captured.

Now, even if there were any number of ACK losses prior to receiving
such an AccECN option, the sender can immediately sync with the receiver's 
state and knows what counter the receiver will be increasing, no need to 
guess that based on any byte counter delta (on the next ACK if it 
doesn't contain AccECN option).

The main point with Cnt is to help the sender to do more accurate 
estimation in the cases where ACKs without AccECN option arrive.
As such estimation needs only run on ACKs, the Cnt information does
not require sending extra ACKs at any point of time. In fact, the
change-triggered ACKs requirement could even be relaxed with it
because together the byte counters and Cnt sync the AccECN state
such that knowing the exact point of the IP-ECN field edge is no
longer as necessary as earlier.

If Cnt is zero, the receiver got a packet with non-ECT and no counter
is increasing as a result of the ACK or subsequent ACKs until next
the AccECN option arrival.

Since the byte counter indicated with Cnt field is typically increasing,
it would be possible to make the Cnt value dual purpose and use it to
indicate which counter appears first in the Option (if there are any
counters in the first place). This would partically resolve Michael's
concern about the field ordering. Other counters that have changed since 
the receiver sent last AccECN option should be included after the
first field (in some order, specifying the order of the remaining
fields is beyond this Cnt proposal).

--
 i.


On Tue, 10 Mar 2020, Bob Briscoe wrote:

> Ilpo,
> 
> Can you please describe the idea more fully? I cannot understand it from the
> description, although I thought I understood it when you explained it to me
> (off list).
> 
> Here's my understanding of the problem, for the benefit of the list (Ilpo
> and I have been discussing this off list, and I don't think the list will
> understand without this):
> 
> * This is all in the context of a Receiver that is sending AccECN TCP
> Options. It doesn't have to send these options at all but, if it does, they
> complement and help disambiguate the 3-bit ACE counter in the main TCP
> header (which is always used).
> * Further context: Even if a receiver is sending the AccECN Option, it can
> choose to sometimes send ACKs without an AccECN Option, and the sender can
> then assume that all the bytes ACK'd had the same ECN marking (perhaps
> wrongly if there have been ACK losses).
> * The proposed rules for when to use the AccECN Option are here:
> https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn-11#section-3.2.3.3
> 
> 
> The first rule is:
> 
>    Change-Triggered ACKs:  If an arriving packet increments a different
>       byte counter to that incremented by the previous packet, the Data
>       Receiver SHOULD immediately send an ACK with an AccECN Option,
>       without waiting for the next delayed ACK [...].
> 
>       Even though this bullet is stated as a "SHOULD", it is important
>       for a transition to immediately trigger an ACK if at all possible,
>       ...
> Originally Ilpo thought this made the sender implementation too difficult,
> even in very common delayed ACK cases, e.g.:
> 
>     ECT1
> ACK#1 (ECT1 counter)
>     ECT1
>     CE
> ACK#2 (ECT1 & CE counters)
>     ECT1
> ACK#3 (ECT1 counter)
>     ECT1
>     ECT1
> ACK#4 (no AccECN Option)
>     ECT1
>     ECT1
> ACK#5 (no AccECN Option)
> ...
> 
> In ACK#2, both ECT1 and CE counters have incremented, so how does the sender
> know which counter triggered this ACK? it needs to know this to guess which
> counter to increment if it gets an ACK without an AccECN Option.
> My answer: the sender can infer that CE triggered the second ACK shown,
> because in the previous ACK, only ECT1 was incrementing.
> 
> Ilpo was concerned that, in the case where ACK#3 is lost or thinned, the
> sender could assume from the subsequent lack of any AccECN Options that more
> and more CE was arriving, when actually more and more ECT1 was arriving.
> 
> I agreed, but my argument was that there are more "SHOULD" rules for the
> receiver, such as:
> 
>       *  It SHOULD always include an AccECN Option if the r.ceb counter
>          is incrementing and it MAY include an AccECN Option if r.ec0b
>          or r.ec1b is incrementing
> So, certainly it is possible that there will be ambiguous occasions when
> there is no AccECN Option. However, the idea is not to omit an AccECN option
> at all unless there is no space (e.g. because some other option takes
> priority). So these cases where assumptions have to be made are not intended
> to be routine - the rules are only written as SHOULDs in order to give the
> receiver flexibility if it needs the space to send other options. it is not
> expected to break all the SHOULD rules all the time.
> 
> Nonetheless, it did make sense to add this rule, at Ilpo's suggestion:
> 
>       *  It SHOULD include a counter that has continued to increment on
>          the next scheduled ACK following a change-triggered ACK;
> For example, ACks #3 and #5 below are due to this rule:
> 
>     ECT1
> ACK#1 (ECT1 counter)
>     ECT1
>     CE
> ACK#2 (ECT1 & CE counters)
>     CE
>     CE
> ACK#3 (CE counter)
>     ECT1
> ACK#4 (ECT1 counter)
>     ECT1
>     ECT1
> ACK#5 (ECT1 counter)
>     ECT1
>     ECT1
> ACK#6 (no AccECN Option)
> ...
> 
> 
> But I pointed out that, of course, this doesn't fully solve the problem,
> because this second AccECN Option could be lost, just as easily as the first
> (obviously a lower probability that they are both lost, but this could be
> common with heavy ACK thinning).
> 
> It was at this point that Ilpo suggested that a 2-bit field in a 1-byte
> option could specify which was the last counter to have changed.
> 
> However, now I re-read both Ilpo's offlist and the onlist emails, I'm not
> sure I understand the rules of the scheme, whereas I thought I did.
> 
> 
> Ilpo, can you write a really precise and full spec of your proposal, stating
> all assumptions?
> Are ACKs without any AccECN Option allowed? Or are you saying this 1 byte
> option MUST always be included?
> 
> 
> 
> Bob
> 
> On 10/03/2020 06:38, Scharf, Michael wrote:
> 
>       Yep, if one byte was added, there would probably be remaining
>       bits left.
> 
>        
> 
>       RFC 5218 Section 2.2.1: „Protocols that are extensible are more
>       likely to be wildly successful in terms of being used for
>       purposes outside their original design.“
> 
>        
> 
>       Michael
> 
>        
> 
>       Von: Ilpo Järvinen
>       Gesendet: Montag, 9. März 2020 23:32
>       An: Scharf, Michael
>       Cc: Bob Briscoe; tcpm IETF list; Richard Scheffenegger; Mirja
>       Kuehlewind
>       Betreff: Re: [tcpm] New Version Notification
>       fordraft-ietf-tcpm-accurate-ecn-11.txt
> 
>  
> 
> Hi Michael,
> 
>  
> 
> I agree with out  that it's somewhat painful to deal with the marker
> bit
> 
> at the beginning for the option field order.
> 
>  
> 
> Another challenge I've come across is to correctly reproduce the
> 
> Accurate ECN state at the sender which would benefit from having
> 
> a few extra bits available.
> 
>  
> 
> It currently requires guessing at every option arrival what the
> receiver
> 
> is doing next (what counter it will be increasing next) which is not
> very
> 
> robust to ACK losses. On change-triggered ACK, two byte counters are
> 
> increasing rather than one. ACK losses will easily confuse approach
> that
> 
> just assumes back-and-forth transitions. Thus, I suggested sending one
> 
> extra option after the change-triggered ACK to allow sender to see
> 
> unambiguous counter delta between the change-triggered ACK and that
> 
> next ACK (it is now in -11).
> 
>  
> 
> To allow the sender to immediately sync with the receiver on any
> 
> AccECN option arrival, it would make sense to encode that information
> 
> into the option itself. It would need 2 extra bits into the option
> 
> so adding that one byte there would help. ...It might be possible to
> 
> combine it to field order and define that the first encoded field is
> 
> the currently increasing counter (the order of the remaining two
> fields
> 
> still need to be addressed somehow).
> 
>  
> 
> --
> 
>  i.
> 
>  
> 
> On Mon, 9 Mar 2020, Michael Scharf wrote:
> 
>  
> 
> >
> 
> > With chair hat off, I really wonder if the solution to encode the
> two
> 
> > different orders in the TCP Option is an example for good and robust
> 
> > protocol engineering.
> 
> >
> 
> >  
> 
> >
> 
> > For instance, the current design makes it hard to decode the field
> in a
> 
> > monitoring tool (such as Wireshark). Also, as far as I understand,
> it does
> 
> > not allow to switch the encoding during a connection, which limits
> 
> > flexibility. We almost certainly do not understand *now* all future
> use
> 
> > cases of this Standard.
> 
> >
> 
> >  
> 
> >
> 
> > Unless I miss something, there would be several other solutions:
> 
> >
> 
> >  
> 
> >
> 
> > First, IMHO, we have enough TCP option codepoints left to spend two
> 
> > codepoints if there is a good reason for doing so. As compared to
> the
> 
> > current design proposal in -10/-11, spending two different option
> kinds
> 
> > would look to me like *much* better protocol engineering.
> 
> >
> 
> >  
> 
> >
> 
> > Second, if the TCPM community insists in only one option kind
> codepoint for
> 
> > whatever reason, IMHO one could add one „sub-type“ byte to the
> option. The
> 
> > TCP Option field has to be multiples of 4 byte, i.e., if a segment
> only
> 
> > contains a 11 byte AccECN TCP option, an additional NOP TCP option
> is needed
> 
> > for padding, no? So, what downside have 12 bytes as compared to 11
> bytes?
> 
> > For the shorter variants, the overhead of a „sub-type“ field
> increases, but
> 
> > it may still be within reasonable limits. What do I miss?
> 
> >
> 
> >  
> 
> >
> 
> > Third, one could use different lengths for the different orders,
> e.g.,
> 
> > lenths 5/8/11 for type 0 and 6/9/12 for type 12. Is this not
> possible?
> 
> >
> 
> >  
> 
> >
> 
> > In all these cases, the resulting protocol looks simpler and more
> robust to
> 
> > me. What prevents us from using the KISS principle?
> 
> >
> 
> >  
> 
> >
> 
> > Michael
> 
> >
> 
> >  
> 
> >
> 
> >  
> 
> >
> 
> >  
> 
> >
> 
> > Von: Bob Briscoe
> 
> > Gesendet: Freitag, 6. März 2020 04:34
> 
> > An: tcpm IETF list
> 
> > Cc: Richard Scheffenegger; Mirja Kuehlewind
> 
> > Betreff: Re: [tcpm] New Version Notification for
> 
> > draft-ietf-tcpm-accurate-ecn-11.txt
> 
> >
> 
> >  
> 
> >
> 
> > tcpm,
> 
> >
> 
> > You will have seen draft-10 then draft-11 in quick succession, as
> already
> 
> > explained.
> 
> > The diffs from draft-09 to -10 were those that had built up since
> Jul'19.
> 
> > The diffs from draft-10 to -11 were solely those for the change from
> EXP
> 
> > track to STD track.
> 
> > Draft-10 doesn't seem to display in the list of links to each
> version, but
> 
> > you can manually write the URL.
> 
> >
> 
> > The main technical changes in draft-10 were numerous - many will be
> 
> > recognized from list discussion since Jul'19.
> 
> > Particular thanks to Ilpo Järvinen who identified many niggles (and
> their
> 
> > solutions) while writing and testing a full Linux implementation
> (based on
> 
> > Olivier Tilmans's, in turn based on Mirja's).
> 
> >
> 
> >  *  Allowed 2 different orders of the fields in the AccECN Option
> 
> >  *  Reflect IP-ECN field of SYN/ACK only on ACK of SYN/ACK, not also
> on
> 
> >     first data packet
> 
> >      +  greatly simplifies implementation, esp with TFO.
> 
> >      +  repeating on first data packet was for reliable delivery,
> which is
> 
> >         now achieved with ACE counter (see next bullet)
> 
> >  *  Increment the ACE counter if CE on SYN/ACK (but still not if CE
> on SYN)
> 
> >      +  Reliable delivery of feedback of CE on SYN/ACK
> 
> >  *  Redefine 'first packet' as first to arrive, not first in
> sequence in 2
> 
> >     cases:
> 
> >      +  Handshake reflection on the ACK of the SYN/ACK
> 
> >      +  In the test for zeroing of ACE
> 
> >      +  Reason: greatly simplifies implementation
> 
> >  *  if ACE could have wrapped more than once, SHOULD assume “safest
> likely
> 
> >     case”
> 
> >     not "conservatively assume" it did cycle
> 
> >      +  Reason: avoid unnecessary hit on performance
> 
> >  *  More robustness (with flexibility) in rules for when to include
> an
> 
> >     AccECN Option
> 
> >      +  Change-triggered AccECN Option as SHOULD, not MUST
> 
> >      +  SHOULD follow change-triggered AccECN Option with another
> (removes
> 
> >         ambiguity if ACK thinning or loss)
> 
> >      +  when same counter continues to increment, SHOULD
> consistently
> 
> >         include it every n ACKs
> 
> >      +  Made rule about precedence of SACK conditional (max 2 SACK
> blocks)
> 
> >      +  MAY exclude counters that have not changed for the whole
> connection
> 
> >  *  Allowed an AccECN server not to implement RFC3168 ECN (all
> clients still
> 
> >     have to)
> 
> >  *  Precluded mixed capability negotiation from either end
> 
> >      +  reduces freedom to choose SYN & SYN/ACK fall-back strategies
> 
> >      +  to prevent cases where each end's outcome after handshake
> could be
> 
> >         inconsistent (in reordering corner-cases)
> 
> >  *  Reserved the codepoint combination used by the historic nonce
> case
> 
> >  *  Merged in a number of points from RFC3168 that we hadn't covered
> 
> >      +  (a whole new subsection about obligations to do with ECN)
> 
> >  *  Explicit about checking "acceptable packets"
> 
> >      +  before counting their ECN markings or before counting the
> ECN
> 
> >         feedback they carry
> 
> >  *  Required retransmitted Fallback SYN to use same ISN
> 
> >      +  allows servers to detect ECN downgrade SYN attacks
> 
> >  *  Handled corner cases like In-window SYN during TIME-WAIT
> 
> >
> 
> >
> 
> >
> 
> > Bob
> 
> >
> 
> > On 06/03/2020 02:24, [e-mail] wrote:
> 
> >
> 
> >  
> 
> >
> 
> > A new version of I-D, draft-ietf-tcpm-accurate-ecn-11.txt
> 
> >
> 
> > has been successfully submitted by Bob Briscoe and posted to the
> 
> >
> 
> > IETF repository.
> 
> >
> 
> >  
> 
> >
> 
> > Name:            draft-ietf-tcpm-accurate-ecn
> 
> >
> 
> > Revision: 11
> 
> >
> 
> > Title:           More Accurate ECN Feedback in TCP
> 
> >
> 
> > Document date:   2020-03-05
> 
> >
> 
> > Group:           tcpm
> 
> >
> 
> > Pages:           58
> 
> >
> 
> > URL:           
> https://www.ietf.org/internet-drafts/draft-ietf-tcpm-accurat
> 
> > e-ecn-11.txt
> 
> >
> 
> > Status:        
> https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ec
> 
> > n/
> 
> >
> 
> > Htmlized:      
> https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn-11
> 
> >
> 
> > Htmlized:      
> https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accura
> 
> > te-ecn
> 
> >
> 
> > Diff:          
> https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-accurate-e
> 
> > cn-11
> 
> >
> 
> >  
> 
> >
> 
> > Abstract:
> 
> >
> 
> >    Explicit Congestion Notification (ECN) is a mechanism where
> network
> 
> >
> 
> >    nodes can mark IP packets instead of dropping them to indicate
> 
> >
> 
> >    incipient congestion to the end-points.  Receivers with an ECN-
> 
> >
> 
> >    capable transport protocol feed back this information to the
> sender.
> 
> >
> 
> >    ECN is specified for TCP in such a way that only one feedback
> signal
> 
> >
> 
> >    can be transmitted per Round-Trip Time (RTT).  Recent new TCP
> 
> >
> 
> >    mechanisms like Congestion Exposure (ConEx), Data Center TCP
> (DCTCP)
> 
> >
> 
> >    or Low Latency Low Loss Scalable Throughput (L4S) need more
> accurate
> 
> >
> 
> >    ECN feedback information whenever more than one marking is
> received
> 
> >
> 
> >    in one RTT.  This document specifies a scheme to provide more
> than
> 
> >
> 
> >    one feedback signal per RTT in the TCP header.  Given TCP header
> 
> >
> 
> >    space is scarce, it allocates a reserved header bit, that was
> 
> >
> 
> >    previously used for the ECN-Nonce which has now been declared
> 
> >
> 
> >    historic.  It also overloads the two existing ECN flags in the
> TCP
> 
> >
> 
> >    header.  The resulting extra space is exploited to feed back the
> IP-
> 
> >
> 
> >    ECN field received during the 3-way handshake as well. 
> Supplementary
> 
> >
> 
> >    feedback information can optionally be provided in a new TCP
> option,
> 
> >
> 
> >    which is never used on the TCP SYN.
> 
> >
> 
> >  
> 
> >
> 
> >        
>                                                                     
> 
> >       
> 
> >
> 
> >  
> 
> >
> 
> >  
> 
> >
> 
> > Please note that it may take a couple of minutes from the time of
> submission
> 
> >
> 
> > until the htmlized version and diff are available at tools.ietf.org.
> 
> >
> 
> >  
> 
> >
> 
> > The IETF Secretariat
> 
> >
> 
> >  
> 
> >
> 
> >  
> 
> >
> 
> >
> 
> >
> 
> > 
> 
>  
> 
> 
>