RE: ECN in QUIC

[Ingemar Johansson S <[e-mail]>]

Hi

Comment on “CE is not going to be a very common occurrence”.
While that may be true for (almost) drop equivalent ECN (a.k.a. classic), it does unfortunately not hold at all for L4S. Typical values of ECN marking ratio with L4S is in the order of 10 to 50%. In addition the CE marks can be spread randomly (for instance with phantom queues).
This means that the ACK blocks in the suggested format below can become very short, in the worst case representing only one packet per ACK block if we have an alternating ECT-CE-ECT-CE… pattern in the received packets.

So my take is that if an ECN bit pattern is really necessary then it is better to have it as an bitmap that represents all packets that are listed in the ACK block. If one make an assumption that an ACK frame contains two ACK blocks that ACKs 5 and 7 packets then the ECN block will become (5+7)*2 = 24bits , i.e. 3 octets.

But with that said I am still not convinced that the packet counter approach is flawed (I’m am happy to be enlightened 😊) .
Taking multipath as an example : Isn’t it the case that two connections are setup, each with its own congestion control context (I omit coupled CC for the moment). If that is the case then I don’t see any problem with the packet counters.
As regards to connection migration. One possible matter is that it can become tricky to detect ECN bleaching or ECN bit access problems in OS immediately , however during the course of 1 RTT or even less, this should be possible to detect as e.g. ECN bleaching will result in that the neither on the ECT(0),ECT(1) and CE counters increase. But I am fully aware that I may miss things here.


/Ingemar



From: Lubashev, Igor [mailto:[e-mail]]
Sent: den 24 januari 2018 19:02
To: Ingemar Johansson S <[e-mail]>; QUIC WG <[e-mail]>; huitema ([e-mail]) <[e-mail]>
Subject: RE: ECN in QUIC

I believe the primary concern during the meeting yesterday was the ECN feedback format (counters).  Here is the alternative, similar to what Christian voiced.

I am making these assumptions:

  *   CE is not going to be a very common occurrence
  *   The endpoint will usually either send (almost) all packets with no ECT markings, or it will send (almost) all packets with a specific ECT (ECT[0] or ECT[1]).
None of these assumptions are required.  They are just used to identify “common cases” and save a few bytes per ACK frame.  We may discard these assumptions and the optimizations.
The worst case for this proposal is a case when ECT[0] and ECT[1] marking alternate, because that prevents acknowledging consecutive packets as a block.  My inclination is to direct the sender to just not do this.



Part 1. Primary ECT
--------------------------
During the handshake, each endpoint notifies the other of the ECT codepoint (ECT[0] or ECT[1]) that is intends to use primarily.  This is known the Primary ECT.  By default, ECT[0] is the Primary ECT.


Part 2.  Two ACK varieties
----------------------------------

In addition to ACK Frame, we introduce a new ACK-ECT frame.  The contents of the frames are identical and unchanged from the current ACK frame contents.
All packets acknowledged by the ACK Frame are packets that have no ECN bits set at all.  All packets acknowledged by the ACK-ECT Frame are packets that have Primary ECT code point.  It is expected that in the common case, all packets will be acknowledged by ACK or ACK-ECT.


Part 3.  ACK-ECN-BITMAP
----------------------------------

The new ACK-ECN-BITMAP frame is used when ACK or ACK-ECT frames cannot be used.  Its format is very similar to the format of the ACK frame, except there is a new ECN Bitmap field.

~~~
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                     Largest Acknowledged (i)                ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          ACK Delay (i)                      ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       ACK Block Count (i)                   ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          ECN Bitmap (*)                     ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          ACK Blocks (*)                     ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~~~
{: #ack-ecn-bitmap format title="ACK-ECN-BITMAP Frame Format"}

The fields in the ACK-ECN-BITMAP frame are as follows:

Largest Acknowledged:  (SAME AS IN ACK FRAME)

ACK Delay:  (SAME AS IN ACK FRAME)

ACK Block Count:
SAME AS IN ACK FRAME, except packets in each ACK Block MUST have the same ECN marking.  This may cause blocks of consecutive packets to be split into multiple ACK Blocks (with Gap=0 in between), if those packets have different ECN markings.

ECN Bitmap:
A sequence of ACK Block Count pairs of bits that indicate ECN markings for the corresponding ACK Block.  The size of the ECN Bitmap field is Ceiling((1+“ACK Block Count”)/4) octets.  The unused bits should be set to 0.
For example:
~~~
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|AA|BB|CC|DD|…
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Bits AA represent ECN bits for packets acknowledged in the First ACK Block.
Bits BB represent ECN bits for packets acknowledged in Additional ACK Block 0.
Bits CC represent ECN bits for packets acknowledged in Additional ACK Block 1
…

ACK Blocks: (SAME AS IN ACK FRAME)



  *   Igor


From: Ingemar Johansson S [mailto:[e-mail]]
Sent: Wednesday, January 24, 2018 3:01 AM
To: QUIC WG <[e-mail]<mailto:[e-mail]>>
Subject: ECN in QUIC

Hi

A few impressions on the ECN in QUIC discussions. The audio was so-so, frequent interruptions and LARGE differences in audio level + badly implemented ears 😊, so sometimes I missed the details and possibly kick in open doors.

*  Feedback format. I understand that the proposed proposal based on ECT(0), ECT(1) and CE counters does not play well with connection migration. I believe, but I am not 100% sure that I understand the problem. I understand that there are two concerns (perhaps more?):
  1. Counters are reset, this means that there will be a potentially very large (negative?) difference in CE marked packets. I believe that this can be detected as false congestion signal and ignored by the congestion signal.
  2. Difficulties to detect ECN bleaching. It is tricky to understand if packets along the new path are bleached based on the ECT counters. This can be a more complicated case.
In any case it was suggested to indicate the ECN bits for the packets in the ACK frame. Such a format was actually outlined in earlier proposal (see page 6, Figure 3 in https://tools.ietf.org/html/draft-johansson-quic-ecn-01<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Djohansson-2Dquic-2Decn-2D01&d=DwMGaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=xVzIA9f7WjK8CVIQ3B-v9ZNl-Uf8HCY4nYPb0fMRO-4&s=S9aqc8UIGaTDX3ZMs1VTyWiLSSG01ndFhhr_Zg01H1Y&e=> ). It is Ok with me to change to a format similar to this if the group prefers that.

* Congestion control for ECT(1), L4S : My feeling is that it is for now OK to leave this as future work. That does not mean that there are absolutely no ideas on how to implement this, for short RTT environments DCTCP style congestion control is quite OK. Furthermore I and recently worked on a L4S congestion control that is based on BBR, the results look quite promising and the changes to the BBR algorithm are quite minimal. I am willing to present this at ICCRG in London if there is any interest in this.
* ECN in QUIC v1 or later : I would very much prefer that ECN support goes into v1. On the technical side it makes it possible to experiment with ECN, potentially on a larger scale. On the political side it helps to make the “chicken and egg” problem become history.

/Ingemar
==================================
Ingemar Johansson  M.Sc.
Master Researcher

Ericsson Research
Network Protocols & E2E Performance
Labratoriegränd 11
971 28, Luleå, Sweden
Phone [phone]
SMS/MMS [phone]
[e-mail]<mailto:[e-mail]>
www.ericsson.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ericsson.com&d=DwMGaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=xVzIA9f7WjK8CVIQ3B-v9ZNl-Uf8HCY4nYPb0fMRO-4&s=tySiMlgTyIT8IJmaKYS-yMYBNj0YUw_M246hX4szKZ4&e=>

The world is full of magical things patiently
    waiting for our wits to grow sharper
               Bertrand Russell
==================================