Re: [tcpm] TARR - the two options for R

[[e-mail]]

> On 5. Jul 2022, at 16:08, Carles Gomez Montenegro <[e-mail]> wrote:
> 
> Dear all,
> 
> (Resuming this thread on the two possible encoding options for the 7-bit
> field "R", that is, the ACK rate requested by the sender in the TARR draft
> [1]...)
> 
> First of all, many thanks for the opinions given so far (and special
> thanks to Jonathan for starting the thread with detailed feedback).
> 
> As a reminder:
> 
>  OPTION 1: the R field corresponds to the binary encoding of the
>  requested ACK rate. The maximum value of R is 127. A receiver MUST
>  ignore an R field with all bits set to zero. (TO-DO: if OPTION 1 is
>  selected, see how to handle all bits of R being equal to zero.)
> 
>  OPTION 2: the R field is composed of two subfields: the 5 leftmost
>  bits represent a mantissa (m) and the 2 rightmost bits represent an
>  exponent (e). The value of the requested ACK rate is obtained as R =
>  (m+1)*2^(2*e). The maximum value of R is 2048.
> 
> In addition to the input below, there are also opinions logged in the IETF
> 113 TCPM session minutes:
> 
> ----------------
>  Yoshi:
>     Option 2 - I don't know the use case for such a large value (1024).
> 
>  Bob:
>     inaudible -> list
> 
>  Gorry:
>     I favor option 1, I don't know any value larger than 64 can be
>     safe.
> ----------------
> 
> Bob had also written a detailed message motivating the need for what we
> are calling Option 2:
> https://mailarchive.ietf.org/arch/msg/tcpm/TIStAhSjsYZ8PLHWkbuJ8Bmx8so/
> 
> And Michael Scharf had asked "Do we really expect values larger than 63
> (or 63) for "R"...?:
> https://mailarchive.ietf.org/arch/msg/tcpm/K8_dlkzXYCa6Xi0HxbCpgyPyCT8/
> 
> Are there other opinions?
Hi Carles,

I would like to give some feedback as an individual.

This feedback is based on using your draft in one of my courses
at university resulting in a prototype implementation for FreeBSD
and packetdrill.

Here are some points:

1. You might want to say which kind (253 or 254) is used on the
   sender side and on the receiver side. The prototype accepts
   both on the receiver side and uses only one on the sender side.
   Which one is used on the sender side is compiled into the code.

2. Regarding the encoding of R: I prefer option 1.
   This allows to use the value of 0 to be used to signal that
   an ACK for this segment should be sent immediately, but no
   persistent change of the ACK ratio should be performed.

3. If the range 1 - 127 is considered too small, one might consider
   using two bytes for the option value. Currently, the length of
   the option is odd, and therefore TCP implementations might add
   some padding (NOP) anyway, since other options used after the
   connection establishment like the timestamp or the sack option
   have an even length.

4. If SYN cookies are used and one is not willing to not spend another
   bit for representing TARR support, you might want to specify that the
   client sends the TARR option on the ACK of the three way handshake.
   That way a server willing to support the TARR option can "learn"
   from the client that it is OK to send the TARR option by observing
   that the client sends it. It can also be learned later...

Best regards
Michael
> 
> Many thanks,
> 
> Carles
> 
> [1]
> https://datatracker.ietf.org/doc/html/draft-gomez-tcpm-ack-rate-request-04
> 
> 
>> Hello,
>> 
>> +1 for Option 2 with a larger max value. We already have 800 Gbps
>> links available off-the-shelf and speeds won't go down in the future.
>> 
>> --
>>  Juhamatti
>> 
>> On Wed, 30 Mar 2022 at 22:18, Jonathan Morton <[e-mail]>
>> wrote:
>>> 
>>> During IETF-113, it was asked which of the two encoding options for the
>>> R value should be chosen, and I suggested that there were arguments in
>>> favour of both, with discussion to follow here.  The text in -04 is as
>>> follows:
>>> 
>>>> R: The size of this field is 7 bits. The field carries the ACK rate
>>> requested by the sender. The minimum value of R is 1.
>>>> 
>>>> Note: there are currently two options being considered regarding the
>>> semantics of the R field:
>>>> 
>>>> OPTION 1: the R field corresponds to the binary encoding of the
>>> requested ACK rate. The maximum value of R is 127. A receiver MUST
>>> ignore an R field with all bits set to zero. (TO-DO: if OPTION 1 is
>>> selected, see how to handle all bits of R being equal to zero.)
>>>> 
>>>> OPTION 2: the R field is composed of two subfields: the 5 leftmost
>>> bits represent a mantissa (m) and the 2 rightmost bits represent an
>>> exponent (e). The value of the requested ACK rate is obtained as R =
>>> (m+1)*2^(2*e). The maximum value of R is 2048.
>>> 
>>> It was further suggested that in Option 1, the value could be encoded as
>>> offset-1, so that (for the 7-bit field described) a range of 1-128
>>> inclusive was available, and an encoding of all-zeroes would be
>>> interpreted as R=1.  I'll assume this suggestion is adopted for this
>>> discussion.
>>> 
>>> The main argument in favour of Option 1 is that it is very easy for the
>>> sender to encode and the receiver to decode, reducing the likelihood of
>>> implementation errors.  The maximum R value of 128 already represents a
>>> 64-fold improvement in the status quo, with respect to the number of
>>> acknowledgement packets generated by a conforming receiver.  It also
>>> allows for the opposite case of disabling Delayed Acks, for those
>>> situations where it's needed or desirable, in the simplest reasonable
>>> manner.
>>> 
>>> Option 2 is slightly more complex to encode and decode, but allows for a
>>> wider range of R values to be represented, still as low as 1, but now as
>>> high as 2048, with granularity decreasing as R increases beyond 32.  A
>>> question was raised as to whether an R value this high was reasonable,
>>> or whether the lower cap of Option 1 would be sufficient for current and
>>> future needs.
>>> 
>>> In today's Internet, link speeds of 1Gbps are commonplace (especially on
>>> LAN but also increasingly on WAN), and under suitable conditions even a
>>> single TCP flow could use it all - with current research going into
>>> ensuring that these "suitable conditions" are also normally encountered.
>>> I will therefore use this speed as my benchmark for illustrative
>>> purposes, with the understanding that many links are slower today, and
>>> conversely will probably be faster still in the future.  For the latter
>>> case, I will also consider the theoretical limits of a TCP connection.
>>> 
>>> The standard Ethernet frame size limits an IP packet to 1500 bytes on
>>> most networks.  At 1Gbps (and neglecting various overheads), this allows
>>> for 83333 data packets per second at the maximum frame size.  Under
>>> standard Delayed Ack rules (ie. R=2), this results in 41667 ack packets
>>> per second travelling the other way, or almost one every 25
>>> microseconds.  This imposes a significant load on the network's ability
>>> to handle individual packets, and also imposes a significant processing
>>> burden on both of the TCP endpoints.
>>> 
>>> On a shared-medium link, many of these ack packets will be enqueued
>>> awaiting access to the medium, which could take several milliseconds.
>>> This could result in as many as a thousand ack packets being released in
>>> a line-rate burst.  Assuming ack decimation does not occur, this burst
>>> will tend to persist all the way back to the sender.  Typically, all but
>>> the final ack in the burst yields only redundant information, which will
>>> be superseded before the next data segment can be prepared and sent.
>>> Ultimately, it is a waste of energy to even emit those redundant acks in
>>> the first place.
>>> 
>>> With R=128, the density of ack packets under the same conditions reduces
>>> to 651 per second, or about one every millisecond and a half.  This is
>>> clearly a major and worthwhile improvement in efficiency.  If we limited
>>> the discussion to Ethernet LANs, an R value even this high would not be
>>> required at 1Gbps, since the RTT is typically below 1ms in such
>>> conditions.  However, Internet paths are typically much longer, and
>>> R=128 is clearly justified as soon as a 1Gbps TCP flow goes beyond the
>>> LAN, with the RTT exceeding 6ms (using the rule of thumb that at least 4
>>> acks per RTT are needed to assure smooth delivery).
>>> 
>>> But the question is whether R values *exceeding* 128 are justifiable.
>>> At a "typical" Internet RTT of 100ms, we would like to have an ack every
>>> 25ms due to the rule of thumb, and this is achieved by R in the vicinity
>>> of 1000.  Longer RTTs (eg. Eastern Europe to US Pacific Coast) might
>>> sometimes justify even higher R values.  Since TARR's option 2 has a
>>> maximum representable R of 2048, it is a good match for this scenario.
>>> 
>>> We can also sanity-check R=2048 by comparing it to an extreme case in
>>> the IP and TCP specifications.  IP packets (and thus TCP segments) can
>>> be as large as 64KB (2^16), and the TCP receive window can be as large
>>> as 1GB (2^30).  In this case, the window is approximately 2^14 (16K)
>>> segments.  Assuming the congestion window actually grows to fill the
>>> receive window (which would imply an LFN par excellence), R=2048 would
>>> produce 8 acks per RTT, which is a reasonable correspondence to the rule
>>> of thumb.  In these extreme conditions, that would be one ack per 128MB
>>> of data.
>>> 
>>> The LFN in the room is that stretch-acks are known to promote burstiness
>>> in TCP senders, and *either* option - R=128 or R=2048 - would make
>>> stretch-acks the norm rather than the exception.  Hence TARR should
>>> RECOMMEND that conforming senders use a burst mitigation mechanism such
>>> as TCP Pacing, commensurate with the R values emitted.
>>> 
>>> - Jonathan Morton
>>> _______________________________________________
>>> tcpm mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/tcpm
>> 
>> _______________________________________________
>> tcpm mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/tcpm
>> 
> 
> 
> _______________________________________________
> tcpm mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/tcpm