Re: [tcpm] Does this fix sufficiently de-ossify overstrict ECN negotiation by Linux TCP servers?

[Bob Briscoe <[e-mail]>]

Brian,
I've asked - I'll let you know once I hear.

Bob

On 16/03/2019 14:32, Brian Trammell (IETF) wrote:
> Hi Bob,
>
> This seems eminently reasonable. I’m curious, would there be 
> equivalent pushback on _only_ adding the SNMP counter? One could 
> always use observation to debug once the counter starts going up...
>
> Cheers,
>
> Brian
>
> Sent from my iPhone
>
> On 16 Mar 2019, at 12:38, Bob Briscoe <[e-mail] 
> <mailto:[e-mail]>> wrote:
>
>> Folks,
>>
>> After offline discussion with the Linux maintainers, we have agreed 
>> on option #2, which is to make overstrict ECN test more specific to 
>> RFC3168 by only disabling ECN if a SYN has a non-zero IP-ECN field 
>> and the first six TCP flags in the main header are 0b000011.
>>
>> This will be a simple 1-line patch that maximizes the likelihood that 
>> it will be back-ported as far back as possible, hopefully back to the 
>> introduction of the over-strict test in May 2012. This should fix a 
>> large proportion of the installed base of Linux servers reasonably 
>> quickly.
>>
>> ==Logging==
>> Separately we will submit a patch to add logging of each event that 
>> disables ECN, but only for future kernel versions, not for 
>> back-porting. However, there is resistance to any logging, so this 
>> might not be accepted. The proposal is
>> * at default logging level, increment an SNMP count when ECN disabled 
>> for a connection due to a protocol violation
>> * at increased logging level, on each event also write rate-limited 
>> kernel messages that record the client IP.
>>
>> Then, if a sysadmin wants to, they can get any network mangling fixed.
>>
>>
>> Bob
>>
>> On 20/01/2019 23:31, Bob Briscoe wrote:
>>> TCPM folks,
>>>
>>> RFC8311 and draft-ietf-tcpm-generalized-ecn enable ECN-capable SYNs. 
>>> However, if a client sends an ECN-capable SYN, existing Linux 
>>> servers disable ECN for the rest of the connection.
>>>
>>> To fix as many existing Linux servers as possible, as soon as 
>>> possible, I would recommend short-term solution #2 (see 'Three 
>>> Fixes' below). An alternative would be short-term solution #3.
>>>
>>> Anyone got a better fix? Anyone disagree with recommending #2? If 
>>> not, pls confirm your agreement.
>>>
>>>
>>>       Summary of Pros and Cons
>>>
>>> #2 pro: No longer ossifies non-ECN SYN
>>> #2 pro: If a path does "unbleach" IP-ECN, protects the connection by 
>>> disabling ECN
>>> #2 con: A residual degree of ossification remains
>>>
>>> #3 pro: No longer ossifies non-ECN SYN
>>> #3 pro: No ossification at all
>>> #3 con: If a path does "unbleach" IP-ECN, connection still attempts 
>>> to use ECN even though the path is mangling the ECN field.
>>>
>>> #1 is the full solution, but it should proceed in parallel as a 
>>> solution for the future, not to fix the past.
>>>
>>>
>>>       Background
>>>
>>> To negotiate ECN for a connection the client sets flag bits 8-9 in 
>>> the TCP header to 11 [RFC3168 Section 6.1.1] 
>>> <https://tools.ietf.org/html/rfc3168#section-6.1.1>. For your 
>>> convenience, I've included a picture of the TCP header flags at the end.
>>>
>>> In 2012 an additional test was patched into Linux servers. It 
>>> disables ECN for the connection unless the IP-ECN field of the SYN 
>>> is also zero (I'll call this the "overstrict ECN test").
>>>
>>> This was because Section 6.1.1. of RFC3168 also said "A host MUST 
>>> NOT set ECT on SYN or SYN-ACK packets." Unfortunately, the 
>>> overstrict ECN test ossifies this requirement into TCP.
>>>
>>> The Linux patch was intended to protect a connection against the 
>>> network mangling the IP-ECN field to a non-zero value. 
>>> Unfortunately, it also disables ECN completely if a client tries to 
>>> set ECT in the IP-ECN field of the SYN.
>>>
>>> RFC8311 has now updated RFC3168 to allow ECT on a SYN, in 
>>> conjunction with the ECN++ experiment. But few developers are going 
>>> to set ECT on a SYN if it causes about 84% of servers to completely 
>>> disable ECN (see Section 4.2.2. of the ECN++ draft 
>>> <https://tools.ietf.org/html/draft-ietf-tcpm-generalized-ecn-03#section-4.2.2.2> 
>>> for further details and the source of these numbers).
>>>
>>>
>>>       Three Fixes
>>>
>>> I can think of three ways to fix existing Linux servers:
>>>
>>>  1. Deploy Accurate ECN
>>>     <https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn> on
>>>     existing Linux servers.
>>>       * AccECN inherently replaces the overstrict ECN test with a
>>>         pair of proper two-ended tests for network mangling, in
>>>         which each end feeds back the arriving IP ECN field in its
>>>         TCP feedback. In contrast, the overstrict ECN test is
>>>         one-ended, with the server only assuming what the client set.
>>>       * Altho AccECN will solve the problem, it should not be relied
>>>         on to fix the last six years of Linux server deployment.
>>>         AccECN is a non-trivial update so it is unlikely to get
>>>         back-ported and auto-deployed to most existing Linux servers
>>>         any time soon.
>>>  2. Make the overstrict ECN test more specific to RFC3168, and add
>>>     logging.
>>>       * Currently, the only proposal for allowing ECT on a SYN
>>>         requires the client to also request Accurate ECN feedback,
>>>         which requires TCP header bits 7-9 to be 111.
>>>       * Therefore, the overstrict ECN test in a Linux server could
>>>         be altered to solely apply if bits 7-9 == 011 (rather than
>>>         just testing if bits 8-9 == 11).
>>>       * However, I am concerned that this ossifies around the
>>>         solution based on ECN++ and AccECN. What if ECN++ needs to
>>>         be superseded by some future protocol that allows ECT on a
>>>         SYN when some other field is different (e.g. one of TCP's
>>>         three remaining reserved flags, or a new TCP option)?
>>>       * Unfortunately, we cannot test for the absence of a
>>>         yet-to-be-invented TCP option. Nonetheless, I suggest *the
>>>         overstrict ECN test in a Linux server should at least be
>>>         altered to solely apply if TCP header bits 4-9 == 000011*.
>>>       * I suggest Linux also *logs whenever it disables ECN due to
>>>         this test*, so that broken paths can be identified and fixed.
>>>       * An extremely simple patch like this would be much more
>>>         likely than #1 (AccECN) to be back-ported and deployed to
>>>         most existing servers via regular Linux auto-updates.
>>>  3. As #2, but do not disable ECN when the test fails, just log
>>>     those paths suspected of mangling ECN.
>>>       * Our most recent measurements of millions of Internet paths
>>>         found no occurrences where zero IP-ECN on a SYN was changed
>>>         to non-zero ("unbleaching"). So we could just remove the test.
>>>       * However, no occurrences found does not mean none exist.
>>>         Altho we tested millions of paths, we only tested from a few
>>>         dozen vantage points, albeit we did test both mobile and
>>>         fixed networks (see Section 4.2.2. of the ECN++ draft
>>>         <https://tools.ietf.org/html/draft-ietf-tcpm-generalized-ecn-03#section-4.2.2.2>
>>>         for summary of tests).
>>>       * If the number of problematic paths is small, it would still
>>>         make sense to run the ECN test as proposed under item #2,
>>>         but not disable ECN if it fails. Instead, just log any
>>>         negative results, so the broken network elements can be
>>>         identified and fixed.
>>>
>>> Cheers
>>>
>>>
>>> Bob
>>>
>>> PS. For convenience, here's the TCP header flags (assuming the 
>>> experimental AccECN protocol is approved as RFC):
>>>         0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15
>>>       +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
>>>       |               |           | A | C | E | U | A | P | R | S | F |
>>>       | Header Length | Reserved  | E | W | C | R | C | S | S | Y | I |
>>>       |               |           |   | R | E | G | K | H | T | N | N |
>>>       +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
>>>
>>>
>>> -- 
>>> ________________________________________________________________
>>> Bob Briscoehttp://bobbriscoe.net/
>>
>> -- 
>> ________________________________________________________________
>> Bob Briscoehttp://bobbriscoe.net/
>> _______________________________________________
>> tcpm mailing list
>> [e-mail] <mailto:[e-mail]>
>> https://www.ietf.org/mailman/listinfo/tcpm

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/