Re: [tcpm] Comments to draft-ietf-tcpm-tcp-edo-13

"D. Wythe" <alibuda@linux.alibaba.com> Thu, 23 November 2023 06:25 UTC

Return-Path: <alibuda@linux.alibaba.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0A7ACC151073 for <tcpm@ietfa.amsl.com>; Wed, 22 Nov 2023 22:25:29 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.998
X-Spam-Level:
X-Spam-Status: No, score=-9.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, ENV_AND_HDR_SPF_MATCH=-0.5, NICE_REPLY_A=-0.091, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, UNPARSEABLE_RELAY=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zDfUn0A2-F4b for <tcpm@ietfa.amsl.com>; Wed, 22 Nov 2023 22:25:25 -0800 (PST)
Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C2CA3C151066 for <tcpm@ietf.org>; Wed, 22 Nov 2023 22:25:23 -0800 (PST)
X-Alimail-AntiSpam: AC=PASS; BC=-1|-1; BR=01201311R171e4; CH=green; DM=||false|; DS=||; FP=0|-1|-1|-1|0|-1|-1|-1; HT=ay29a033018045192; MF=alibuda@linux.alibaba.com; NM=1; PH=DS; RN=2; SR=0; TI=SMTPD_---0VwyJGsO_1700720718;
Received: from 30.221.149.60(mailfrom:alibuda@linux.alibaba.com fp:SMTPD_---0VwyJGsO_1700720718) by smtp.aliyun-inc.com; Thu, 23 Nov 2023 14:25:19 +0800
Message-ID: <0c8c3062-1efd-fc7c-2497-49a99f8f0b9d@linux.alibaba.com>
Date: Thu, 23 Nov 2023 14:25:17 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.15.1
Content-Language: en-US
To: Yoshifumi Nishida <nsd.ietf@gmail.com>
Cc: tcpm@ietf.org
References: <fc2d032a-eed3-536a-7131-f4c8ba697136@linux.alibaba.com> <CAAK044QNA9dUw=yfPLkRDOFZ58q7vFhbSKNNNADDFNayoZsuUg@mail.gmail.com>
From: "D. Wythe" <alibuda@linux.alibaba.com>
In-Reply-To: <CAAK044QNA9dUw=yfPLkRDOFZ58q7vFhbSKNNNADDFNayoZsuUg@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"; format="flowed"
Content-Transfer-Encoding: 8bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/G8D8skHigPX6DVMXnBmMwltFzhc>
Subject: Re: [tcpm] Comments to draft-ietf-tcpm-tcp-edo-13
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Nov 2023 06:25:29 -0000

Hi Yoshi,

Thanks for you comments.

On 11/23/23 3:26 AM, Yoshifumi Nishida wrote:
> Hi Wythe,
>
> I'm not the author of the draft. But, there are the following texts in 
> the draft.
>
> "Once enabled on a connection, all segments in both directions MUST 
> include the EDO Extension option."
> "If EDO has been negotiated, any subsequent segments arriving without 
> the EDO Extension option MUST be silently ignored."
>
> So, I think you'll probably see connection timeouts in such cases. 
> This may look suboptimal behavior, but at the same time, we probably 
> don't want to think about buggy implementations too much because it 
> can be endless. The behavior you describe is prohibited by RFC9293.

I believe that in that case, connections may not end with timeouts but 
instead receive an erroneous application
response from the server. This is because the server does not support 
EDO, and an incorrectly implemented middlebox
copies the EDO support option, which misleading the client.

Consequently, a legacy server might interpret the TCP option as 
application data.

> BTW, I'm curious about how frequently we see this kind of behavior.

Previously, I had the belief that this might was a widespread scenario, 
but as I conducted further investigation,
I realized that this issue may only exist in specific organizations or 
regions..

We are now convinced that this issue is caused by a type of incorrect 
syn-proxy implementation, who
acts as an intermediary to protect against SYN flood attacks by 
completing TCP handshakes on behalf of clients,
preventing server overload.

The problem is that when the syn-proxy receives a syn from the client, 
in order to avoid memory allocation and copying,
some implementations reuse the SYN packets, exchange IP addresses and 
ports, and send it out as SYN/ACK to client.
Unfortunately, it did not consider the impact of TCP options being 
simply copied.

A example I found is:

https://github.com/iqiyi/dpvs/blob/master/src/ipvs/ip_vs_synproxy.c

 From this perspective, it is worth considering that these issues may 
only affect certain organizations and regions.
I agree with your point about buggy implementations, as these types of 
issues can be never-ending and are
beyond the protocol's control or responsibility to handle.

Best wishes,
D. Wythe

> --
> Yoshi
>
>
> On Wed, Nov 22, 2023 at 12:28 AM D. Wythe <alibuda@linux.alibaba.com> 
> wrote:
> >
> >
> > Hi all,
> >
> > I am D. Wythe from Alibaba, currently conducting research on EDO. I
> > would like to report an issue
> > involving certain middleboxes that may have an impact on EDO.
> >
> > We have observed that some middleboxes incorrectly copy unknown TCP
> > options from SYN packets into the SYN-ACK packets.
> > We believe this behavior is entirely incorrect, although we are
> > uncertain of the reasons behind it.
> >
> > Here is an example of a TCP SYN packet I constructed with an unknown SYN
> > TCP option:
> >
> > local.32218 > remote.http: Flags [S], seq 4026107292, win 8192, options
> > [unknown-100 0x01020304,nop,nop], length 0
> > remote.http > local.32218: Flags [S.], seq 3220955308, ack 4026107293,
> > win 8192, options [unknown-100 0x01020304,nop,nop], length 0
> >
> > Unfortunately, this is not an isolated incident. We have observed
> > several publicly available services on the internet
> > exhibiting the same incorrect behavior.
> >
> > We would like to know if anyone else has encountered similar issues and
> > how you perceive the impact of EDO in this scenario
> > (we believe it may result in the server interpreting the TCP options as
> > application data).
> >
> > Additionally, is there any possibility of adding an explicit
> > notification when one side detects a violation of the EDO protocol.
> > This could potentially prevent all segments from being discarded by
> > restricting the peer from further extending the TCP header.
> >
> >
> > Best regards,
> > D. Wythe
> >
> >
> >
> >
> > _______________________________________________
> > tcpm mailing list
> > tcpm@ietf.org
> > https://www.ietf.org/mailman/listinfo/tcpm