Re: [iccrg] [nwcrg] Disadvantages of TCP connection splitters

"" <> Sun, 12 January 2020 08:19 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 09B1D12003E for <>; Sun, 12 Jan 2020 00:19:14 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id EvSlLHfx8848 for <>; Sun, 12 Jan 2020 00:19:09 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 8EB4E1200C7 for <>; Sun, 12 Jan 2020 00:19:08 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=s2048; t=1578817146; bh=BF5ggLHKuWlwZe2ULG1u6Fko+HkEUwXjLrkO/yVVyfM=; h=Date:From:To:Cc:In-Reply-To:References:Subject:From:Subject; b=jQEw6K+fFTap4+XIzNVoZr2SKBkjKqHhVyHHwKOngSJO0lNZOvgAPRiNTjWBny6YC1xdKW4eWa7r6OzfbcpuYSUJx8fO122MJT5/jBAyCAhREXRHEYPRC8rELmoFDmJdtSaqZyBuK5dhVZU2LmOPs9+Kg2nqKIeYKjIyjreyJdwYrx2VXP36c7pqXz+XDkvzYfrLBrAlVSGmNtuFMUTWkcG/R2kqPoQqoWXKxFvxAqlLHMefWI4jXG+QVwvlV7TUcpFjp4AaPQPk7ijSySNd93ecIGngxJ0V+f1yqo02CW6s25A1zXUDNMBEuFY+LbnXBCIYJKm6kcNHt9KJ8SdkCQ==
X-YMail-OSG: KvkUu_AVM1nTCO9El5gju7P4eNusgtXMHDxEQWCdxKDHR_zSh4qn76yqeOktnTk fYDyVDUxwA5qkseOauDvBABM7gd6ryFwRsiP3cc1vIFU0TNepGzCHah7NnWTtg6ptV8grsyiAG5O gAuY2_NU9dUvLVTU7aTT7pfDDjcVVOyGt8iDgO9D4SGuhaUeFtRiUEbOhU3D_tGJ15npqXKfnhym KGc1yiP77c_7e1ctX2VALZyDi32dVV.JC1TA83lSDoSOrSOuYLvMZSWCR4BpEthxz5zmCebq3i7_ WvLHUGGxGYL8Q9iWsXtjsmOiDbA92T637VfJxDjgVqaI2.AuUo_9afEDe_vU2toH2nbMulCK4S4Z OHvx79T.Whb8uc5Ih08o05gervHdb.DkWKGrj7C8jCKqblNHQMQ7O5tBS0lsjLnm2kr28gxo.6R6 owxonudOO2_tAzFsEqnQSbKnhob7BiO8Rg6xdc7X2_EOVf_mmxusrOLqgG4ypwT5_Fs7Fye493CN oCt85ak8mQOlEA7HoOwVJ_zM8AIDMujCHIGdH0Z4l882pCCGY9Us9tXJx3FFeFMKrRXlPM8kFdN6 7_7QxrNqsR7WjXiFjBXYsz_NymVgWvnYbzEu14t0b0XAAY9XMLE1ygQ2Ptv14COOh2qeb8AMhg_s 6d7Fmmm68I5Un_oVH.NAg9j13V4YKIjLC8KCFU9149YGGgqc5YFK0NptaEA8mxjL9m9lrM.KBL5j TVdUU0nyeJQF4m0RqKmVsPQQfnLNhT0RHohpAEGu6zz9KlUiG8CshhwW1ZSCwAMhybqdRLqJ6jsq eO_PbMRSsJuKKQLuJy7OoGYDA7ARYGUDhkcpM_CrWZBaR.xcbJXJwfjLyYWGCKXm_hrawzCk5hqD HHYoDinbOYyU4M0zce9pP5wjinpt3skD5UoI8lVTV_NB_hDpSbidb1E72rboiS1Vxz30G8lwip0Y f_paWuDHn7oQJV7d2ACJ0dsJywH7ih5_Oxc3tg33DdzPG1r8xlNk7oAXX5fw39P9_bWafVabzoiO DxL1F16Gmw9M5oIlXdoiD3gRerNUAXg1d67n6Jz_XPVg914baaCpkgShxJeRyfyFVkd.8iQRF67e nnqx3QiQYqjWVtxaB_FGVfWegiD40qvrnXl3iCHigUjYjilY41L75Mr8sEEgAXYsar2NgocPStqk 62t5XO50.mWjD8TT6xWq3QQURNXj.7sLqT7Xa1BE5J6SBPOxDlfKaVuaLGni01i8WHKC.qloL96F raJHj..n9Snd.Yw2B9sn.0J_lGqM9VYX94aBoDdGmqmRfVyl.1wLzUNCnrfKpOJrDS708GLJc20Z aRIxczgfcFrR5xGDE2OSeWsThgas5ttpygCfrJv7i4mQHfg--
Received: from by with HTTP; Sun, 12 Jan 2020 08:19:06 +0000
Date: Sun, 12 Jan 2020 08:19:03 +0000 (UTC)
From: "" <>
To: Yuchung Cheng <>, Michael Welzl <>, Marie-Jose Montpetit <>
Cc: "" <>, iccrg IRTF list <>, Keith Winstein <>
Message-ID: <>
In-Reply-To: <>
References: <> <> <> <> <>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----=_Part_16648771_1417578415.1578817143111"
X-Mailer: WebService/1.1.14873 YMailNorrin Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0
Archived-At: <>
Subject: Re: [iccrg] [nwcrg] Disadvantages of TCP connection splitters
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussions of Internet Congestion Control Research Group \(ICCRG\)" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 12 Jan 2020 08:19:14 -0000

My take is that transparent TCP interception and optimisation(window sizing and buffering) will continue. It's usefulfor remote sites including over geostationary communications,and after decades of experience some of the commercialaccelerators are... well, pretty good at it. Transportacceleration is here to stay.

But above that, *transparent* application acceleration (read:interception, and 'helpful we know better, really' rewriting)will simply go away thanks to the increased use of transportsecurity that means it doesn't get used as much -- if at all.

Application acceleration is simply not worth the effort,and the interactions that remain (bugs, "secure" websitescalling non-secure website content and getting surprisedwith race conditions when it turns up at the wrong time)are too hard to debug and are increasingly corner cases --bad/unexpected website coding/behaviour meets bad/unexpectedPEP coding/behaviour over bad (insecure) connections, butthe number of insecure connections is decreasing, anddebugging this is not where you want to spend time.

We're looking at disabling application-level accelerationfor that reason. Vendor doesn't have to maintain theircode, we don't have to troubleshoot and test their code,most users don't even notice.

However, the use case that I can see remaining is thereturn of the non-transparent explicit proxy doingdeliberate caching and filtering, as opposedto the transparency we (don't) see everywherewith e.g. Akamai, content caching and local datafarms etc.

A practical example: business on remote islandconnected via geo satellite gets all its emailthrough an accelerator, but it's 99% spam thatthe accelerator is just delivering faster,eating the link capacity, so a mail proxy,filter and forwarder is set up on the otherside of the link, explicitly breaking theend-to-end path and allowing transportacceleration in cleanly too where bothends of the TCP connection are undercontrol. That's leveraging SMTP the wayit was designed, too.

Hypothetical thought: the return of visible
explicit web caching, but caching anddelivering signed assets to decrease tampering.

And there might be some recommendations documentto write around such use cases and how tohave applications allow and authorise offloadedfunctions into a proxy to do as much processingas possible before traffic hits the constrainedlink.

But there's a clear distinction to be madebetween layer-4 TCP stuff and higher-layerstuff.


Lloyd Wood 

    On Saturday, 11 January 2020, 10:41:32 GMT+11, Marie-Jose Montpetit <> wrote:  
 (NWCRG chair off) We did peps years ago when satellite networks were impacted by TCP - it used to be our main sales pitch for Teledesic that we did not need it. But of course other networks needed them for like was all mentioned in this thread.
Open source PEPs now are really rudimentary and application layer PEPs like in "I am an app needing fast ACK” are more or less non existent. 
What should we do to make progress? 
 Marie-José Montpetit, Ph.D.Research Affiliate, MIT Media Laboratory 

On January 10, 2020 at 4:33:51 PM, Michael Welzl ( wrote:
Hi all,

This is all interesting, but it’s not going in the direction that I hoped. I mean, these problems are well known and obvious due to the nature of PEPs.
Imagine that connection-splitting PEPs would be a part of the architecture - known, signaled to, and officially doing what they’re doing, rather than secretly “cheating”.
Think of something more like an application-layer proxy, for example.

Then, some problems would remain, due to the way these devices operate - what they do with buffering, what they do with congestion control. That’s what I was interested in.

I’m getting the impression that problems in the style of those mentioned below are the ONLY types of problems that people have noticed…. is that true?


> On Jan 10, 2020, at 8:51 PM, Yuchung Cheng <> wrote:
> On Fri, Jan 10, 2020 at 11:41 AM Keith Winstein <> wrote:
>> In practice, I suspect some of the main downsides to these TCP (transparent) connection splitters are probably the ones cited by Google in their QUIC paper at SIGCOMM 2017: they have led to ossification of the TCP protocol by enforcing various assumptions about transport behavior on traffic that passes through.
>> See, e.g., "Is it Still Possible to Extend TCP?" (IMC 2011), "Fitting Square Pegs Through Round Pipes" (NSDI 2012), or "How Hard Can It Be? Designing and Implementing a Deployable Multipath TCP" (NSDI 2012).
>> A whole bunch of interesting TCP behavior seems to be frustrated by a non-negligable percentage of these middleboxes. I've personally experienced several middleboxes (for me, I've seen this on local virus scanners and airplane WiFi services) that will kill a connection as soon as one side sends FIN. This frustrates any application that uses half-open connections in an interesting way. Middleboxes will forget about an idle connection even if the endpoints still have the state. Middleboxes will freak out if there is payload alongside SYN or SYN/ACK. SomemMiddleboxes will freak out if they see TCP options they don't know about (including ENO/tcpcrypt). Middleboxes will freak out if a segment appears unacked to them but never gets retransmitted, or if a segment is acked but they didn't see it on the forward path. And, as you say, the TCP connection splitters frustrate any attempt by the endpoints to deploy newer/better/more path-appropriate congestion-control schemes.
>> Just the FUD itself about what some lazy middlebox might freak out about probably contributes substantially to the ossification of TCP.
> Similar sentiments here on real practical disadvantages as a TCP
> developer dealing with Internet issues over a decade.
> There are good PEPs. The real disadvantages are the poorly implemented
> ones and the upkeep. In my personal experience I've worked with a
> cellular provider to disable their PEPS that ended up delivering much
> better (YouTube video) performance. They were very happy to get rid of
> those boxes to save both latency and money.
>> -Keith
>> On Fri, Jan 10, 2020 at 12:54 AM Michael Welzl <> wrote:
>>> Hi,
>>> I’ve been thinking a lot about TCP connection splitters lately ( ).
>>> I’m curious: what are the real practical disadvantages of this type of PEPs that people have seen?
>>> I'll appreciate any kind of feedback, also anecdotes, but pointers to citable papers would be best.
>>> BTW, let’s keep multi-path apart from this discussion please. My question is about single path TCP.
>>> Cheers,
>>> Michael
>>> PS: I’m not trying to indirectly hint that such devices would be *always good*. However, the scenarios where they are not strike me as surprisingly narrow, so I wonder if I’m missing more.