Re: [tsvwg] Naive question on multiple TCP/IP channels

Piers O'Hanlon <p.ohanlon@gmail.com> Thu, 05 February 2015 10:06 UTC

Return-Path: <p.ohanlon@gmail.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CB4F31A014E; Thu, 5 Feb 2015 02:06:49 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level:
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LzsYry7IN2g0; Thu, 5 Feb 2015 02:06:47 -0800 (PST)
Received: from mail-wg0-x22b.google.com (mail-wg0-x22b.google.com [IPv6:2a00:1450:400c:c00::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 012401A0141; Thu, 5 Feb 2015 02:06:46 -0800 (PST)
Received: by mail-wg0-f43.google.com with SMTP id y19so6663285wgg.2; Thu, 05 Feb 2015 02:06:45 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=vNalM8nx4f/NS/VMw+5D1WLOAZoqLHNJqfKW7X/ENH8=; b=K92Fj8PalOun+5MjisNx1iAXytqafRd1Ctsg5664PuYjvLHr3fLuDV/d+iBPYTT2Up 5cXiBxktUYGFMuSUPnWO8VHVl2fwdaJLYyqsYaUUEgGE5HdHTZB+ccTUyG/+502jD0Vx zdJ2AWIXiXicfumrmuS+dqJp/sKRxodRTqgWe/SImfkc9+OeLA+eoxmxqg23k+NbblIc 6tx1sUvmNhQlrQqF3iXhnuo32BMHsGfjikbs9NZ6Pqo/vh/3pU4+GQ3Y9o9huimTY8Gi NRB3Dhyb4+SkzImIYnezBMKrlNYIPQ0FFQ4CDqLpLoZ8+muKicHpgjslA0DkceKBv/eY HEsA==
X-Received: by 10.180.103.33 with SMTP id ft1mr13722114wib.19.1423130805566; Thu, 05 Feb 2015 02:06:45 -0800 (PST)
Received: from black.lan (bcddfef6.skybroadband.com. [188.221.254.246]) by mx.google.com with ESMTPSA id k1sm6631672wjn.9.2015.02.05.02.06.43 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 05 Feb 2015 02:06:44 -0800 (PST)
Subject: Re: [tsvwg] Naive question on multiple TCP/IP channels
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset="us-ascii"
From: Piers O'Hanlon <p.ohanlon@gmail.com>
In-Reply-To: <54D2B150.3020502@gmail.com>
Date: Thu, 05 Feb 2015 10:06:42 +0000
Content-Transfer-Encoding: quoted-printable
Message-Id: <53117D83-2F80-4CC7-97FC-8C6FAA49857C@gmail.com>
References: <CAMm+Lwgb9L9bUG6ommBDYJzQTCU1cC_zLSEf_5JPeJ+c=yrYmA@mail.gmail.com> <5DF6DC77-E476-408F-9FA5-F107DDC9F857@netapp.com> <54D2B150.3020502@gmail.com>
To: Phillip Hallam-Baker <phill@hallambaker.com>
X-Mailer: Apple Mail (2.1283)
Archived-At: <http://mailarchive.ietf.org/arch/msg/ietf/LpPLEn8Pvmh5oTDol2WadhSgXzc>
X-Mailman-Approved-At: Thu, 05 Feb 2015 08:52:13 -0800
Cc: IETF Discussion Mailing List <ietf@ietf.org>, tsvwg WG <tsvwg@ietf.org>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 05 Feb 2015 10:06:50 -0000

One of the snags with using multiple [slower] TCP streams is that although you might get a faster throughput you end up losing more packets as the equivalent TCP loss rate for multiple lower rate flows is higher than for one larger stream. This can be explained by looking at the TCP [simple] equation Xrate = 1.2*PacketLen/(RTT*sqrt(loss)) where it can be seen that the lower the send rate the higher the loss needs to be on a path with the same RTT.

I guess there were also other approaches like BEEP, and Google's QUIC which multiplexed multiple connections over UDP. 

Piers

On 4 Feb 2015, at 23:54, Brian E Carpenter wrote:

> On 05/02/2015 08:49, Eggert, Lars wrote:
>> Hi,
>> 
>> CC'ing tsvwg, which would be a better venue for this discussion.
>> 
>> On 2015-2-4, at 20:22, Phillip Hallam-Baker <phill@hallambaker.com> wrote:
>>> 
>>> Today most Web browsers attempt to optimize download of images etc. by opening multiple TCP/IP streams at the same time. This is actually done for two reasons, first to reduce load times and second to allow the browser to optimize page layout by getting image sizes etc up front.
>>> 
>>> This approach first appeared round about 1994. I am not sure whether anyone actually did a study to see if multiple TCP/IP streams are faster than one but the approach has certainly stuck.
>> 
>> There have been many studies; for example, http://www.aqualab.cs.northwestern.edu/publications/106-modeling-and-taming-parallel-tcp-on-the-wide-area-network
> 
> GridFTP only exists because of lots of experience that several parallel FTP
> streams achieve better throughput than a single stream, especially on paths
> with a high bandwidth-delay product. I'm guessing that since buffer bloat
> creates an artificially high BDP, that could apply pretty much anywhere.
> 
> SCTP is not the only one-acronym answer: try MPTCP. The interesting thing there
> is that because there is explicit coupling between the streams, the throughput
> increases sub-linearly with the number of streams.
> 
>    Brian
> 
>> 
>>> But looking at the problem from the perspective of the network it is really hard to see why setting up five TCP/IP streams between the same endpoints should provide any more bandwidth than one. If the narrow waist is observed, then the only parts of the Internet that are taking note of the TCP part of the packet are the end points. So having five streams should not provide any more bandwidth than one unless the bandwidth bottleneck was at one or other endpoint.
>> 
>> You don't get more bandwidth in stead state (well, with old Reno stacks, you got a little more, but not much). The real win is in getting more bandwidth during the first few RTTs of TCP slow-start, which is the crucial phase when transmitting short web objects.
>> 
>>> Now there are some parts of the deployed Internet that do actually perform statefull inspection. But I would expect increasing the number of channels to degrade performance at a firewall or any other middle boxen.
>>> 
>>> So we have a set of behavior that seems at odd with the theory. Has anyone done any experiments recently that would show which is right?
>> 
>> I haven't seen any performance study, but another concern is that middleboxes obviously need to maintain state per connection, and multiple parallel connections eat that binding space up more quickly. (And for a NAT, reduce the number of clients it can serve.)
>> 
>>> The reason it makes a difference is that it is becoming clear that modern applications are not best served by an application API that is limited to one bi-directional stream. There are two possible ways to fix this situation. The first is to build something on top of TCP/IP the second is to replace single stream TCP with multi-stream.
>> 
>> SCTP has what you call multiple streams in your second option, and is designed the same way.
>> 
>>> My preference and gut instinct is that the first is the proper architectural way to go regardless of the performance benefits. When Thompson and co were arguing that all files are flat sequences of bits, they were saying that was the right O/S abstraction because you could build anything you like on top.
>>> 
>>> But then I started to ask what the performance benefits to a multi-stream TCP might be and I am pretty sure there should not be any. But the actual Internet does not always behave like it appears it should.
>> 
>> See above.
>> 
>> Also, one motivation for SPDY/HTTP2.0 is to reduce the number of parallel connections, since web people have noticed that more is not always better here.
>> 
>>> I suspect that the preference for multiple streams probably comes from the threading strategies it permits. But that is an argument about where the boundary between the kernel and application is placed in the network stack rather than where multiplex should live in the stack. Microsoft already provides a network stack for .NET where the boundary is in the HTTP layer after all.
>>> 
>>> 
>>> So anyone got hard data they could share?
>> 
>> The TSVWG folks may have.
>> 
>> Lars
>> 
>