Re: Naive question on multiple TCP/IP channels

David Morris <dwm@xpasc.com> Wed, 04 February 2015 19:54 UTC

Return-Path: <dwm@xpasc.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 97DD41A702C for <ietf@ietfa.amsl.com>; Wed, 4 Feb 2015 11:54:55 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 2.154
X-Spam-Level: **
X-Spam-Status: No, score=2.154 tagged_above=-999 required=5 tests=[BAYES_50=0.8, IP_NOT_FRIENDLY=0.334, MISSING_HEADERS=1.021, SPF_PASS=-0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nRPlC2vZaWxG for <ietf@ietfa.amsl.com>; Wed, 4 Feb 2015 11:54:54 -0800 (PST)
Received: from c2w3p-2.abacamail.com (c2w3p-2.abacamail.com [67.231.154.153]) by ietfa.amsl.com (Postfix) with ESMTP id 83C091A6FF2 for <ietf@ietf.org>; Wed, 4 Feb 2015 11:54:54 -0800 (PST)
Received: from xpasc.com (h-68-164-244-186.snva.ca.megapath.net [68.164.244.186]) by c2w3p-2.abacamail.com (Postfix) with ESMTP id D0F043F534 for <ietf@ietf.org>; Wed, 4 Feb 2015 19:54:51 +0000 (UTC)
Received: from egate.xpasc.com (egate.xpasc.com [10.1.2.49]) by xpasc.com (8.13.8/8.13.8) with ESMTP id t14JsqZF008728 for <ietf@ietf.org>; Wed, 4 Feb 2015 11:54:52 -0800
Date: Wed, 04 Feb 2015 11:54:52 -0800
From: David Morris <dwm@xpasc.com>
cc: IETF Discussion Mailing List <ietf@ietf.org>
Subject: Re: Naive question on multiple TCP/IP channels
In-Reply-To: <CAMm+Lwgb9L9bUG6ommBDYJzQTCU1cC_zLSEf_5JPeJ+c=yrYmA@mail.gmail.com>
Message-ID: <alpine.LRH.2.01.1502041147120.4284@egate.xpasc.com>
References: <CAMm+Lwgb9L9bUG6ommBDYJzQTCU1cC_zLSEf_5JPeJ+c=yrYmA@mail.gmail.com>
User-Agent: Alpine 2.01 (LRH 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset="US-ASCII"
Archived-At: <http://mailarchive.ietf.org/arch/msg/ietf/_r1WsJg5_8DqktqWxhscPyKBhI4>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: ietf@ietf.org
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 04 Feb 2015 19:54:55 -0000

Not a statistically valid survey, but watching many last mile connections
to end user clients, I saw at least two issues that are mitigated by
multiple connections:
1) Even small pipes aren't filled by 1 connection ... for sure at
   startup
2) Error recovery from lost packets delays only one connection so
   the last mile can remain utilized.

In a context where we had a local proxy (same system), altering the
default number of connections between the browser and proxy from a few to
on the order of 16-32 improved overall page load times. (The proxy used a
single SPDY like connection to a server in all cases.)

Dave Morris

On Wed, 4 Feb 2015, Phillip Hallam-Baker wrote:

> Today most Web browsers attempt to optimize download of images etc. by opening multiple TCP/IP streams at the same time. This is actually done for two reasons, first to
> reduce load times and second to allow the browser to optimize page layout by getting image sizes etc up front.
> This approach first appeared round about 1994. I am not sure whether anyone actually did a study to see if multiple TCP/IP streams are faster than one but the approach
> has certainly stuck.
>
> But looking at the problem from the perspective of the network it is really hard to see why setting up five TCP/IP streams between the same endpoints should provide any
> more bandwidth than one. If the narrow waist is observed, then the only parts of the Internet that are taking note of the TCP part of the packet are the end points. So
> having five streams should not provide any more bandwidth than one unless the bandwidth bottleneck was at one or other endpoint.
>
> Now there are some parts of the deployed Internet that do actually perform statefull inspection. But I would expect increasing the number of channels to degrade
> performance at a firewall or any other middle boxen.
>
> So we have a set of behavior that seems at odd with the theory. Has anyone done any experiments recently that would show which is right?
>
>
> The reason it makes a difference is that it is becoming clear that modern applications are not best served by an application API that is limited to one bi-directional
> stream. There are two possible ways to fix this situation. The first is to build something on top of TCP/IP the second is to replace single stream TCP with multi-stream.
>
> My preference and gut instinct is that the first is the proper architectural way to go regardless of the performance benefits. When Thompson and co were arguing that all
> files are flat sequences of bits, they were saying that was the right O/S abstraction because you could build anything you like on top.
>
> But then I started to ask what the performance benefits to a multi-stream TCP might be and I am pretty sure there should not be any. But the actual Internet does not
> always behave like it appears it should.
>
> I suspect that the preference for multiple streams probably comes from the threading strategies it permits. But that is an argument about where the boundary between the
> kernel and application is placed in the network stack rather than where multiplex should live in the stack. Microsoft already provides a network stack for .NET where the
> boundary is in the HTTP layer after all.
>
>
> So anyone got hard data they could share?
>
>