Re: [ledbat] list of reasons for needing multiple TCP connections

Murari Sridharan <muraris@microsoft.com> Wed, 26 November 2008 00:55 UTC

Return-Path: <ledbat-bounces@ietf.org>
X-Original-To: tana-archive@ietf.org
Delivered-To: ietfarch-tana-archive@core3.amsl.com
Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5269328C222; Tue, 25 Nov 2008 16:55:44 -0800 (PST)
X-Original-To: ledbat@core3.amsl.com
Delivered-To: ledbat@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 2CB2128C21F for <ledbat@core3.amsl.com>; Tue, 25 Nov 2008 16:55:43 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.599
X-Spam-Level:
X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bmUQEiS7IpNs for <ledbat@core3.amsl.com>; Tue, 25 Nov 2008 16:55:42 -0800 (PST)
Received: from smtp.microsoft.com (mail2.microsoft.com [131.107.115.215]) by core3.amsl.com (Postfix) with ESMTP id 2F76828C223 for <ledbat@ietf.org>; Tue, 25 Nov 2008 16:55:42 -0800 (PST)
Received: from tk5-exhub-c104.redmond.corp.microsoft.com (157.54.88.97) by TK5-EXGWY-E802.partners.extranet.microsoft.com (10.251.56.168) with Microsoft SMTP Server (TLS) id 8.1.291.1; Tue, 25 Nov 2008 16:55:39 -0800
Received: from NA-EXMSG-C110.redmond.corp.microsoft.com ([157.54.62.150]) by tk5-exhub-c104.redmond.corp.microsoft.com ([157.54.88.97]) with mapi; Tue, 25 Nov 2008 16:55:38 -0800
From: Murari Sridharan <muraris@microsoft.com>
To: Joe Touch <touch@ISI.EDU>, "Kartik Chandrayana (karchand)" <karchand@cisco.com>
Date: Tue, 25 Nov 2008 16:55:21 -0800
Thread-Topic: [ledbat] list of reasons for needing multiple TCP connections
Thread-Index: AclLUqo0EoV9uKrPS469Bx2rJPHC2wEDf6QA
Message-ID: <FCA794787FDE0D4DBE9FFA11053ECEB61AFFFAE6F0@NA-EXMSG-C110.redmond.corp.microsoft.com>
References: <4925BDEE.6090101@isi.edu> <8c99930d0811201206yb0ef259v28c361438cb14773@mail.gmail.com> <A0988C2F192F124FAFE3D70264C04587077A12EC@xmb-sjc-231.amer.cisco.com> <4925CC2A.9080606@isi.edu> <A0988C2F192F124FAFE3D70264C04587077A130C@xmb-sjc-231.amer.cisco.com> <4925CF2D.4070908@isi.edu>
In-Reply-To: <4925CF2D.4070908@isi.edu>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
MIME-Version: 1.0
Cc: "ledbat@ietf.org" <ledbat@ietf.org>
Subject: Re: [ledbat] list of reasons for needing multiple TCP connections
X-BeenThere: ledbat@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Mailing list of the LEDBAT WG <ledbat.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/ledbat>, <mailto:ledbat-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/pipermail/ledbat>
List-Post: <mailto:ledbat@ietf.org>
List-Help: <mailto:ledbat-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ledbat>, <mailto:ledbat-request@ietf.org?subject=subscribe>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Sender: ledbat-bounces@ietf.org
Errors-To: ledbat-bounces@ietf.org

I am not sure I get the entire context of Kartik's comment but Windows Server supports a feature called Receive-side scaling which essentially distributes receive processing and tries to spread TCP connections across multiple cores and tries to keep all processing for a connection aligned on one specific CPU (to the extent that apps cooperate) starting with packet indication all the way to deliver to the app. Connections can get moved around depending on load but in steady state they should remain pinned with the some caveats.

Some links
http://download.microsoft.com/download/a/d/f/adf1347d-08dc-41a4-9084-623b1194d4b2/RSS_Server2008.docx

http://msdn.microsoft.com/en-us/library/ms795616.aspx

Thanks

-----Original Message-----
From: ledbat-bounces@ietf.org [mailto:ledbat-bounces@ietf.org] On Behalf Of Joe Touch
Sent: Thursday, November 20, 2008 12:57 PM
To: Kartik Chandrayana (karchand)
Cc: ledbat@ietf.org
Subject: Re: [ledbat] list of reasons for needing multiple TCP connections

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Kartik Chandrayana (karchand) wrote:
> Joe,
>
>   I am also not aware if parallelization of network code is implemented
> in OSes. However, I think it would be reasonable to expect that tcpcb
> lookups are done in a centralized fashion and once that is done,
> multiple tcp's can be processed in different cpus. Even the lookups can
> be distributed as they are only a read operation.

Parallelizing TCP includes dealing with not only the TCP code, but also
the IP input queue access, timers, etc. I am not as sure that there's a
one TCP per core mapping in current OS's - though I'd be glad to hear
someone *confirm* it.

Joe

> -----Original Message-----
> From: Joe Touch [mailto:touch@ISI.EDU]
> Sent: Thursday, November 20, 2008 2:44 PM
> To: Kartik Chandrayana (karchand)
> Cc: Andrew G. Malis; ledbat@ietf.org
> Subject: Re: [ledbat] list of reasons for needing multiple TCP
> connections
>
>
>
> Kartik Chandrayana (karchand) wrote:
>> Andy, Joe:
>
>> One reason I can think of using multiple TCP connections is to
>> overcome the non-negotiation of window-scale parameter on most boxes.
>> With default window scale parameters, i.e 0, the congestion window is
>> limited to 64KB only and thus for any pipe with
>> bandwidth-delay-product greater than 64KB, a single TCP connection
>> cannot keep the pipe full. In such a case, it would be beneficial to
> have multiple TCP connections.
>> Also, since TCP design was mostly with single cpu in mind, I think
>> multiple TCP connections can harness the multi-cpu architectures
>> better than a single lean fast TCP.
>
> That presumes that the OS implements parallelization in the network
> code; I don't know if that's recently offered, but it has been the
> sticking point in the past.
>
> Joe
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkklzy0ACgkQE5f5cImnZrtxIwCgw+uW7JxMRDZja1pTuixQUwS8
m5IAoPHHCB2ed20zXkkv15m09kF4q+mm
=30jd
-----END PGP SIGNATURE-----
_______________________________________________
ledbat mailing list
ledbat@ietf.org
https://www.ietf.org/mailman/listinfo/ledbat

_______________________________________________
ledbat mailing list
ledbat@ietf.org
https://www.ietf.org/mailman/listinfo/ledbat