Re: [tcpm] TCP tuning

rick jones <perfgeek@mac.com> Thu, 04 February 2010 15:59 UTC

Return-Path: <perfgeek@mac.com>
X-Original-To: tcpm@core3.amsl.com
Delivered-To: tcpm@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id EB9BC3A689B for <tcpm@core3.amsl.com>; Thu, 4 Feb 2010 07:59:57 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.599
X-Spam-Level:
X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TUVp9MHWlsk4 for <tcpm@core3.amsl.com>; Thu, 4 Feb 2010 07:59:56 -0800 (PST)
Received: from asmtpout022.mac.com (asmtpout022.mac.com [17.148.16.97]) by core3.amsl.com (Postfix) with ESMTP id 66B1F3A676A for <tcpm@ietf.org>; Thu, 4 Feb 2010 07:59:56 -0800 (PST)
MIME-version: 1.0
Content-transfer-encoding: 7bit
Content-type: text/plain; charset="US-ASCII"; format="flowed"; delsp="yes"
Received: from [192.168.1.101] (76-220-56-223.lightspeed.sntcca.sbcglobal.net [76.220.56.223]) by asmtp022.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0KXB00ARAR561C60@asmtp022.mac.com> for tcpm@ietf.org; Thu, 04 Feb 2010 08:00:43 -0800 (PST)
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=5.0.0-0908210000 definitions=main-1002040123
Message-id: <6C3B471F-6ACB-467E-9D11-EE68388B3F5B@mac.com>
From: rick jones <perfgeek@mac.com>
To: Joe Touch <touch@ISI.EDU>
In-reply-to: <4B6A6848.80904@isi.edu>
Date: Thu, 04 Feb 2010 08:00:42 -0800
References: <7BE9742D-6EDC-43FE-84FC-D22C52D23152@nokia.com> <133D9897FB9C5E4E9DF2779DC91E947C025F1861@SLFSNX.rcs.alcatel-research.de> <d1c2719f1002031110v3b76ca9eu14c9a110847548e7@mail.gmail.com> <4B69CDD7.6060802@isi.edu> <E086E248-DEFA-4AA4-B25D-F7FBB0FB9E7D@mac.com> <4B6A43D1.2010706@isi.edu> <9E43CA53-5BFD-4377-93D1-1AED32C09B0E@mac.com> <443C71B6-1D1A-49CA-B5E4-FB81FDCB7FD1@mac.com> <4B6A6848.80904@isi.edu>
X-Mailer: Apple Mail (2.936)
Cc: tcpm@ietf.org, "SCHARF, Michael" <Michael.Scharf@alcatel-lucent.com>
Subject: Re: [tcpm] TCP tuning
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/tcpm>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 04 Feb 2010 15:59:58 -0000

On Feb 3, 2010, at 10:25 PM, Joe Touch wrote:
> Did you use a netperf option to disable Nagle?

No, there should be no need for that.  For a TCP_RR test netperf  
presents all the data of a request or response to the transport in a  
single send call,  there should be no delays Nagle at all.  Further,  
in this case, there was only ever the one transaction in flight at a  
time, so at the time of each send() the connection was "idle" in the  
Nagle sense.

> There are also many reasons this could happen; it might be easier to  
> see
> in a plot.

Anyone should feel free to grab either the binary (.pcap) or  
"cooked" (ascii) traces off of netperf.org and run them through  
whatever post processing tools they like.  I would have shoved them  
into the original email, but didn't want to stuff peoples' inboxes  
unnecessarily.  Still, if you think that copying some examples from  
the ascii into email would help I'm willing to do that.

I'm not the acme of tcpdump trace reading, but it seemed pretty  
conclusive in the traces - with slowstart after idle, the 2.6.31  
server sent the data of each 16K response with what certainly appears  
to be slow-start behaviour, with that sysctl set to disable slowstart  
after idle, no such slow start behaviour in the middle of the trace.   
So, for Linux 2.6.31, as distributed by Ubuntu, it would seem that the  
arrival of the next request is not going to preclude slowstart after  
idle behaviour.

rick jones

>
> Joe
>
> rick jones wrote:
>>
>> On Feb 3, 2010, at 9:41 PM, rick jones wrote:
>>> PS - I suppose if I'd been thinking more clearly, I'd have also
>>> included --enable-historgram in the ./configure command and been  
>>> able
>>> to give a histogram of the response times (adding a -v 2 option to  
>>> the
>>> global command line options), but I suppose that can be guesstimated
>>> from the traces.
>>
>> Ok, so perhaps I like running netperf too much :)  Here then are the
>> application-level measured response times.  The first netperf is with
>> tcp_slow_start_after_idle = 0, the second with it set to 1, I'll
>> leave-out the rest of the netperf output as it is repetitive:
>>
>> tcp_slow_start_after_idle = 0 on server:
>>
>> Histogram of request/response times
>> UNIT_USEC     :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> TEN_USEC      :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> HUNDRED_USEC  :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> UNIT_MSEC     :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> TEN_MSEC      :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> HUNDRED_MSEC  :    0:  294:    4:    0:    2:    0:    0:    0:     
>> 0:    0
>> UNIT_SEC      :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> TEN_SEC       :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>>> 100_SECS: 0
>> HIST_TOTAL:      300
>>
>> That means 294 times 100ms <= responsetime < 200ms, 4 times it was  
>> 200ms
>> <= responsetime < 300ms etc etc.
>>
>> tcp_slow_start_after_idle = 1 on server:
>>
>> UNIT_USEC     :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> TEN_USEC      :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> HUNDRED_USEC  :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> UNIT_MSEC     :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> TEN_MSEC      :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> HUNDRED_MSEC  :    0:    0:    0:  290:   10:    0:    0:    0:     
>> 0:    0
>> UNIT_SEC      :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>> TEN_SEC       :    0:    0:    0:    0:    0:    0:    0:    0:     
>> 0:    0
>>> 100_SECS: 0
>> HIST_TOTAL:      300
>>
>> Now it was 290 with 300ms <= responsetime < 400ms
>>
>> Just as a refresher, the request size was 128 bytes, the response  
>> size
>> 16384.
>>
>> The ICMP Echo RTT is ~120 ms between my home system and netperf.org:
>>
>> --- netperf.org ping statistics ---
>> 10 packets transmitted, 10 packets received, 0% packet loss
>> round-trip min/avg/max/stddev = 119.523/120.410/121.248/0.493 ms
>>
>> rick jones
>> Wisdom teeth are impacted, people are affected by the effects of  
>> events
>

Wisdom teeth are impacted, people are affected by the effects of events