Re: [tcpm] [Tmrg] Increasing the Initial Window - Notes

Jerry Chu <hkchu@google.com> Fri, 19 November 2010 17:12 UTC

Return-Path: <hkchu@google.com>
X-Original-To: tcpm@core3.amsl.com
Delivered-To: tcpm@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8B6813A67E9 for <tcpm@core3.amsl.com>; Fri, 19 Nov 2010 09:12:28 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -109.19
X-Spam-Level:
X-Spam-Status: No, score=-109.19 tagged_above=-999 required=5 tests=[AWL=-0.786, BAYES_00=-2.599, FB_YOU_CAN_BECOME=1.258, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_HI=-8, SARE_MILLIONSOF=0.315, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pyB-F-xLwmev for <tcpm@core3.amsl.com>; Fri, 19 Nov 2010 09:12:27 -0800 (PST)
Received: from smtp-out.google.com (smtp-out.google.com [74.125.121.35]) by core3.amsl.com (Postfix) with ESMTP id 971183A67E1 for <tcpm@ietf.org>; Fri, 19 Nov 2010 09:12:26 -0800 (PST)
Received: from wpaz9.hot.corp.google.com (wpaz9.hot.corp.google.com [172.24.198.73]) by smtp-out.google.com with ESMTP id oAJHDEUN008097 for <tcpm@ietf.org>; Fri, 19 Nov 2010 09:13:15 -0800
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=google.com; s=beta; t=1290186795; bh=eb5aV4aQn4c6zEIeu7/L89zyfoA=; h=MIME-Version:In-Reply-To:References:Date:Message-ID:Subject:From: To:Cc:Content-Type:Content-Transfer-Encoding; b=GkyuiDOeHnYtEwPOAPjs929dA1gg5k3brV0Z3muEuZ4jVwdoKTLuNnFhgIsAIx7GC fQlfDqidD3IwOnjMczfGg==
Received: from gxk8 (gxk8.prod.google.com [10.202.11.8]) by wpaz9.hot.corp.google.com with ESMTP id oAJHCRlP000599 for <tcpm@ietf.org>; Fri, 19 Nov 2010 09:13:13 -0800
Received: by gxk8 with SMTP id 8so2913022gxk.39 for <tcpm@ietf.org>; Fri, 19 Nov 2010 09:13:13 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=beta; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=aQS0r1YJQXohBiA7DCuGoH9eo3B+3Co/ZabgS7llT5k=; b=j8Gg+FuWaiMcKfVb7Tk9S6XjsTHM5WcWuoV8G4g3egy5Vwf4Ui/rmhxhI1jfKIGZ0X V6cLe7k1R/AcIkGXJPGQ==
DomainKey-Signature: a=rsa-sha1; c=nofws; d=google.com; s=beta; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=W8rIqqNpkUzX+I+KR6XEr+IOKrJMv3sxtlv8UpEKwe0LBAimgovR+euiVeSfN5TV61 +S3GSUHgyuNv0SiM3SOg==
MIME-Version: 1.0
Received: by 10.150.91.13 with SMTP id o13mr4022149ybb.6.1290186792664; Fri, 19 Nov 2010 09:13:12 -0800 (PST)
Received: by 10.151.158.13 with HTTP; Fri, 19 Nov 2010 09:13:12 -0800 (PST)
In-Reply-To: <4CE597AB.1020306@isi.edu>
References: <20101110152857.GA5094@hell> <804D02FE-39AF-4437-BB15-C2247842E120@mac.com> <20101110170017.GF5094@hell> <97C75EA8-6CC7-444C-A19D-370148B81918@mac.com> <20101110174056.GH5094@hell> <AANLkTim7g=XqfSMHpHVbw1qqPOL-oNApt2i_2RCt0SCi@mail.gmail.com> <AD2BFE84-CA5B-4CDC-8822-1FC2713E3AE0@cisco.com> <alpine.DEB.2.00.1011161345170.11898@wel-95.cs.helsinki.fi> <E798B9A8-29BB-425B-B0C2-2B2735C49948@cisco.com> <5FDC413D5FA246468C200652D63E627A0B7BD0DF@LDCMVEXC1-PRD.hq.netapp.com> <686EBD23-7B65-455F-9348-196BBFD88ECD@comsys.rwth-aachen.de> <931FAE2C-F66E-43B2-8EE1-CFEB17DABD5E@windriver.com> <7309FCBCAE981B43ABBE69B31C8D213909B72A7B1F@EUSAACMS0701.eamcs.ericsson.se> <36F89B79-EABA-4C38-A59E-023D9A630832@windriver.com> <4CE585E9.6060203@isi.edu> <6B25AF56-AFED-4085-AF42-F8AD47CB9F41@windriver.com> <4CE58FED.608@isi.edu> <0C53DCFB700D144284A584F54711EC580B36940A@xmb-sjc-21c.amer.cisco.com> <4CE595E3.3010109@isi.edu> <AANLkTik3MpEyOZ+EWdtD8C+e2m3BrgKXK0KJrSoF=DMO@mail.gmail.com> <4CE597AB.1020306@isi.edu>
Date: Fri, 19 Nov 2010 09:13:12 -0800
Message-ID: <AANLkTikQK2diSib+U3bgyMpkurdojQip24M+3m9aA2we@mail.gmail.com>
From: Jerry Chu <hkchu@google.com>
To: Joe Touch <touch@isi.edu>
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: quoted-printable
X-System-Of-Record: true
Cc: "Anantha Ramaiah (ananth)" <ananth@cisco.com>, tcpm <tcpm@ietf.org>, David Borman <david.borman@windriver.com>
Subject: Re: [tcpm] [Tmrg] Increasing the Initial Window - Notes
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/tcpm>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 19 Nov 2010 17:12:28 -0000

On Thu, Nov 18, 2010 at 1:16 PM, Joe Touch <touch@isi.edu> wrote:
> Maybe it should have been for IW=3 too, but it didn't occur to me at the
> time.
>
> The benefit is that had we used this mechanism then, we wouldn't be asking
> for a new IW now, though.

The problem is that had we used this mechanism we would've quickly found out
it just doesn't work well. It's always easy to hand wave and point to something
never well tested and claim that would've worked better.

Linux has a "dst cache" feature on a per destination IP basis for years.
The problem is that the effectiveness of the dst cache is determined by the
cache hit rate, which is highly dependent on client-server access pattern.
An ftp server serving RFC documents to a handful of clients (e.g., TCP
researchers) may enjoy high cache hit ratio. But consider > 1B users
accessing millions of servers, what kind of cache hit ratio would you expect?

Frustrated by very low cache hits from Linux's dst cache, we went on to
modify dst cache from per dst IP to per dst /24 subnet basis, but it only
showed limited improvement.

Then we discovered out of the small % of cache hits many still suffered
low cached cwnd. We quickly realized simple cwnd cache is not sufficient
- the prior connection must also have sufficient data to grow its cwnd for
the later connection to enjoy a larger IW.

Adding on top of all the above were transient congestions on well
provisioned users causing cwnd to be clamped down, and later connections
to suffer a small IW even though the congestions were long gone (presumably).

BTW,  we've covered this topic in our 1st presentation (see
http://www.ietf.org/proceedings/77/slides/tcpm-4.pdf) and list
discussions
many times. I really hope we don't have to repeat the same topic over
and over again.

Best,

Jerry

>
> Joe
>
> On 11/18/2010 1:11 PM, John Heffner wrote:
>>
>> Why is this mechanism required for IW=10 but not for IW=3?
>>
>>   -John
>>
>>
>> On Thu, Nov 18, 2010 at 4:08 PM, Joe Touch<touch@isi.edu>  wrote:
>>>
>>> That's basically what I'm suggesting; it can easily be per subnet, per
>>> interface, or per the entire machine as desired.
>>>
>>> The ultimate point is:
>>>
>>>        - put something in the end node that notices if/when
>>>        it fails objectively, and fixes it
>>>
>>>        - that same thing can allow the IW to increase
>>>        over time if there are no problems
>>>
>>> I'll be glad to write this up if people need a more concrete proposal.
>>>
>>> Joe
>>>
>>> On 11/18/2010 1:05 PM, Anantha Ramaiah (ananth) wrote:
>>>>
>>>> Why can't you do something like this :
>>>>
>>>> - If any one of the TCP connections egressing out of that interface
>>>> (this can determined in TCP layer) is in the TCP retransmit state (or
>>>> has experienced congestion in the past xxx secs/mins), then go back to a
>>>> lower IW for new TCP connections which are using the same output
>>>> interface..
>>>>
>>>> - When you want to start a connection, use the connection history (Joe's
>>>> RFC)
>>>>
>>>> Well, there may be some gotchas of this scheme, but you can become
>>>> conservative when there is some concrete information.
>>>>
>>>> Thanks,
>>>> -Anantha
>>>>>
>>>>> To be more clear, here's the case:
>>>>>
>>>>>        start 1000 connections in a row.
>>>>>
>>>>>        during the first connection, lose some packets and do
>>>>>        normal TCP backoff
>>>>>
>>>>>        so what do the other 999 connections start with?
>>>>>        ans: 10 packets
>>>>>
>>>>> The point is that subsequent connections don't do anything different.
>>>>> If
>>>>> you have 1000 connections, you're sending a certain amount of data
>>>>
>>>> into
>>>>>
>>>>> the network without reacting. We're tripling that.
>>>>>
>>>>> That can easily cause congestion. At which point the *existing*
>>>>> connections will backoff, but new connections keep making problems.
>>>>>
>>>>> Joe
>>>>> _______________________________________________
>>>>> tcpm mailing list
>>>>> tcpm@ietf.org
>>>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>
>>> _______________________________________________
>>> tcpm mailing list
>>> tcpm@ietf.org
>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm
>