Re: [tcpm] usage for timestamp options in the wild

Yuchung Cheng <> Thu, 27 September 2018 23:56 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 6B31E128A6E for <>; Thu, 27 Sep 2018 16:56:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -17.501
X-Spam-Status: No, score=-17.501 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id f3nJrcAtHptN for <>; Thu, 27 Sep 2018 16:56:25 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::12c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 9B60B128CB7 for <>; Thu, 27 Sep 2018 16:56:24 -0700 (PDT)
Received: by with SMTP id m9-v6so666830ita.2 for <>; Thu, 27 Sep 2018 16:56:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=QFMUevKApwbfIAp64yTovha1FTw0+KV0kzcaS8wUgNg=; b=ZVkGOya9OqyXQ9qh8DXAXRgCEx3/R9tGgSX48pw2EY6Bg57kKEeOh6IQ+o8JySsBP+ 9PCKStAu+Qv8ByShJn2LPDdLeBqET/s1ggeHHmfjyDA4Vqh5sjoI5DEdXWMwe/yvGXRK WBx2fAyZ7MHx0cQINwv9MLe/LVkSID91V8oNVoycCTJnO0vrAtkYvOOQt5/CmmIrw2AG 96UqedgfjhX3QXIqgA1dP/i01vni/gAKRykF2LEQ886iHI6OdCTSdwEODvQKJqck/dC3 SVYp9wDWX2U1AB1MH+9gWMiU5euYMn77rUUXunbA59EAPK0RpJOK12Hq1TAFMhfIUmW9 7kfQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=QFMUevKApwbfIAp64yTovha1FTw0+KV0kzcaS8wUgNg=; b=kwbvnn+UUhFNxVSFq5ZKIp4FRauy9bC71zO8Zp2o94iFLbHfVILae6hu0/AqqsyTW5 ipl9wlAZaPGPxT2SEfvIrg+L6tq1GZLU9rkD/MeaTiT825WsjVWnL0d/GIRItrZ0aOG/ qZkZNsvMA4OH7H7pyjXA85HbldYqFlAfwlAPiINPkNSN4C7TJfsvLaS0m7ljuld38Gss Sg5mFhTRz6Or4KC8/XpKrDm/5V41ZU8pYDpLPPAP96qevf1JHvKORuRhVjKRkK+FvDyF yE/SRsM5XFIlW4GfXeEJXWMrHl2UPlurAlQs7Hs5gWZpf1pqyGIX3+T4+f6WHybzaZIE qC+A==
X-Gm-Message-State: ABuFfojctug0hcYWknvmUSy5zxNtEPBvAwglOlO5rmma3hIo2s/GTma9 4lHk+Rq0g8/J/GyKS3iZFU0jFC8yg+36cDpYCjjOmTYo
X-Google-Smtp-Source: ACcGV635fegZQf2orYGE8MiLAe7yPSDkjz9M9zsGUeKpp83cffmHRLxZt+aLW2KUZBM9sCEYE4As332FsjPtZjtqd9g=
X-Received: by 2002:a24:5d4a:: with SMTP id w71-v6mr655620ita.118.1538092583289; Thu, 27 Sep 2018 16:56:23 -0700 (PDT)
MIME-Version: 1.0
Received: by 2002:a6b:f715:0:0:0:0:0 with HTTP; Thu, 27 Sep 2018 16:55:42 -0700 (PDT)
In-Reply-To: <>
References: <> <> <> <>
From: Yuchung Cheng <>
Date: Thu, 27 Sep 2018 16:55:42 -0700
Message-ID: <>
To: Yoshifumi Nishida <>
Cc: Praveen Balasubramanian <>, "" <>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <>
Subject: Re: [tcpm] usage for timestamp options in the wild
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 27 Sep 2018 23:56:28 -0000

On Wed, Sep 26, 2018 at 12:43 PM, Yoshifumi Nishida
<> wrote:
> On Wed, Sep 26, 2018 at 11:37 AM, Yuchung Cheng <> wrote:
>> On Wed, Sep 26, 2018 at 11:21 AM, Praveen Balasubramanian
>> <> wrote:
>> >
>> > Windows doesn’t do timestamp option by default. It’s 10 byte per packet overhead for marginal benefits.
>> >
>> >
>> >
>> > From: tcpm <> On Behalf Of Yoshifumi Nishida
>> > Sent: Wednesday, September 26, 2018 10:49 AM
>> > To:
>> > Subject: [tcpm] usage for timestamp options in the wild
>> >
>> >
>> >
>> > Hello,
>> >
>> > this is just out of my curiosity.
>> >
>> > I've checked traffic archives in CAIDA ( and WIDE ( on a whim and tried to see how many connections utilize timestamps.
>> >
>> >
>> >
>> > As far as I've checked some archives recently captured, it seems that around 60-70% TCP connections use timestamp option. But, this ratio seems to be a bit lower as I thought most of implementations these days support TS option and activate it by default.
>> how recent are the archives?
> They are 2018 data.
There's some number dated in 2011

"2. GOOGLE MEASUREMENTS We sampled TCP statistics on unmodified Google
Web servers world-wide for one week in May 2011. The servers use a
recent version of Linux 2.6 with settings listed in Table 4. The
servers do not include the changes proposed in this paper. The global
statistics for interactive Web services are summarized in Table 1.
Among the billions of connections sampled, 96% of the connections
negotiated SACK but only 12% of the connections negotiated Timestamps.
The majority of clients are Microsoft Windows which by default do not
use TCP Timestamps."

While this is comparing apple (google) to oranges (caida), I would say
70% is already a significant jump.

However I am sensing that the value of TCP timestamp options is
under-valued - Linux for example uses TS for many things:

1. RTT estimation on retransmission - this is crucial when a TCP is in
constant recovery mode such as encountering a traffic policer. Note
that the standard ACK time approach no longer works if any sequence
acked has been transmitted, thus RTO may stale during extended
recovery period.

2. Undo operations (i.e. TCP Eifel RFC4015) requires TCP timestamps -
being able to detect false reactions to losses is particularly key for
"loss-based" congestion control. Such false positives are common in
wireless or even wired networks due to all sorts of dark queues and L2
optimizations. While DSACK and F-RTO can also be used to detect false
recoveries - both of them take at least one more round-trip and/or
require new application data, while timestamp can detect false
retransmission instantly on the original ACK

3. Receiver buffer auto-tuning uses TCP ACKs' TS value to echo to
measure RTT from the receiver to tune the buffer size.

4. Potentially TCP timestamps being generated with known units, can be
used to gauge the data arrival rate. The measurement can be valuable
for congestion control b/c it's not subject to hectic ACK delays in
modern networks. I know folks at NetFlix is doing some deep research
into this aspect.

QUIC has emphasized the importance of unique packet identifier - TCP
TS provides a close equivalent and is super useful, especially w/ more
precision / unit vs an opaque value.

>> AFAIK TS is enabled by default on Linux, FreeBSD, and iOS. Linux has
>> that by default more than a decade
> that's why I was wondering.
> --
> Yoshi