[v6ops] Apple and IPv6 - Happy Eyeballs

David Schinazi <dschinazi@apple.com> Thu, 09 July 2015 22:00 UTC

Return-Path: <dschinazi@apple.com>
X-Original-To: v6ops@ietfa.amsl.com
Delivered-To: v6ops@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com []) by ietfa.amsl.com (Postfix) with ESMTP id 5D35D1A1A47 for <v6ops@ietfa.amsl.com>; Thu, 9 Jul 2015 15:00:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.412
X-Spam-Status: No, score=-2.412 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id ueN1c2vPIo1t for <v6ops@ietfa.amsl.com>; Thu, 9 Jul 2015 15:00:42 -0700 (PDT)
Received: from mail-in5.apple.com (mail-out5.apple.com []) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2B1101A1A42 for <v6ops@ietf.org>; Thu, 9 Jul 2015 15:00:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; d=apple.com; s=mailout2048s; c=relaxed/simple; q=dns/txt; i=@apple.com; t=1436479241; x=2300392841; h=From:Sender:Reply-To:Subject:Date:Message-id:To:Cc:MIME-version:Content-type: Content-transfer-encoding:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=JpIrW4VWKlelviJ4x5FHgsN43Oev2rp0XZicttti+HY=; b=KGx85oasapNP1EJ00iLRzvX5zwkCaQ+6u7KA6hEtrEY0cltRCL7YjShNmF0Vb8Oa c8Dc4DLmTHkBBfvOCqMgsGPB1M6Kg1vbZ7c+ee8XqXX8yG8bUI3tmoEd0sXEeYu3 AgTdQjIpJiOqwTRyrE5HrFjzqWGyBCtqznhrs3oF38Z7GWpmQ3u4eZAiZXfxwz33 qghlwvRAjjkMvPvscx8QXGKTcRDk02ALMoaFeNwDxqPXUAbfhCYqN/9WNcCUurhA 3aIDStV8L+MIpN4r0n7in9gmQEpkAd6pYgOrxQ5mqXlXsSw5pRREqNKmE6yB6Al1 P9BUcAF5SAnH2sifPdNdnA==;
Received: from relay4.apple.com (relay4.apple.com []) by mail-in5.apple.com (Apple Secure Mail Relay) with SMTP id 7E.89.12430.90FEE955; Thu, 9 Jul 2015 15:00:41 -0700 (PDT)
X-AuditID: 11973e13-f79d56d00000308e-9b-559eef09c33e
Received: from koseret (koseret.apple.com []) (using TLS with cipher DES-CBC3-SHA (168/168 bits)) (Client did not present a certificate) by relay4.apple.com (Apple SCV relay) with SMTP id DC.56.11814.90FEE955; Thu, 9 Jul 2015 15:00:41 -0700 (PDT)
Received: from da0602a-dhcp109.apple.com (da0602a-dhcp109.apple.com []) by koseret.apple.com (Oracle Communications Messaging Server 64bit (built Oct 22 2013)) with ESMTPSA id <0NR800JXYQH4DU50@koseret.apple.com> for v6ops@ietf.org; Thu, 09 Jul 2015 15:00:41 -0700 (PDT)
From: David Schinazi <dschinazi@apple.com>
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: quoted-printable
Date: Thu, 09 Jul 2015 15:00:40 -0700
Message-id: <C997EDC0-593F-47E4-A1D3-1493AF371B73@apple.com>
To: v6ops@ietf.org
MIME-version: 1.0 (Mac OS X Mail 9.0 \(3067\))
X-Mailer: Apple Mail (2.3067)
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprBLMWRmVeSWpSXmKPExsUi2FAYrsv5fl6owY/Pehanj+1ldmD0WLLk J1MAYxSXTUpqTmZZapG+XQJXRvO0jawF/aIVu68+Y2xgvCTQxcjJISFgIvGhfyMLhC0mceHe ejYQW0hgL6PE1SuMXYwcYDXN63y7GLmAwp1MEmt6f7FCOOuYJBZPXsQMUsQmoCVxYI0RSC8z kLl+53EmCFtb4sm7C6wgtrCAhsT3E9fAbBYBVYnTz/eA7eUVsJH4PuMTG0S9lcSjziVgvSIC QhI7njUxQdToSXSvmckEcaesxMb7fxlBbpAQuMoqsePse8YJjIKzkOyehWT3LCT9CxiZVzEK 5SZm5uhm5pnqJRYU5KTqJefnbmIEBeV0O+EdjKdXWR1iFOBgVOLh1dg+N1SINbGsuDL3EKM0 B4uSOO+fK/NChQTSE0tSs1NTC1KL4otKc1KLDzEycXBKNTAuiD5qtcnyqK3AjmmGF0+yTRSO LH/WpVB+dfPCY0sNPk/7wNWgJpFgP29ZNdfvCCPRkt/BmulfzrWIVFf80ohf9k/nVbb9vl9z kk6/rv7L6Lqfp6o2v/bsFzYuEybWut21+jc2BWQ+SnS+cv1d16YnWzxVkg52ZDlv2a418+5i ibY9DwqDDgcosRRnJBpqMRcVJwIAonaTZisCAAA=
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFuphluLIzCtJLcpLzFFi42IRnG6nrsv5fl6owYQtOhanj+1ldmD0WLLk J1MAYxSXTUpqTmZZapG+XQJXRvO0jawF/aIVu68+Y2xgvCTQxcjBISFgItG8zreLkRPIFJO4 cG89WxcjF4eQQCeTxJreX6wQzjomicWTFzGDNLAJaEkcWGME0sAMZK7feZwJwtaWePLuAiuI LSygIfH9xDUwm0VAVeL08z0sIDavgI3E9xmf2CDqrSQedS4B6xUREJLY8ayJCaJGT6J7zUwm iINkJTbe/8s4gZFvFpJ1s5Csm4WkZQEj8ypGgaLUnMRKE73EgoKcVL3k/NxNjKAwaigM38H4 b5nVIUYBDkYlHl6N7XNDhVgTy4orcw8xSnAwK4nwpr6eFyrEm5JYWZValB9fVJqTWnyIUZqD RUmcV3PKlFAhgfTEktTs1NSC1CKYLBMHp1QDo8jZbS+us31rlb3+P69XJVH2ea9WYu/8+433 P9gd432/niuF+8ObaR3yjz8LHdpyaQ/r4+dPQpIcVOqCetImrdxw/tnRR3rH1ThSVSVsFXa3 tFd9l9hcVmWw9t4fA8f9e878Kd3j+yDx1PUaV5WqIs3p+5s1uTsv1lg3vozo+nogMs9Ms6lS T4mlOCPRUIu5qDgRAGvTa74fAgAA
Archived-At: <http://mailarchive.ietf.org/arch/msg/v6ops/DYiI9v_O66RNbMJsx0NsatFkubQ>
Cc: Paul Saab <ps@fb.com>
Subject: [v6ops] Apple and IPv6 - Happy Eyeballs
X-BeenThere: v6ops@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: v6ops discussion list <v6ops.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/v6ops>, <mailto:v6ops-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/v6ops/>
List-Post: <mailto:v6ops@ietf.org>
List-Help: <mailto:v6ops-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/v6ops>, <mailto:v6ops-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Jul 2015 22:00:43 -0000

Hi everyone,

Today Apple released the first public seeds of iOS 9 and OS X El Capitan.
These seeds (and the third developer seeds released yesterday) include an improved version of Happy Eyeballs.

Based on our testing, this makes our Happy Eyeballs implementation go from roughly 50/50 IPv4/IPv6 in iOS 8 and Yosemite
to ~99% IPv6 in iOS 9 and El Capitan betas.

While our previous implementation from four years ago was designed to select the connection with lowest latency
no matter what, we agree that the Internet has changed since then and reports indicate that biasing towards IPv6 is now
beneficial for our customers: IPv6 is now mainstream instead of being an exception, there are less broken IPv6 tunnels,
IPv4 carrier-grade NATs are increasing in numbers, and throughput may even be better on average over IPv6.

The updated implementation performs the following:
- Query the DNS resolver for A and AAAA.
   If the DNS records are not in the cache, the requests are sent back to back on the wire, AAAA first.
- If the first reply we get is AAAA, we send out the v6 SYN immediately
- If the first reply we get is A and we're expecting a AAAA, we start a 25ms timer
   - If the timer fires, we send out the v4 SYN
   - If we get the AAAA during that 25ms window, we move on to address selection
- When we have a list of IP addresses (either from the DNS cache or by receiving them close together with v4 before v6),
   we perform our own address selection algorithm to sort them. This algorithm uses historical RTT data to prefer addresses
   that have lower latency - but has a 25ms leeway: if the historical RTT of two compared address are within 25ms of each
   other, we use RFC3484 to pick the best one.
- Once the list is sorted, we send out the SYN for the first address and start timers based on average and variance of the
   historical TCP RTT. Roughly speaking, we start the second address around the same time we send out a SYN retransmission
   for the first address.
- The first address to reply with a SYN-ACK wins the race, we then cancel the other TCP connection attempts.

If this behavior proves successful during the beta period, you should expect more IPv6 traffic from Apple products in the future.
Note however that this only describes the current beta and all these details are subject to change.

Please test this out if you have the means to, we'd love to see test results and receive feedback!

I would like to personally thank Jason Fesler and Paul Saab for their help investigating these issues and testing this.

David Schinazi
CoreOS Networking Engineer