Re: [v6ops] An Update to Happy Eyeballs

[David Schinazi <[e-mail]>]

Mark,

We have operational data proving that asynchronous DNS is not "for no good reason".
DNS queries fail rarely (it still happens) but they often can take several hundreds of milliseconds longer.
The user shouldn't have to wait that delay for their webpage to load, and our data shows that this timer
fires regularly in the field indicating that users would have been needlessly waiting for their content.
Why should we artificially delay the user's experience for an A response if we already have the AAAA?

Erik,

Thanks for the suggestions!
- we'll add a section about problems that Happy Eyeballs do not solve
- regarding the increased retry timer, this is somewhat covered by historical RTT information.
    I don't think failures to reach the first address should harm the next ones, as that defeats redundancy
- I do agree that the main downside of Happy Eyeballs is hiding failures,
    but enforcing clients have a reporting system seems uncommon for IETF protocols, do you know of any?
- Regarding TFO / TLS 1.3 0RTT I think the requirement is that you MUST NOT use those
    if your data isn't idempotent, as packets could be duplicated in the network. This is orthogonal to Happy Eyeballs.

Thanks,
David Schinazi


> On Mar 13, 2017, at 19:02, Erik Nygren <[e-mail]> wrote:
> 
> It's great to see an updated version of this guidance!
> 
> > There is zero reason for making async DNS a MUST.  
> 
> The bare minimum is that revolvers must do A and AAAA lookups in parallel.
> The current behavior of some stacks is to do the AAAA and A lookups in series. 
> This means effectively means that adding IPv6 connectivity to a client adds an 
> extra RTT in for almost all DNS lookups.  
> For example, see section 5 in:  https://www.akamai.com/us/en/multimedia/documents/technical-publication/a-case-for-faster-mobile-web-in-cellular-ipv6-networks.pdf <https://www.akamai.com/us/en/multimedia/documents/technical-publication/a-case-for-faster-mobile-web-in-cellular-ipv6-networks.pdf>
> For clients such as mobile device visiting sites with lots of hostnames,
> this can have a very substantial performance hit.  This also shows up
> in some RUM-based measurement reports making IPv6 look slower 
> (due to clients with IPv6 spending more time doing DNS lookups before 
> doing page loads). 
> 
> Doing the lookups in parallel but not waiting for both responses is better than serial,
> but still has a perf hit (whatever the recovery time plus an RTT) when 
> the A or AAAA lookup packet is lost.
> 
> Some additional comments/thoughts after reading through the -01 version:
> 
> * It would be good to add a section on failure cases NOT detected/mitigated by this form of Happy Eyeballs.
>    Even if not covering mitigations, it would still be good to discuss them for awareness.
> 
>     - In particular, PMTUD seems to be the most common.  ie, if there is a PMTUD issue between
>       the client and server, then the connect will often succeed but the connection will fail to function.
>       I think most large-scale IPv6 server operators (plus many of small ones) have broken this at least once.
>       One client-side mitigation for TCP might be for the client to offer progressively smaller MSS 
>       as it retries different IPs within a protocol family.
>       (I don't know if anyone does or has tried this?  There is the server-side pmtud probing feature.)
>       For UDP protocols, using full-frame packets for the SYN and the SYN-ACK (as QUIC does) seems
>       like one approach to at least detect breakage early, although QUIC doesn't key have a PMTUD 
>       mitigation solution AFAIK.
> 
>     -  Servers that return different content for IPv6 vs IPv4 (eg, "404 not found" due to an unconfigured server on the IPv6 side).
>        "Don't do this" as advice to server operators is likely the best way to fix it as hacking around it on the client side is unhelpful.
> 
> * It may make sense to recommend some form of back-off in the retry timing.  Rather than a fixed value (eg, 250ms), adding
>   an increasing time value with some jitter into each retry may be safer in the cases of overloaded servers
>   or a network connection that is borderline near the retry time.  I've seen congestive failure and lack-of-progress
>   scenarios from having a fixed retry timer.  For example, with servers that do FIFO queueing of connections to accept,
>   if the queue becomes longer than the retry period then all clients fail to make forward progress and you reach 
>   congestive collapse.  The same can happen with links that become high-latency, however.
> 
> * It may be worth adding some guidance into reporting and visibility.  I'm not sure how?  
>   One of the big complaints against Happy Eyeballs is that it masks brokenness (latency spikes
>   but things keep working so no one complains enough to fix the root cause).
>   Having a recommendation that stacks or applications at least keep counters and telemetry
>   on failures may at least make it more viable to debug?
> 
> * It would be good to provide guidance or a reminder around protocols that send data along with a SYN
>   or an initial flight  (eg, TCP Fast Open / TFO and TLS 1.3 0RTT).  In particular, you likely
>   want to send this ONLY on one connection attempt (eg, the IPv6 attempt?) as otherwise 
>   the operation may be executed twice by the server.  This may cause undue server load
>   and for apps/clients incorrectly using TFO or 0RTT for non-idempotent operations 
>   it could cause duplicate actions. 
> 
> Thanks!  Erik
> 
> 
> 
> 
> 
> 
> On Sun, Mar 12, 2017 at 11:53 PM, David Schinazi <[e-mail] <mailto:[e-mail]>> wrote:
> Hi everyone,
> 
> Thanks a lot for the comments and feedback.
> We've incorporated them into -01, please let us know if they were properly addressed.
> https://www.ietf.org/internet-drafts/draft-pauly-v6ops-happy-eyeballs-update-01.txt <https://www.ietf.org/internet-drafts/draft-pauly-v6ops-happy-eyeballs-update-01.txt>
> 
> Regards,
> David Schinazi
> 
> 
>> On Mar 10, 2017, at 14:54, Philip Homburg <[e-mail] <mailto:[e-mail]>> wrote:
>> 
>> In your letter dated Fri, 10 Mar 2017 09:29:55 -0800 you wrote:
>>> We can certainly soften some of the language to make it clear that if 
>>> your system has no such option, you are not necessarily out of spec, but if su
>>> ch an option is available, we believe that it SHOULD indeed be used. This fits
>>> with the Happy Eyeballs paradigm: if I am waiting for one of the DNS response
>>> s to come back, I could have already made my connection in that time, getting 
>>> the user the resource loaded more quickly.
>> 
>> If the DNS requirements can be toned down to the point that an application
>> can use getaddrinfo if that fits the application, then that's fine
>> with me.
>> 
>> 
>> _______________________________________________
>> v6ops mailing list
>> [e-mail] <mailto:[e-mail]>
>> https://www.ietf.org/mailman/listinfo/v6ops <https://www.ietf.org/mailman/listinfo/v6ops>
> 
> 
> _______________________________________________
> v6ops mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/v6ops <https://www.ietf.org/mailman/listinfo/v6ops>
> 
>