Re: [Doh] [DNSOP] Do53 vs DoT vs DoH Page Load Performance Study at ANRW

Rob Sayre <> Fri, 19 July 2019 21:14 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 83BDF120A44; Fri, 19 Jul 2019 14:14:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id gJAXu6ADdvPm; Fri, 19 Jul 2019 14:14:11 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::d44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 6A4871207E2; Fri, 19 Jul 2019 14:14:11 -0700 (PDT)
Received: by with SMTP id e20so30932813iob.9; Fri, 19 Jul 2019 14:14:11 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=r1oPtaiul8VDnunpzcU36ny4SjTGcEvQhylLSkf+GAE=; b=Wr6xLoZnN8/TQt4GHC7vig4lMIZNYieNv1SQRmTGOem5jPphBh5CcKyR18Os2vDsJw KYO7r/yr2PJ2aRHLBdyZek1jDsz/QJN0fbX2yL5TlK+5LU5dCR13UqAm8gcQoqmLQJss TLWRWprLWrudt0W4NXUNh2fpW9vU13n3eZvLz0QyGgQ55BwH9nkbWwTalCRin8wX6qqK Gjh8xWd27WF2ZCxe74O4NqvtYLDTeYVCDV3imsdjDLIsi1Xbgk+ix8Zdz7qfmlZbciAK B/AfOCEBgDC3Dbs/PNdXyBpLdp22JFCEb3p+dcc/3sd3OvwLeBeq9pKmbnOoF17JcaF4 +6sw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=r1oPtaiul8VDnunpzcU36ny4SjTGcEvQhylLSkf+GAE=; b=IlCmWrqJwgQw6jTZkUaBq2L8XJ6DHLeqqqBBGnuZisOpNYVIGUCGHcZqKnTDf96h6W 1M7htUHwZrAUr3Ot+dPdS3KubIylG0bJl6D1/Eo/M7L2eApLPOQ/JEXL5qvaJaFqZXp6 zqwrOC9YCAO2RP66xW+8/enagb9h4jnfbAb9aicwh3fk53df8c0I4aBOhgiI01KEQ8xj Yi/K2evzU2EcPj/+BpzrMJ+cvYYLYbNI3ctJZO/QYPm4ogrP+6UHnAlQ2URli0Vj8Xck MuhL9pGSHn1ifiB24w1aUdfzq8xiLRuSiTXnv5jMikm337+i5xkjDXO56mWTCeDiApFz Z0vQ==
X-Gm-Message-State: APjAAAV45XZVQivLBy4IlEFS3SnpWWJb2fKp2obKLULcsXvImAFFnhLV OJC+X+MEBJelnNKQL1z7iRs77qLNi/uwOO/80lQ=
X-Google-Smtp-Source: APXvYqyAMWk8HJ2W0QNKNq3G771gq6mmfg2GQql+DkgofJ3hheKMR8NSOCmpyUrVJ+XZ73NU4FLSKO18p5uq3oQyzd4=
X-Received: by 2002:a6b:bc42:: with SMTP id m63mr37462272iof.189.1563570850573; Fri, 19 Jul 2019 14:14:10 -0700 (PDT)
MIME-Version: 1.0
References: <> <> <> <> <>
In-Reply-To: <>
From: Rob Sayre <>
Date: Fri, 19 Jul 2019 14:13:59 -0700
Message-ID: <>
To: Kevin Borgolte <>
Cc:, DoH WG <>, dnsop WG <>,,,,,
Content-Type: multipart/alternative; boundary="0000000000002131bb058e0f354c"
Archived-At: <>
Subject: Re: [Doh] [DNSOP] Do53 vs DoT vs DoH Page Load Performance Study at ANRW
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DNS Over HTTPS <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 19 Jul 2019 21:14:14 -0000

On Fri, Jul 19, 2019 at 3:10 AM Kevin Borgolte <> wrote:

> > But, I think you should add the list and the reason for the range choice
> to the paper. For example, I can't tell what range you actually used from
> your description (although that might just be due to a hurried reply).
> Section 3.2.4 talks about the selection of websites:
> We collect HARs (and resulting DNS lookups) for the top 1,000 websites on
> the Tranco top-list to understand browser performance for the average user
> visiting popular sites. Furthermore, we measure the bottom 1,000 of the top
> 100,000 websites (ranked 99,000 to 100,000) to understand browser
> performance for websites that are less popular.

I see, that makes sense. I'm still having trouble interpreting the
measurements, given that pageload can be blocked by additional DNS queries
and HTTP requests. For example, could an analytics provider interfere with
performance evaluation here, if it were present in a lot of pages? Or, what
if the results indicate a performance problem in one common hosting vendor
(anything from AWS to Wordpress)? As I said, the paper is a welcome
contribution, but I find the results surprising and want to look more. I
would have expected DoH and DoT to be about the same in practice, but your
paper does not show this.

> We didn't include the full list in the paper itself for space reasons and
> because extracting the list from the paper would be cumbersome. It will be
> part of our future open source release though.

Understood--I meant that I looked for a separate download and didn't see
one (thank you for sending it along).