Re: [Pearg] Call for adoption: draft-learmonth-pearg-safe-internet-measurement-02.txt

Eric Rescorla <> Mon, 27 May 2019 16:14 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 45E6F12016C for <>; Mon, 27 May 2019 09:14:37 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id wsNM6aAykQaQ for <>; Mon, 27 May 2019 09:14:35 -0700 (PDT)
Received: from ( [IPv6:2a00:1450:4864:20::132]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id AEF44120074 for <>; Mon, 27 May 2019 09:14:34 -0700 (PDT)
Received: by with SMTP id m15so11855305lfh.4 for <>; Mon, 27 May 2019 09:14:34 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=9L8ynu6EkvjnaUxSPWBkTP7yv0Q5upJJum7HkNAesjM=; b=kWGZ/YjoggzUORp+a92lw4JMpysN/Z92ADUwQ3A/2Alc/jSWjQ3o10Ysv0s2GkcdVo gr7ZRTzZ4IkJls4BeorjUx5dmp2DZvlOO+xR2DOXTKdNorWvUiPQA8wf65lhgqwBaV6n yxf4dD0n8fsoLpydfxty6cLo+xqoJ4XxAAJCPeQ1P5Q0CXztNUEyEVTdEQxhfqMn1lkD cgdNkhiOn/hPdYHr0lfUzCAkE+6ZmsIn3vNUOHHU8OuN1J6u7rQE48F8ckER4HABSYAX 8h6sMQ7BWMYBVvK2mwgKHQSdYw5TKLjuCnKtpz0JMFyo3sjoaEMlGXfoK9WgX+yLBJjw ot0g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=9L8ynu6EkvjnaUxSPWBkTP7yv0Q5upJJum7HkNAesjM=; b=L/C1gNAnquIgz0XPOtOxsRIjSwcL2V+ONkiL2MCqW3WgE6HpqoTzj6t3aGHBZXg1s4 eBkAIjfy+tssr0v9tiNpqXrIr5VBCYGNNX3vCX9OykML55KzAgQW/HR8GC/gWe57ePPo VCXNpcnaY2dA1EOnJWjeA254iCasKNn57kDvqNDMAft1g9PcITrFu+v6KsRYV+Vt2upG +kGnDYVT9B9uPdmd4IDzUCAedYWpTl5ruJ58osqrUnbwT+52b6LSRpGnKtjTmP5cgbM7 HCJoXpJJK/ZdCCdaAChVeqEOLAi6L8yXnCfufQ7fHud9ADfkqDn0Do+HpKOp+K2CJb8g Ur/w==
X-Gm-Message-State: APjAAAV/ZJeu1OOlUGsKaQi88Jc2aIBGMzk1Og8aB5/enX1GyoOoRoxe 3TISpOy2bS1LFZ/FjRi477LlMkJAsJacgTT+moQkSg==
X-Google-Smtp-Source: APXvYqxYwErvbF/3NOWFS+x5LPUyR5HIhb5hAo+WDNeTuHf5AyAYUH5IdhPC9iktLYjtk5uGjIMIC2d/qIogsezYQjQ=
X-Received: by 2002:a19:9c8f:: with SMTP id f137mr61080271lfe.94.1558973672809; Mon, 27 May 2019 09:14:32 -0700 (PDT)
MIME-Version: 1.0
References: <> <> <> <> <>
In-Reply-To: <>
From: Eric Rescorla <>
Date: Mon, 27 May 2019 09:13:55 -0700
Message-ID: <>
To: Iain Learmonth <>
Content-Type: multipart/alternative; boundary="000000000000fb7a270589e0d765"
Archived-At: <>
Subject: Re: [Pearg] Call for adoption: draft-learmonth-pearg-safe-internet-measurement-02.txt
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Privacy Enhancements and Assessment Proposed RG <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 27 May 2019 16:14:37 -0000

On Mon, May 27, 2019 at 8:39 AM Iain Learmonth <> wrote:

> Hi Eric,
> On 27/05/2019 14:34, Eric Rescorla wrote:
> > I have reviewed this document and while I think some of the advice
> > here is potentially useful, I don't think the recommendations really
> > match what's current practice or what's practical. As such, I don't
> > think it should be adopted without quite a bit more work.
> In my opinion, a lot of current practice is not safe. This document does
> not aim to set out current practice. It aims to raise the bar on user
> safety when it comes to performing Internet measurement.

Yes, I appreciate that. However, as written I don't think it captures those
boundaries well and so is not very useful.

> I don't really want to get into a long debate about whether any
> > particular study type is appropriate. Rather, these are common study
> > types and so if the advice in this document is to be useful, then it
> > needs to reasonably match what people do -- or at least have a much
> > stronger argument that people should change what they do than is
> > offered here.
> Not necessarily. If it turns out that upon analysis, a lot of studies
> are dangerous for users, this document should not weaken its guidelines
> to allow those studies to continue. That would be silly.

Hence the text you are directly quoting above.

"or at least have a much stronger argument that people should change what
they do than is offered here.

Neither the draft as written, nor your response, seems to me to offer any
such argument.

>     The experiment uses an online advertisement campaign to deliver
> >     the test code to end systems. When the end system is passed an ad
> >     that is carrying the experiment the system runs embedded Adobe
> >     Flash code. The code is executed when the ad is passed to the
> >     user, and does not rely on a user "click" or any other user
> >     trigger action. The active code interrogates one of two experiment
> >     controllers by performing a URL fetch. The contents of the fetched
> >     experiment control URL are a dynamically generated sequence of
> >     four URLs. These four URLs are the substance of the test setup.
> This is great until you run on a user machine in a country which has
> some censorship/monitoring infrastructure in place that has
> misinterpreted the URL as some proscribed content, landing the user in
> trouble.

Well, this isn't my study, so I'm not going to account for how it's
but it seems like you're missing the context here that this is served
as part of an ad, and that the way that ads work *in general* is to load
content from remote servers. It's not clear to me why you think that
the URL that the study checks is somehow more likely to be interpreted
as proscribed than the ad content that would normally be served.

With that said, we also do studies where the browser loads specific URLs
we provide it, and we do try to make them innocuous.

> At the very least you may have generated costs for the user's bandwidth
> there, which are uncompensated.

Again, this is served via an ad network, so the user would be loading
some other ad that used up random amounts of bandwidth instead of
the ad that runs the study. This seems like a pretty weak argument.

> > It's worth noting at this point that the Web is a platform for running
> > remote code, and by browsing you're opting into that, and ad studies
> > just leverage that behavior.
> Tell that to the 1,543,235 users that have installed NoScript from
> Firefox Add-ons. Now you could say that they have opted out and the code
> won't be run, but the way you've phrased this makes me think that
> actually you just haven't understood the wider range of Internet users
> that exist.

I'm quite aware of NoScript (you do know that I work for Mozilla, right?)
but this is a trivially small fraction of the browsing user base. As a
matter, the Web is a platform for executing code sent by the server,
which of course users can disable if they choose.

> WRT to the first point, as a general matter, modern browsers
> > auto-update, so the user has generally opted into regularly getting
> > whatever new code the vendor thinks makes the best browser.
> Say a mobile phone vendor wanted to test out how its camera was doing.
> It ships you an auto-update that sends back every photo you've taken to
> work out things like light levels, noise, and what might need tuned.
> This is for the purpose of improving the camera.

I think you've misunderstood my point. I'm not saying that the browser
vendor  -- or a mobile phone vendor -- can just send malicious code to
the user (and for avoidance of doubt I would consider reporting every photo
to be malicious).

Rather, what I'm saying is that browsers already do auto-updates, and so
it's not
useful to have a much stricter set of rules for experiments/measurements
that are delivered to some users than vendors do for code they would deliver
to all users as part of an update. I.e., it's not that useful to focus on
rather than on overall behavior.

This draft is still in its early stages and there's a lot to flesh out
> still.

Understood. I don't think at this early stage it's ready for adoption. Once
you've fleshed it out, I'll be happy to take another look.