Re: [Pearg] Call for adoption: draft-learmonth-pearg-safe-internet-measurement-02.txt

Iain Learmonth <irl@torproject.org> Mon, 27 May 2019 15:39 UTC

Return-Path: <irl@torproject.org>
X-Original-To: pearg@ietfa.amsl.com
Delivered-To: pearg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 635A8120199 for <pearg@ietfa.amsl.com>; Mon, 27 May 2019 08:39:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FY6YZ2iHT0TL for <pearg@ietfa.amsl.com>; Mon, 27 May 2019 08:39:44 -0700 (PDT)
Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7354D1201B4 for <pearg@irtf.org>; Mon, 27 May 2019 08:39:44 -0700 (PDT)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 7A08D2229C for <pearg@irtf.org>; Mon, 27 May 2019 11:39:43 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Mon, 27 May 2019 11:39:43 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=50RQJo E6wWJFTAl05M/5y+d9lFkJBwF4bI7CBe5Tozo=; b=ETQDkshVCNiBCp90Q8BeD2 D/oHfvN3OE2NQqLqZSN6X/2yc1PkimeUE30AVmo3yP5iPGlDjbi7NUwr4AWN+9pi wIexo72BCeR9TeNxwmG3yLlrEE9hkueVHCz/OsNnEA23pCtqim/yQYDjiJBWJCAI 7UVYlgkt2FFLie4N1fd95FkPgYQ3aoeHkADmhzPRT/350PygNDEdae39Q1b8jJQA Bg7UFLPz3p2HI+HlcYZo8UV//gz3TnpRFw0UO9PapzXCT5PgCndT41Z+8b7cu/ud 6pLMcNi/NhT4V++di5n/HN+jLPvH7tlhNw0z30mZnRslLlA5/zMejbwe7vk/IiNQ ==
X-ME-Sender: <xms:vwTsXC0r0gqlJF1GrE0hKZvi3MrHYMKxh3zcGoRFsub4WIKRLvuq4g>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduuddruddvvddgleefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefvfhfhufhokffffgggjggtsehgtd erofdtfeejnecuhfhrohhmpefkrghinhcunfgvrghrmhhonhhthhcuoehirhhlsehtohhr phhrohhjvggtthdrohhrgheqnecuffhomhgrihhnpehmohiiihhllhgrrdhorhhgnecukf hppeekgedrledvrdeguddrvddtudenucfrrghrrghmpehmrghilhhfrhhomhepihhrlhes thhorhhprhhojhgvtghtrdhorhhgnecuvehluhhsthgvrhfuihiivgeptd
X-ME-Proxy: <xmx:vwTsXKs5Ym5Qe2HCL0xzejL_Nq6KAFHsSYj9ZSR-Ed3VAZN-wJl7Zg> <xmx:vwTsXC_lHekPMgU62qPLkm7AY4BTgPLP_DVc9O-_42p_I4yBNE7aAA> <xmx:vwTsXMipRnVrIwBRY_up7lOqM1-i8h9gGKJ89Ps2I6XFQTduTSaAng> <xmx:vwTsXCFNvsZy6h2vL8cliVjZYQHI0CTSU9rQp_KXcz2-zu076WzJKQ>
Received: from [172.22.152.130] (shiftout.plus.com [84.92.41.201]) by mail.messagingengine.com (Postfix) with ESMTPA id A554C380089 for <pearg@irtf.org>; Mon, 27 May 2019 11:39:42 -0400 (EDT)
To: pearg@irtf.org
References: <155800230363.19745.1496619794666703625.idtracker@ietfa.amsl.com> <6d285cf5-4c38-b6ef-66dd-a0fd1c207268@torproject.org> <AF390529-6D66-4679-9572-83BDB1753DEE@sinodun.com> <CABcZeBNNh3pwSTiF7QX3eoeZkoWi0YTa63YBYeiSEfgHTQeFLQ@mail.gmail.com>
From: Iain Learmonth <irl@torproject.org>
Openpgp: preference=signencrypt
Autocrypt: addr=irl@torproject.org; prefer-encrypt=mutual; keydata= mQINBFZp8zEBEACxOYriD+tEuc3Wpnbh+GGnyiaLEMABBrfn6JlDQphbBq/YTz9M9OPkttjx hLL/yrxlM1nD69XbGKQ9gIL3LEgOz9+OdivPbN+Q5iNMqk/WCQUqd3bCFbbsn1yvoTumFy9S 9kYX45Db3jRJoN/Nye6Stf7KKPxHxot14iY+PUR/5Gx5KbeWVKfDtQejGnhxQD73KjrX4wds BAaxnQ7KbjQyUf+IxE+8qSDcyTP+pPqxspVzx+eFqsW5+kK1eJMHxJmY/KsAs6IsGf5lvyDJ JECc2iE0mFS6vc14lGcD7BAYMPRnvlK3OcDlbdJS3ZU0LQu3/AplM7cNcesq2Btm06OUTsbj 10ZiyLi7Q0WZRuUbn7t3jOQVyOlNfjUpJhKPMMobBL2R0KzcptJbUrKc08wZD/TPaXuHKWAE JuA6kFMXtHhV8Qhxz5/d2KUA8ex+zpVd2xSR6q4llcYu1w8zHZtLN+YKSmjjKs+AjiTrCMYs OYxt4cwxuaIIhBNvCC9WqZOxHX7YHmpVcSV6K9Wwhk9mVIU3Ii0G2HWs6OQ0vIueCDGMEdVk ig/a7cVlfXNz7WuaXuhOJmHz6d6Yk4dFn5mLbEY9cZhBxf5hjCwtp9b6v+ueuptfcnOd+38G 9KH6NyHKZyS4jcd3E6Dp0+9Isbl/EohjPCujevoW3/DlT08OKQARAQABtDRJYWluIFIuIExl YXJtb250aCAoVG9yIFByb2plY3QpIDxpcmxAdG9ycHJvamVjdC5vcmc+iQJUBBMBCgA+AhsD BQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAFiEEqPe6UEHhMzOcuhaWdtWAk/VAq80FAlxe8QQF CQbiS9IACgkQdtWAk/VAq80isw//QGWLLb6wjPW9K/seio9dSVM3gDk6jU7/FhlUuz7oq/6H gPrl7jrxH9PY5jAlkWCK5iPFWpFczUKC+AzzuZQt8VXQp5iYllm1KRrZA6mxpx085Yo3cSLp H9B55IHENo5RFdXMgXZ2aNc6ENG31vB6IZDHXjw9fUxtqyB7fCbIhFtO9pCqwnH/WipISdrl P5/gllUJKsvZiQj8vFfOe2rEHfldhqXxN0ZFvQtQMy0SipqfY2eM47HoDLZ2L5U1iK3iG1fa UWN059Y92zcc9B/AFqHK9xFMObUlWEvqhCGKgullDp4mVF4/OPAC/l02IS85BeFxWbV/HxTY AeoEzO9qrWVitCPTuFC4TQxJq4TbVr/jriFugmbvlE3IIXz5OjqTpyJZw+A8jJ3lwF+V5z21 LdJpEbh+fBBz6tl99Hnn7rjN9WhhsH+BvSC9fCnxs21h+oiGw9zom6blFsLW1yP7mNX2ZRJR DmQs+7kPL4ZS8XekNN82UHXE2pd9M2fSEi5dHK0R9xKJmMSNi/cnEZp1SRx+e+FFDZwzxiFO R30sDEEf+Z1MB/7o5uQXtw59W5Q7p0KycJMfacnb3dulOhFF4Qu1w1K3Iqefq8cQpnlmIGV0 dYT6juPyDlRW19LQVQSTejkLAbkBuXVE0T6dAYVyudJ2lVa3aDvQV5PqL1Y5tdK5Ag0EWmt/ jQEQANhwgR5fq1Slk0T0Sh7Tkc5LjanQH1VuNTxOE7wzcYXrgva2ic4zdhLc2QHP9G4kz0AY /oLxtw5Sj/IMtdzBHKDBXgiqBmvkrz7mOZSQQh6K/JBcKau4MorzB80Z/Z6XQcfFKQh61+8e Im/I7AnJFUELxDe8CYmkJYKVJS9b+i9RNFvXAsamOkClcyXqPYBB9bBI8QlZHweTWDsXJqul xjHLjCWOQyJxfl9xFmlSJtskrLEmprw/PaOtXglrz/2vadn2lL3ack9V1ux9ALa5q8Oc6dnx vA/W3palFpdBoz75FckhRfliYNfdCpgo86w00bJvbJ1f6XfBIBYvsAvrIRWIkMYEmhTYm4xG gEDWz5CvGuuzkF7kJwfPdj1RgVe80JHr9OJc8ZrafBb/p35ldLxmhDi16j2VOj58CKwpi3En nMOMPteDxmCZrWeYKhfwgVAP4/zc4+9vmTp5Wpze00DnQ9MEinmJ2bxf6TyQP36Bf/8zgJVs rpqMOaumbwwmcUh85Q1PXneVWX2ryEv1I7xVMpW4APZDeOMiiHb+EpXPLTPBvXTerd/Pwzb7 WLKNpZI4tl/vh88INdrs3xZd174skDJNaNSnqzUP2kNYcxWZSH/FP4AH1IXvIxwIr9g0RpvY oX55ALOSQOdZ9ioCjf9x9mvnBsehcoEPrgozljzpABEBAAGJBHIEGAEKACYCGwIWIQSo97pQ QeEzM5y6FpZ21YCT9UCrzQUCXF7xLQUJAuC/nwJAwXQgBBkBCgAdFiEE/ps4MHN0fw3t6Aud F4mIfdjVvF0FAlprf40ACgkQF4mIfdjVvF0NJBAAuAVw1aY3IUtp8ZRI5xuhieVFDbye7z86 gP5KW8cAMKTpuowD7E3Rm31R2Pb5dX+vZlunSTBPuzRVnVNd6Kj7CR8BR426L48ogSUfIh4V IWiYxXn4DqqNGow+saPI8VwJ/y80NSb2v+qUdv7us052vs9UBBnI/zl8XrnUNix+B0g+jekT ENEVD9fNJ+9YE9cn80wLGKBx047id1IqgOeJShjE13WRj7sa+LhB8FtoOUeXduQ1sJBZTVJq XW9v0e2PNc37cc8VATrA+3HNSQJY1PETDxjJ1TW8gALpZQrfR+FDu/d2vALdULw/djCeIpl0 dr79isd6mINBKTkEcR8XPQjDUnrnO1U18YETo71N5Z6HpvaV5/XWLUl81wovddpVRvwbwuFP bJE506UiQJrthPKi6s670VBV6u5I+5+h6D+CHp1xp0Bq8PLz0EwqjZwOiLAByBT/ryD2ZY8+ 2rP9gphnEBcCtyp6Fz6Q0KyutpCPv94b4OU0eUTvl3Nroq2GWGGHAV+8r1uXpqdNSHxYmQgy b3H1dvObmE37L3BWrVNzeFb0USNiV04up/55QUrqrz7kLdBO0ZVMVLU3j7e48KcbJG2yzNHZ dnQa+ZMMIRFtm8vzSt+jEIwodi6nOeBgEiqODF8TZPbTc6183ErFoYc/FuhkUh0aR0MLZ92Q bPEJEHbVgJP1QKvNZ94QAK9KrF8hpdlSLTdlQptQ+pfKH2AyC1zyyrSxoxh56inKMgETXLVJ WmEYc1OCLWktUU0qQkCzo6umR7TNVZjx86L6jKKlFTptulQyFVaeaOU8DgFuiqnDXWn5E1cu bp1KCVU9TpBqIC7hN4ifJeY1lVoRGR+x+PCDeYgT+birtIanK5mqKzx74pbLeTTQZALqD8Hr UouxO41NyOGMv4BgZrr9z2ATIEIHZDLihKMEbF6LMjC6oCOgf51x6gV8doVw4ySNc3i/mk6h ztLQBhEc1q5SWfNgXcPDnzzoYsVJMSmV6yMCw3OEwtfZ0BYPQ6hihShOBKVQEHVnGbMIT4md V1S/axXDsLPFLkq1CxZ1L7N8Y5n2lUH00iSo+yWs7lhvt9WKl7sTewjXtibgrAC89uKvIyHe bA0GO9wojlwGAQboCMxZkkADgpHkDNvY675QLWaEs7+JJq4iWwojFh6QrDuOZmR3e9a5xZDb nQScTuPWvhZUtq7ZNZxmjPDdSUomSHe/ahBd+FHNKbgumN3PuXitFTg7LozIZV05oVv9arMs 9KyKGOS6rbim5C3Nx+MitMdr2Vh3tRfHugy0iWFSX5ITkG/RH4fMOzJMgMz/SftZMwMOU4J/ RBiPRyeaMBMqwomg/7o+NjulWHrUlAX+DAsyVtV+K9OLkglikNQtkG0MuQINBFprf6sBEADO ANf22so7uoGcvok2TM/T8BHI5+TqHEc4hVe+JGGJ1ZnWlgtGmpOs0fOQj3WAgGI0ZmTqMuoz KF/K9ljbjaMXsLD+JIBTD4rINy60VX2zHhmWhNaOcJvq+wbuHx0tMbhqsTStGnSkvRhH61nc MqVqlTTTLVQQSxKl9D2l7ZGwEPLHRFlydTOOix+F+Y1ehxYLVaPkaycs8wvgjYsDLo3T8Tmu OL+rcEfvxJ6lT2V5I51xqievqoBazAfXvA8FW/0G+Z9LUJmViOVluWg3xjP8okKYgOkOeX00 vMBCVaiEA08oaxY0ebS7uBEgppjWSwn+WAhB+6spd67d4W+DmAnM262lxFMhVYhXpfeV9zyg ULQOofdE6xtFkaxr/y8xQ4Bf7zX8ko6X9aFQFB/vc+zUtjzjg4VaQvWrThjaHlbEKR55MDxJ u2T9S7g5bR4zxZNv36gwlIdmx33a1AeR1nGcWa/7OtoS53+lUwyFVWLOnucqKh71Y38AAMd5 L9Fsb+ArQem71knEUTC+HvBGkPb2Y2PzGnnzhZyC8zgE8AjVD0wB+RMDNI3+fIW6biKAHDqr S6ZCVkzJ1R9nOjXMHRYZ5qlG+rCOeu6Jp4yNwp46z4PqiiLJ9NtmdNttLCEn5PDVF3g9g811 JcadvFVH1ZELoDGWMg3Q+QOHQBFYj7cj2QARAQABiQI8BBgBCgAmAhsMFiEEqPe6UEHhMzOc uhaWdtWAk/VAq80FAlxe8S0FCQLgv4EACgkQdtWAk/VAq81y0Q//YAbM50BQSjWAyVk52AMv PDEalSOau9jd1zllgurs1+BD4eWkNOJtN9BKaR5pd5VsjYPhL6ypUv/lRsyDPM3h7knuCZmI XYSBkloDqutZ020kR1jNhIZLL3Nma1xqh+oMsB7M+1AcT+Nlez74WgYjNyb7uuFWr99pUBuJ KbVjBq772Jjz3U0q7sa35wQvLaMC+AG/8L+e/fos6jgJhvIq2QeVLuVyeKVlEEO6tpe07Q+e mdCNu4l5eNbWAuvOgvtbVCLz5C4nLNZzr27u0thXzVhZ8ovAzdQTx6d6X4YI3fwBhPxXpqkg eTo3+B8D5lKUgkSZ9xTo+JP9yjCWwytezTL3oElL5LBVlYxdnpDWwOx8rCIgi5OlfVwU03/P O7IrEkqU5jvhZYT4c5/ktCx99kJcYwPUbT4wuhI1JnXyILrtmfC9a0vK8hpIUP4HU9GJTnVH Bl/XZZ6OJWEmSlJWm8KbageOcfLewc3BdWFnY6k5TmrzsFbamaBOgZmJgkNgylEyTjxVnLTk i5wciAp+N9K+tOP5FvgR943RSESZrxHo+XAV+BAK6K6Oae/jlrzGzNLAyJKWjefhyGL3AOru EBhSQaRemp/IUmyKREfowz3f8Lw2NFyJuWGzDgo9/1fmqmZ1JegtfWw0uPHB/rooajODBaol obxU65Qt1SOk2Ws=
Organization: Tor Project
Message-ID: <628af973-abb5-40f3-637f-b7a1a84c70d0@torproject.org>
Date: Mon, 27 May 2019 16:39:40 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1
MIME-Version: 1.0
In-Reply-To: <CABcZeBNNh3pwSTiF7QX3eoeZkoWi0YTa63YBYeiSEfgHTQeFLQ@mail.gmail.com>
Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="2r3ElaJiclwhIzAtb4oRBeCWvpGgHqbQr"
Archived-At: <https://mailarchive.ietf.org/arch/msg/pearg/ZgAEuQZ-r7UsVocJNFLjwIF4CGY>
Subject: Re: [Pearg] Call for adoption: draft-learmonth-pearg-safe-internet-measurement-02.txt
X-BeenThere: pearg@irtf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Privacy Enhancements and Assessment Proposed RG <pearg.irtf.org>
List-Unsubscribe: <https://www.irtf.org/mailman/options/pearg>, <mailto:pearg-request@irtf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/pearg/>
List-Post: <mailto:pearg@irtf.org>
List-Help: <mailto:pearg-request@irtf.org?subject=help>
List-Subscribe: <https://www.irtf.org/mailman/listinfo/pearg>, <mailto:pearg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Mon, 27 May 2019 15:39:47 -0000

Hi Eric,

On 27/05/2019 14:34, Eric Rescorla wrote:
> I have reviewed this document and while I think some of the advice
> here is potentially useful, I don't think the recommendations really
> match what's current practice or what's practical. As such, I don't
> think it should be adopted without quite a bit more work.

In my opinion, a lot of current practice is not safe. This document does
not aim to set out current practice. It aims to raise the bar on user
safety when it comes to performing Internet measurement.

> As background, here is a partial list of the kinds of studies that
> are commonly done:

Thanks for these examples. I see no reason why each of these could not
be done safely. I have refrained so far from relating to specific
examples to try to keep the guidelines general but an "applied" section
may be useful to readers to better understand how to use the guidelines.

> I don't really want to get into a long debate about whether any
> particular study type is appropriate. Rather, these are common study
> types and so if the advice in this document is to be useful, then it
> needs to reasonably match what people do -- or at least have a much
> stronger argument that people should change what they do than is
> offered here.

Not necessarily. If it turns out that upon analysis, a lot of studies
are dangerous for users, this document should not weaken its guidelines
to allow those studies to continue. That would be silly.

You can do more dangerous things as long as either you have the consent
of the users, or you believe that what you are doing would be expected
by the users anyway. Neither of these things removes risk, but ensures
that the user is taking an informed risk.

> S 2. CONSENT
> The text in this draft leans pretty heavily on getting consent,
> either direct consent (including all users of the shared network)
> or "proxy consent".

This version contains a first draft of this section. Perhaps the
"implied consent" portion could do with more work.

>     The experiment uses an online advertisement campaign to deliver
>     the test code to end systems. When the end system is passed an ad
>     that is carrying the experiment the system runs embedded Adobe
>     Flash code. The code is executed when the ad is passed to the
>     user, and does not rely on a user "click" or any other user
>     trigger action. The active code interrogates one of two experiment
>     controllers by performing a URL fetch. The contents of the fetched
>     experiment control URL are a dynamically generated sequence of
>     four URLs. These four URLs are the substance of the test setup.

This is great until you run on a user machine in a country which has
some censorship/monitoring infrastructure in place that has
misinterpreted the URL as some proscribed content, landing the user in
trouble.

At the very least you may have generated costs for the user's bandwidth
there, which are uncompensated.

> It's worth noting at this point that the Web is a platform for running
> remote code, and by browsing you're opting into that, and ad studies
> just leverage that behavior.

Tell that to the 1,543,235 users that have installed NoScript from
Firefox Add-ons. Now you could say that they have opted out and the code
won't be run, but the way you've phrased this makes me think that
actually you just haven't understood the wider range of Internet users
that exist.

> As another example, Mozilla's Shield Studies system
> (https://wiki.mozilla.org/Firefox/Shield/Shield_Studies),is generally
> opt-out, with a specific opt-in for when the study collects
> more sensitive data:

Exactly, when a study would collect more data than a user might expect
would be collected, you ask for informed consent. When a study is
looking at non-personal information then it is OK to go with implied
consent.

> There seem to be two core issues here:
> - The effects of various changes on the user or the network they are
>   on.
> - The data collection inherent in doing the study.

This is a good split that I had not considered before.

> WRT to the first point, as a general matter, modern browsers
> auto-update, so the user has generally opted into regularly getting
> whatever new code the vendor thinks makes the best browser.

Say a mobile phone vendor wanted to test out how its camera was doing.
It ships you an auto-update that sends back every photo you've taken to
work out things like light levels, noise, and what might need tuned.
This is for the purpose of improving the camera.

> Similarly, on the topic of data collection, browsers report back quite
> a bit of technical data about their behavior. In both Chrome and
> Firefox, this is on by default, though you can turn it off. As
> suggested by the quote above on Shield studies, many studies just
> gather this kind of data (and often data that the browser would
> already report back) and Mozilla, at least, has a pretty
> well-developed framework for determining what kind of data requires
> what level of consent
> (https://wiki.mozilla.org/Firefox/Data_Collection).

The question is not about whether or not you can report back data. The
question is about what safeguards you can take to protect your users
from harm. You can collect data and report it back safely. We have
techniques for doing this in ways that respect users. The only reason to
not use them is if you don't actually care about protecting your users.

This draft is still in its early stages and there's a lot to flesh out
still. This feedback is useful and I'll be incorporating some of the
topics, but I feel that you've maybe missed the point of it.

Thanks,
Iain.