Re: [Pearg] Call for adoption: draft-learmonth-pearg-safe-internet-measurement-02.txt

Sara Dickinson <> Fri, 31 May 2019 09:08 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 8B16C1200B2 for <>; Fri, 31 May 2019 02:08:35 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -4.3
X-Spam-Status: No, score=-4.3 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id xxA0Q0RebopZ for <>; Fri, 31 May 2019 02:08:33 -0700 (PDT)
Received: from ( [IPv6:2a00:1098:0:82:1000:0:2:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 61EE3120052 for <>; Fri, 31 May 2019 02:08:33 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; ; s=mythic-beasts-k1; h=To:Date:Subject:From; bh=/Tl5ZK+sRdT/FTj6aTXvDSWN19yqWiv/FvKwGv4o6rw=; b=HhwrGWfLwWcPYTXcg5GDDV0tOi EL7coijM2SFgma6iU6wBsU1AGY8y6KfpCRWtOc2om6LAWHsPU7/5yBeE7GhX1rfqFQyjFjb1JG2DU jNCx4vBIRcmZXihwA0geyv1cTIA1wxQTzmBnlhlSUs3WdPpTlY7xiB++/+fE49EP8yB5SpIjR6Tb8 9vpEKEta15/KYTppT0XrsoAMBxOdgdU+wQyllMMr5qgDSI5rtl29qEY0CFtYYGBSGbSMaMJmFhxnY EvJJk4EbKSx3WoVQzpDIhkq4qwJ7ByaX6fQteSsgxPREV86rHKRxecxGfKT2C36ghPLx8B87rFWlW DnLTkcvw==;
Received: from [2001:b98:204:102:fffa::41e7] (port=54258) by with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <>) id 1hWdWV-0007X1-I2; Fri, 31 May 2019 10:08:31 +0100
From: Sara Dickinson <>
Message-Id: <>
Content-Type: multipart/alternative; boundary="Apple-Mail=_FAB7BFE8-6746-4830-8B11-1B074C333B4D"
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Date: Fri, 31 May 2019 10:08:22 +0100
In-Reply-To: <>
To: Eric Rescorla <>, Iain Learmonth <>
References: <> <> <> <> <> <>
X-Mailer: Apple Mail (2.3445.104.11)
X-BlackCat-Spam-Score: 4
Archived-At: <>
Subject: Re: [Pearg] Call for adoption: draft-learmonth-pearg-safe-internet-measurement-02.txt
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Privacy Enhancements and Assessment Proposed RG <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 31 May 2019 09:08:35 -0000

> On 27 May 2019, at 17:13, Eric Rescorla <> wrote:
> On Mon, May 27, 2019 at 8:39 AM Iain Learmonth < <>> wrote:
> Hi Eric,
> On 27/05/2019 14:34, Eric Rescorla wrote:
> > I have reviewed this document and while I think some of the advice
> > here is potentially useful, I don't think the recommendations really
> > match what's current practice or what's practical. As such, I don't
> > think it should be adopted without quite a bit more work.
> In my opinion, a lot of current practice is not safe. This document does
> not aim to set out current practice. It aims to raise the bar on user
> safety when it comes to performing Internet measurement.
> Yes, I appreciate that. However, as written I don't think it captures those
> boundaries well and so is not very useful.
> > I don't really want to get into a long debate about whether any
> > particular study type is appropriate. Rather, these are common study
> > types and so if the advice in this document is to be useful, then it
> > needs to reasonably match what people do -- or at least have a much
> > stronger argument that people should change what they do than is
> > offered here.
> Not necessarily. If it turns out that upon analysis, a lot of studies
> are dangerous for users, this document should not weaken its guidelines
> to allow those studies to continue. That would be silly.
> Hence the text you are directly quoting above.
> "or at least have a much stronger argument that people should change what they do than is offered here.
> Neither the draft as written, nor your response, seems to me to offer any such argument.

I’d like to support addressing these concerns in the way that Iain proposed i.e. with a clear separation in the document:

- Retain a ‘Recommendations' section which remains close to a description of desired behaviour for Safe Measurement practices

- Add a ‘Current Practices’ section which can cite studies such as the ones mentioned with the analysis of if and why they fall short of the recommendations and the specific risks to users due to that difference. Where possible suggest modifications to mitigate those risks.

I agree it would be useful for the document to capture acknowledged or existing practical limitations in more detail and these can be reference from the recommendations. However I would argue the primary goal should remain providing strong standalone guidance on best practices for these and future studies.