Re: [Pearg] More comments on draft-irtf-pearg-safe-internet-measurement-08

Mallory Knodel <> Fri, 27 October 2023 18:12 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 983E5C15109C for <>; Fri, 27 Oct 2023 11:12:17 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.106
X-Spam-Status: No, score=-2.106 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id RbPEmzqcRVKU for <>; Fri, 27 Oct 2023 11:12:13 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::433]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by (Postfix) with ESMTPS id 848D0C15108D for <>; Fri, 27 Oct 2023 11:12:13 -0700 (PDT)
Received: by with SMTP id d2e1a72fcca58-6b26a3163acso2238499b3a.2 for <>; Fri, 27 Oct 2023 11:12:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=google; t=1698430333; x=1699035133;; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=pe7QnIrkyuFa8cCuOWTP9TbcqfvkJ/Y1QgV3EBvc3Fg=; b=lY4PMcD6EHYJFffVa8L5U2yxImhXeQuIGGmxkcqrHj3UZkftsZPe/D/uwRfIlB4DsR QqAWeuMZvmV68jfjoCa8ajEvyeYecCVFe0mK1KXi/6g2KlhV9uNeOvIsRR59Fs3+Xa+K 7UuSU51HmksFKAataU+P1pWR4nuZy2WkGtWcA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20230601; t=1698430333; x=1699035133; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pe7QnIrkyuFa8cCuOWTP9TbcqfvkJ/Y1QgV3EBvc3Fg=; b=iPt4ZcgK74ZkwPigNUMiB4Xco7KYGx8YhFPtHdfLlinylAY7BOUjakFWSwbc/XGpG7 rMvSkAQNV1yahX+frS5unz2MVIVUqPSHv8dJI82G2IwCT1RIKWE/9C1zagO1JLo4GRDH jeV1zwqktwCu9pNJucY5B+xMyoxIFkJ+UyF0ayeB7mbD5RCL4zCcgQe6PVp32GQUiv92 6SuYeW1wowMynPtbPNajNtrvHapr7IZoFTaEGqEfaUgvzlXQ1CQEbB3n94jkxs3xA6On s4+4WITmoCJzDDjuRSRnh8kdwUVxeSVfhQjx8tj2USEBIPgmuoEu41m8iL3Qj3ndlwW+ iYcA==
X-Gm-Message-State: AOJu0Yzoj0DDmZH1QnnTIMd4YW4gbXj6uVgBGzZ5krNOJXJqgtSv98G+ JHB15Ycjfk9TnPKgznJ3mOkW1TOQivJ0ly/PtTTz9EnqtbC64Adv
X-Google-Smtp-Source: AGHT+IHSaReWNzHQkEl/3KAZ5fv39WJFq2cwrLum5CXMMmC3dpGorxatUAqDu/lyoNjb+/lpLw0/8/WXwho3mFedInY=
X-Received: by 2002:a05:6a20:7f92:b0:17e:5d2f:f439 with SMTP id d18-20020a056a207f9200b0017e5d2ff439mr4841497pzj.46.1698430332818; Fri, 27 Oct 2023 11:12:12 -0700 (PDT)
MIME-Version: 1.0
References: <>
In-Reply-To: <>
From: Mallory Knodel <>
Date: Fri, 27 Oct 2023 14:12:01 -0400
Message-ID: <>
To: Marwan Fayed <>
Content-Type: multipart/alternative; boundary="000000000000a9f3860608b6a01a"
Archived-At: <>
Subject: Re: [Pearg] More comments on draft-irtf-pearg-safe-internet-measurement-08
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Privacy Enhancements and Assessment Proposed RG <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 27 Oct 2023 18:12:17 -0000

Thanks for the review Marwan,

Please share elsewhere outside IRTF for reviews just ask folks to send them
with the list on CC or in GitHub issues.

I have shared with the Tor research review board as per our last meeting.

I top posted to make clear that under no circumstances do the authors of
this draft consider measurement bad. In fact it is based on the Tor network
safe measurement guidelines— so, from the community of measurement and
internet research.

This isn’t about making hard trade offs, it’s accepting responsibility to
do the right thing— all researchers in every field do this— and documenting
what that right thing is.

Kind regards,

On Fri, 27 Oct 2023 at 13:01, Marwan Fayed <marwan=> wrote:

> Opening with thanks for the reminders to comment on this draft! Perhaps a
> most useful suggestion is to solicit feedback from beyond the IRTF/IETF
> community since so much of our best practice and guidance has been
> developed, and is tracked, outside of the IRTF/IETF. I'm happy to help in
> this regard.
> As a member of the measurement community, I can echo earlier sentiments
> that this draft is useful. I also see gaps, but have additional concerns
> about the way it positions measurement, overall, in ways that (at a
> minimum) could adversely bias institutional review or other forms of
> scrutiny by panels and people unfamiliar with the measurement domain.
> First, and most importantly, my reading of the draft's set-up in its
> current form is that it (unintentionally no doubt) presupposes that
> measurement is bad and harmful. Specifically, the draft explicitly equates
> many if not all forms of measurement each as being a type of attack. Some
> examples:
>  - [1.2] "Passive measurements use surveillance..." -- the Oxford
> definition of surveillance is "[noun] close observation, especially of a
> suspected spy or criminal."
>  - [1.3] encapsulates all measurements as being sources of attack. There
> are 12 explicit definitions, 9 of which start by "An attack".
> It's important to recognise that a draft like this would be beneficial to
> a wide audience. At the same time I am unable to see how an outside
> observer or reviewer, e.g. institutional review, using guidelines of this
> form (and likely have to consider liability along with safe and responsible
> practice), wouldn't automatically be forced into a starting position of
> defensiveness against measurement. (Aside: one additional suggestion may be
> to use 'responsible' alongside 'safe', which is a common convention in the
> research community.)
> The 'risk' in this current draft is that its current set-up reverses a lot
> of the incredibly hard and tireless efforts from the measurement community
> to define, account for, report, and review best practices, as well as call
> out violations.
> Second, there are a few subtleties missing from some 'Guidelines' that
> cannot be overlooked. Their omission could lull a reader into false senses
> of security that they've checked certain boxes. In particular,
>  - [2.1] suggests that informed consent is sufficient, when in fact we
> know that even fully informed consent in some circumstances may not be
> enough. For example, the value and merit of consent to statements like "by
> helping this measurement you could go to jail" is questionable, at best.
> Alongside, the seeker of consent has to be taken into consideration; for
> example, there is no way to know the effect of a consent-seeker on a
> consent-giver who may be more or less inclined to trust a multinational
> corporation, foreign government, civil liberty group, or a university lab
> -- these are just not the same, and an individual may trust more or less
> depending.
>  - [2.3] This is a good one! If it's helpful, even the seemingly obviously
> 'ok' can have caveats. Speedtests are a great example. It might be
> reasonable to assume that any speedtest server is happy to receive a test
> request, and any subscriber who pays for access is entitled to request the
> test -- but what about the transit provider in-between?
>  - [2.4] Also great, and easy to overlook. Here, too, however, I wonder if
> all requests are equal? As an example, what does it mean for Google to
> request do-not-scan? (I honestly don't know!) One possible addition may be
> to suggest that "self-identify" is good practice, especially when releasing
> measurement tools into the world. For example, the popular zmap tool self
> identifies itself by placing "54321" in the IP-ID field.
> One last suggestion: It's hard to know which are the most useful
> references in this space, so a draft like this would be a great place to
> have a comprehensive list of pointers to best and current practice in this
> space (e.g. the CACM written by Partridge and Allman), and docs that might
> include case studies. Even better might be to have one or two sentences for
> each to describe why the reference is useful, or what it offers.
> (Acknowledging in advance that distilling down from multiple inputs is a
> challenge in itself, and would be a great feat!)
> Hope some of this helps.
> --marwan
> --
> Pearg mailing list