Re: [Pearg] Call for adoption: draft-learmonth-pearg-safe-internet-measurement-02.txt

[Eric Rescorla <[e-mail]>]

On Mon, May 27, 2019 at 8:39 AM Iain Learmonth <[e-mail]> wrote:

> Hi Eric,
>
> On 27/05/2019 14:34, Eric Rescorla wrote:
> > I have reviewed this document and while I think some of the advice
> > here is potentially useful, I don't think the recommendations really
> > match what's current practice or what's practical. As such, I don't
> > think it should be adopted without quite a bit more work.
>
> In my opinion, a lot of current practice is not safe. This document does
> not aim to set out current practice. It aims to raise the bar on user
> safety when it comes to performing Internet measurement.
>

Yes, I appreciate that. However, as written I don't think it captures those
boundaries well and so is not very useful.


> I don't really want to get into a long debate about whether any
> > particular study type is appropriate. Rather, these are common study
> > types and so if the advice in this document is to be useful, then it
> > needs to reasonably match what people do -- or at least have a much
> > stronger argument that people should change what they do than is
> > offered here.
>
> Not necessarily. If it turns out that upon analysis, a lot of studies
> are dangerous for users, this document should not weaken its guidelines
> to allow those studies to continue. That would be silly.
>

Hence the text you are directly quoting above.

"or at least have a much stronger argument that people should change what
they do than is offered here.

Neither the draft as written, nor your response, seems to me to offer any
such argument.



>     The experiment uses an online advertisement campaign to deliver
> >     the test code to end systems. When the end system is passed an ad
> >     that is carrying the experiment the system runs embedded Adobe
> >     Flash code. The code is executed when the ad is passed to the
> >     user, and does not rely on a user "click" or any other user
> >     trigger action. The active code interrogates one of two experiment
> >     controllers by performing a URL fetch. The contents of the fetched
> >     experiment control URL are a dynamically generated sequence of
> >     four URLs. These four URLs are the substance of the test setup.
>
> This is great until you run on a user machine in a country which has
> some censorship/monitoring infrastructure in place that has
> misinterpreted the URL as some proscribed content, landing the user in
> trouble.
>

Well, this isn't my study, so I'm not going to account for how it's
constructed,
but it seems like you're missing the context here that this is served
as part of an ad, and that the way that ads work *in general* is to load
content from remote servers. It's not clear to me why you think that
the URL that the study checks is somehow more likely to be interpreted
as proscribed than the ad content that would normally be served.

With that said, we also do studies where the browser loads specific URLs
we provide it, and we do try to make them innocuous.



> At the very least you may have generated costs for the user's bandwidth
> there, which are uncompensated.
>

Again, this is served via an ad network, so the user would be loading
some other ad that used up random amounts of bandwidth instead of
the ad that runs the study. This seems like a pretty weak argument.




> > It's worth noting at this point that the Web is a platform for running
> > remote code, and by browsing you're opting into that, and ad studies
> > just leverage that behavior.
>
> Tell that to the 1,543,235 users that have installed NoScript from
> Firefox Add-ons. Now you could say that they have opted out and the code
> won't be run, but the way you've phrased this makes me think that
> actually you just haven't understood the wider range of Internet users
> that exist.
>

I'm quite aware of NoScript (you do know that I work for Mozilla, right?)
but this is a trivially small fraction of the browsing user base. As a
general
matter, the Web is a platform for executing code sent by the server,
which of course users can disable if they choose.


> WRT to the first point, as a general matter, modern browsers
> > auto-update, so the user has generally opted into regularly getting
> > whatever new code the vendor thinks makes the best browser.
>
> Say a mobile phone vendor wanted to test out how its camera was doing.
> It ships you an auto-update that sends back every photo you've taken to
> work out things like light levels, noise, and what might need tuned.
> This is for the purpose of improving the camera.
>

I think you've misunderstood my point. I'm not saying that the browser
vendor  -- or a mobile phone vendor -- can just send malicious code to
the user (and for avoidance of doubt I would consider reporting every photo
to be malicious).

Rather, what I'm saying is that browsers already do auto-updates, and so
it's not
useful to have a much stricter set of rules for experiments/measurements
that are delivered to some users than vendors do for code they would deliver
to all users as part of an update. I.e., it's not that useful to focus on
studies
rather than on overall behavior.



This draft is still in its early stages and there's a lot to flesh out
> still.
>

Understood. I don't think at this early stage it's ready for adoption. Once
you've fleshed it out, I'll be happy to take another look.

-Ekr