[hops] HOPS bar bof notes

Aaron Falk <aaron.falk@gmail.com> Mon, 23 March 2015 17:51 UTC

Return-Path: <aaron.falk@gmail.com>
X-Original-To: hops@ietfa.amsl.com
Delivered-To: hops@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com []) by ietfa.amsl.com (Postfix) with ESMTP id DE58C1ACEFF for <hops@ietfa.amsl.com>; Mon, 23 Mar 2015 10:51:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id oMl2Gsaw0R_n for <hops@ietfa.amsl.com>; Mon, 23 Mar 2015 10:51:50 -0700 (PDT)
Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0CDE51ACED2 for <hops@ietf.org>; Mon, 23 Mar 2015 10:51:50 -0700 (PDT)
Received: by iecvj10 with SMTP id vj10so41983601iec.0 for <hops@ietf.org>; Mon, 23 Mar 2015 10:51:49 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=gRNxz1xNBXylhHvf42MY8NiVmOjX1jbFPrEiDx1fdMo=; b=Di3F7XFpDW9ZfSBqxgXJze107ptZpdiBmH4HEAFKCsSl2W5/5ylo5GVOGSW4HJtyUB s7LE0M70roiqZjt6PZffdNIu7a3GzIjpH7BeOVuYUtx86mBMA0TPZBdR305LTDHydGYL ovD3o0oW1YqPxPJeOMXplmqQbFQuCCT8G/NAHfwwH+j4evi7Xmv5IsAerXcaUhrQr+q5 P7E4qgGGnfBMkIJaKJBhdK/WkA5M5BqPDWB/nqa8WYzMMtEfR+VUO3N4Zc3yhWESuWQl dig9EwQGToTEisj2tvtReegkNTJDhnxXHcPo2swujeYFzXo97UmVFJevpEJ3kmK7zMS5 Zu7A==
MIME-Version: 1.0
X-Received: by with SMTP id x201mr468438iod.33.1427133109532; Mon, 23 Mar 2015 10:51:49 -0700 (PDT)
Received: by with HTTP; Mon, 23 Mar 2015 10:51:49 -0700 (PDT)
Date: Mon, 23 Mar 2015 13:51:49 -0400
Message-ID: <CAD62q9UKguyrDTiDYSG7nzBNYoSovdO1K91OAAYvqU2ssz8+fA@mail.gmail.com>
From: Aaron Falk <aaron.falk@gmail.com>
To: hops@ietf.org
Content-Type: multipart/alternative; boundary=001a1140fe660a80520511f85230
Archived-At: <http://mailarchive.ietf.org/arch/msg/hops/8SItHgcq5su8ak4AKwaqPEl03Rk>
Subject: [hops] HOPS bar bof notes
X-BeenThere: hops@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Measuring deployability of new transport protocols <hops.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/hops>, <mailto:hops-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/hops/>
List-Post: <mailto:hops@ietf.org>
List-Help: <mailto:hops-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hops>, <mailto:hops-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 23 Mar 2015 17:51:54 -0000

In the interest of getting input from folks while their memories are fresh,
I'm sending these out less polished than would be ideal.  Please send
comments to the list.


March 23, 2015 - IETF-92 - Dallas


As the IETF considers how to approach the difficulties faced in deploying
new or extended transport protocols, it seems the discussion would be
helped by more data.  Specifically, observations of network behavior from
the perspective of clients on a range of network types including mobile,
wifi, enterprise, residential in a wide range of geographic locations.  The
hypothesis is that there is some clustering of behaviors that restrict
protocol transparency which could assist protocol developers, implementors,
and operators.  A specific vision was put on the table of instrumented
browsers and mobile OSs that would actively probe for a range of protocol
transparency and report it in a common and anonymized fashion to permit
aggregation across measurements and inform interested stakeholders.

The objective of the meeting was to expand on the idea above, look for
interest and think about how to move forward.  There was not expectation
that an IETF or IRTF activity would be a goal.  Action items are identified
below by *AI*.

Measurements could inform discussions on network neutrality, creating
funding opportunities to support work in this area.  Regulators are
interested in getting data on QoS and pay to do this because there is
public interest.

Some challenges mentioned include the effort to instrument clients for
active measurement, privacy issues, data sharing policies.

Below the highlights of the meeting are summarized with attribution where
possible.  Please send amplifications and corrections to hops@ietf.org.
Thanks to Mirja Kühlewind for taking notes.

*Passive data collection*

   - Rich Barnes: people that own stacks can tell what would be easy or
   hard to collect; (and here is what we collecting (Jana))
   - Mirja: might be to much effort; how could we reduce this effort?
   - Aaron: other thing is to clearly articulate what we want to get
   - Jana: the data is already there; I’m interested in exposing existing
   data (instead of handling each data request separately).  Need to talk
   - Christian Huitema: agree better to provide data in the structure that
   is already there that to get caught up in stating measurement needs.
   - Jana: collected data provides different angles that are all valuable:
   modifiying server or just clients but a lot deployment work for active
   measurement; but passive is done by browser vendors and data can already be
   used to answers some questions; difference in quality of data
   - Aaron: would be useful to know what is collected passively by e.g.
   Google because others (e.g., Akamai) might be able to collect the
   comparable data; would be useful to just expose what data is collected as
   first step
   - *AI:* Jana will figure out what data are available (at browser
   vendors/measurement platforms)

*Active data collection*

   - Rich Barnes: we know that the data we have on hand is not all the data
   we want; need to first look at what we got and how expensive it is to build
   what we want
   - Dave Thaler: agree, there is a cost to make things available:
   instrument code; move data; publish data. It’s better to know sooner what
   you want to make visible. One possible output is to capture what people
   would like to know to starting working on this now.
   - Brian Trammell: it’s no secret what we want.  Look at the tables from
   the Honda and Hicups papers and collect that data for the whole Internet;
   also difference between UPD and TCP in connectivity and performance (for
      - AI: Brian to send pointer to the Hicups paper on hops list
   - *AI:* Brian will collect (wish) list of what data people would like to


   - David (Apple): seems like you need IP addresses but privacy is an issue
   - Aaron: a possible output of this activity could be a test set that
   operators & vendors could use to verify transparency.
   - Lee Howard (TWC): would like to see numbers broken down to ASN level
   to know that my network is broken because I might have never thought about
   testing myself -> testset would be useful (e.g. using a platform like lmap)

*Aggregated stats*

   - General agreement that aggregated statistics can be very useful and
   remove privacy concerns
   - Brian: just getting numbers on how bad the impairment is would be
   useful.  Eg., how often does a problem appear?
   - Jana: needed for informing design compromises and infrastructure to
   ask these question (histograms)

*Test infrastructure*

   - Colin (RIPE Atlas): have large active measurement structure; could
   help to standardize what measurements need to be made.
   - Bob Briscoe: Honda paper shows data of different networks but
   RIPE might only get limited scope (e.g., no mobile)
   - Google MLABs also mentioned
   - Dave P.: Akamai does a lot of measurement (see
   http://www.stateoftheinternet.com/), can do more
   - *AI:* group to share on the list what measurement (platforms) exists
   and what capabilities they have

*Sharing data*

   - Instead of describing tests, define an observatory where people can
   store data as they have them -> a wiki
   - Brian: have been many tries to improve data sharing e.g. DatCat but
   problem is metadata and effort to maintain metadata

*Fuzz testing for middleboxes*

   - Dave Plonka: could be useful but may also be more terrifying than
   reality since it will test things that don’t occur in the wild.
   - Suresh: interop lab testing does give no info about what happens in
   the internet
   - Jana: fuzz testing has been explored to a very low extend but time is
   important; action item might be to get data exposed
   - Stuart: fuzzing is difficult because option are infinite what can be
   done and whats valid so it mist be done manually by a human