Re: Gen-ART review of draft-livingood-woundy-p4p-experiences-07

"Livingood, Jason" <Jason_Livingood@cable.comcast.com> Thu, 28 May 2009 02:29 UTC

Return-Path: <jason_livingood@cable.comcast.com>
X-Original-To: ietf@core3.amsl.com
Delivered-To: ietf@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id AC0893A6D9C; Wed, 27 May 2009 19:29:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.246
X-Spam-Level:
X-Spam-Status: No, score=-5.246 tagged_above=-999 required=5 tests=[AWL=-1.150, BAYES_00=-2.599, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, MANGLED_NAIL=2.3, RCVD_IN_DNSWL_HI=-8, RCVD_NUMERIC_HELO=2.067]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id moD-V8C4-muq; Wed, 27 May 2009 19:29:30 -0700 (PDT)
Received: from pacdcimo01.cable.comcast.com (PacdcIMO01.cable.comcast.com [24.40.8.145]) by core3.amsl.com (Postfix) with ESMTP id 899D83A6D4A; Wed, 27 May 2009 19:29:29 -0700 (PDT)
Received: from ([10.52.116.31]) by pacdcimo01.cable.comcast.com with ESMTP id 5503620.39600540; Wed, 27 May 2009 22:30:30 -0400
Received: from PACDCEXCMB04.cable.comcast.com ([24.40.15.86]) by PAOAKEXCSMTP02.cable.comcast.com with Microsoft SMTPSVC(6.0.3790.3959); Wed, 27 May 2009 22:30:30 -0400
Received: from 68.83.175.155 ([68.83.175.155]) by PACDCEXCMB04.cable.comcast.com ([24.40.15.86]) via Exchange Front-End Server webmail.comcast.com ([24.40.8.153]) with Microsoft Exchange Server HTTP-DAV ; Thu, 28 May 2009 02:30:18 +0000
User-Agent: Microsoft-Entourage/12.17.0.090302
Date: Wed, 27 May 2009 22:30:17 -0400
Subject: Re: Gen-ART review of draft-livingood-woundy-p4p-experiences-07
From: "Livingood, Jason" <Jason_Livingood@cable.comcast.com>
To: Spencer Dawkins <spencer@wonderhamster.org>, Chris Griffiths <Chris_Griffiths@Cable.Comcast.com>, Laird Popkin <laird@pando.com>, yry@cs.yale.edu
Message-ID: <C6436F79.C2AA%Jason_Livingood@cable.comcast.com>
Thread-Topic: Gen-ART review of draft-livingood-woundy-p4p-experiences-07
Thread-Index: AcnfPDztQm1wq/DZaES9qTNmN14z8Q==
In-Reply-To: <5C3B4F7EC6014E4B80680F1F6487F8E9@china.huawei.com>
Mime-version: 1.0
Content-type: text/plain; charset="US-ASCII"
Content-transfer-encoding: 7bit
X-OriginalArrivalTime: 28 May 2009 02:30:30.0067 (UTC) FILETIME=[44B76030:01C9DF3C]
Cc: General Area Review Team <gen-art@ietf.org>, Lisa Dusseault <lisa@osafoundation.org>, ietf@ietf.org
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ietf>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 28 May 2009 02:29:31 -0000

Thanks for the detailed review, Spencer!  I'll set aside some time next week
to review your comments and then make necessary changes as needed in the
draft to address this in a -08 version.

Regards
Jason


On 5/27/09 9:55 PM, "Spencer Dawkins" <spencer@wonderhamster.org> wrote:

> I have been selected as the General Area Review Team (Gen-ART) reviewer for
> this draft (for background on Gen-ART, please see
> http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html). Please resolve
> these comments along with any other Last Call comments you may receive.
> 
> Document: draft-livingood-woundy-p4p-experiences-07
> Reviewer: Spencer Dawkins
> Review Date: 2009-05-27
> IETF LC End Date: 2009-06-16
> IESG Telechat date: (not known)
> 
> Summary: This document is nearly ready for publication as an Informational
> RFC. I did identify some minor issues (listed below) that should be
> considered as this document moves forward in the approval process.
> 
> I also identified some nits, which aren't actually part of the Gen-ART
> review but are included for the convenience of the editor.
> 
> Thanks,
> 
> Spencer
> 
> 1.  Requirements Language
> 
>    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
>    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
>    document are to be interpreted as described in RFC 2119 [RFC2119].
> 
> Spencer (nit): idnits says no 2119 keywords in the document, so this section
> can be deleted (along with the [rfc2119] reference.
> 
> 2.  Introduction
> 
>    Comcast is a large broadband ISP, based in the U.S., serving the
> 
> Spencer (nit): 
> http://www.rfc-editor.org/rfc-style-guide/abbrev.expansion.txt doesn't list
> "ISP" as an abbreviation that isn't expanded ("please expand on first use").
> 
>    majority of its customers via cable modem technology.  A trial was
>    conducted in July 2008 with Pando Networks, Yale, and several ISP
>    members of the P4P Working Group, which is part of the Distributed
>    Computing Industry Association (DCIA).  Comcast is a member of the
>    DCIA's P4P Working Group, whose mission is to work with Internet
>    service providers (ISPs), peer-to-peer (P2P) companies, and
>    technology researchers to develop "P4P" mechanisms, such as so-called
>    "iTrackers" (hereafter P4P iTrackers), that accelerate distribution
>    of content and optimize utilization of ISP network resources.  P4P
>    iTrackers theoretically allow P2P networks to optimize traffic within
>    each ISP, reducing the volume of data traversing the ISP's
>    infrastructure and creating a more manageable flow of data.  P4P
>    iTrackers can also accelerate P2P downloads for end users.
> 
>    The P4P iTracker trial was conducted, in cooperation with Pando,
>    Yale, and three other P4P member ISPs, from July 2 to July 17, 2008.
>    This was the first P4P iTracker trial over a cable broadband network.
>    The trial used a Pando P2P client, and Pando distributed a special 21
>    MB licensed video file in order to measure the effectiveness of P4P
> 
> Spencer (nit): suggest s/21 MB/21-MB/ for clarity
> 
>    iTrackers.  A primary objective of the trial was to measure the
>    effects that increasing the localization of P2P swarms would have on
> 
> Spencer (minor): it would be helpful to add a definition for "swarm" -
> everyone kind of knows what you're talking about, but it's not even defined
> in Wikipedia :-) (per
> http://en.wikipedia.org/wiki/Swarm_(disambiguation))...
> 
>    P2P uploads, P2P downloads, and ISP networks, in comparison to normal
>    P2P activity.
> 
> 
> 3.  High-Level Details
> 
>    There were five different swarms for the content used in the trial.
>    The first was a random P2P swarm, as a control group.  The second,
>    third, and fourth used different P4P iTrackers: Generic, Coarse
>    Grained, and Fine Grained, all of which are described in Section 4.
>    The fifth was a proprietary Pando mechanism.  (The results of the
>    fifth swarm, while very good, are not included here since our focus
> 
> Spencer (minor): "while very good" is slightly more marketing-speak than I
> was comfortable with... the ADs can ignore this comment, of course.
> 
>    is on open standards and a mechanism which may be leveraged for the
>    benefit of the entire community of P2P clients.)  Comcast deployed a
>    P4P iTracker server in its production network to support this trial,
>    and configured multiple iTracker files to provide varying levels of
>    localization to clients.
> 
> 4.1.  P4P Fine Grain
> 
>    The Fine Grain topology was the first and most complex P4P iTracker
>    that we built for this trial.  It was a detailed mapping of Comcast
>    backbone-connected network Autonomous System Numbers (ASN) to IP
>    Aggregates which were weighted based on priority and distance from
>    each other.  Included in this design was a prioritization of all Peer
>    and Internet transit connected ASNs to the Comcast backbone to ensure
>    that P4P traffic would prefer settlement free and lower cost networks
>    first, and then more expensive transit links.  This attempted to
>    optimize and lower transit costs associated with this traffic.  We
>    then took the additional step of detailing each ASN and IP aggregate
>    into IP subnets down to Optical Transport Nodes (OTN) where all Cable
>    Modem Termination Systems (CMTS) reside.  This design gave a highly
> 
> Spencer (minor): you're referring to components of cable networking that
> probably aren't familiar to many IETF participants - is there a generic
> reference you could include here, for people who see a bunch of terms they
> aren't familiar with, and want to investigate further? If not, you could
> probably end the sentence at "into IP subnets".
> 
>    localized and detailed description of the Comcast network for the
>    iTracker to disseminate.  This design defined 1,182 P4P iTracker node
>    identifiers, and resulted in a 107,357 line configuration file.
> 
> 4.2.  P4P Coarse Grain
> 
>    Given the level of detail in the Fine Grain design, it was important
>    that we also enable a high-level design which still used priority and
>    weighting mechanisms for the Comcast backbone and transit links.  The
>    Coarse Grain design was a limited or summarized version of the Fine
>    Grain design, which used the ASN to IP Aggregate and weighted data
>    for transit links, but removed all additional localization data.
>    This insured we would get similar data sets from the Fine Grain
>    design, but without the more detailed localization of each of the
>    networks off of the Comcast backbone.  This design defined 22 P4P
> 
> Spencer (nit): "off of" wasn't clear to me - could you rephrase? is this
> "attached to"? "adjacent to"?
> 
>    iTracker node identifiers, and resulted in a 998 line configuration
>    file.
> 
> 4.3.  P4P Generic Weighted
> 
>    The Generic Weighted design was a copy of the Coarse Grained design
>    but instead of using ISP-designated priority and weights, all weights
>    were defaulted to pre-determined parameters that the Yale team had
>    designed.  All other data was replicated from the Coarse Grain
>    design.  Providing the information necessary to support the Generic
>    Weighted iTracker was roughly the same as for Coarse Grain.
> 
> Spencer (minor): is this "the level of effort was roughly the same"? not
> quite sure what you're saying here...
> 
> 5.2.  Impact on Download Speed
> 
>    The results of the trial indicated that P4P iTrackers can improve the
>    speed of downloads to P2P clients.  In addition, P4P iTrackers were
>    effective in localizing P2P traffic within the Comcast network.
> 
> Spencer (minor): I'm not sure I understand how the table below shows
> localization (speedup which could be attributed to localization, maybe?)...
> is the table supposed to show this?
> 
>                    Impact of P4P iTrackers on Downloads:
> 
>    +--------------+------------+------------+-------------+------------+
>    |     Swarm    | Global Avg |   Change   | Comcast Avg |   Change   |
>    |              |     bps    |            |     bps     |            |
>    +--------------+------------+------------+-------------+------------+
>    |    Random    |   144,045  |     n/a    | 254,671 bps |     n/a    |
>    |   (Control)  |     bps    |            |             |            |
>    |  ----------  | ---------- | ---------- |  ---------- | ---------- |
>    |   P4P Fine   |   162,344  |    +13%    | 402,043 bps |    +57%    |
>    |    Grained   |     bps    |            |             |            |
>    |  ----------  | ---------- | ---------- |  ---------- | ---------- |
>    |  P4P Generic |   163,205  |    +13%    | 463,782 bps |    +82%    |
>    |    Weight    |     bps    |            |             |            |
>    |  ----------  | ---------- | ---------- |  ---------- | ---------- |
>    |  P4P Coarse  |   166,273  |    +15%    | 471,218 bps |    +85%    |
>    |    Grained   |     bps    |            |             |            |
>    +--------------+------------+------------+-------------+------------+
> 
> 
>            Table 2: Per-Swarm Global and Comcast Download Speeds
> 
> 
> 6.  Important Notes on Data Collected
> 
>    We also recommend that readers not focus too much on the absolute
>    numbers, such as bytes downloaded from internal sources and bytes
>    downloaded from external sources.  Instead, we recommend readers
>    focus on ratios such as the percentage of bytes downloaded that came
>    from internal sources in each swarm.  As a result, the small random
>    variation between number of downloads of each swarm does not distract
>    readers from important metrics like shifting traffic from external to
>    internal sources, among other things.
> 
> Spencer (minor): it would be great if there was a table that showed these
> ratios explicitly, if we're supposed to focus on them... :-) 5.1 and 5.2
> with tables are a lot easier to grok than 5.3 without ...
> 
> 7.  Next Steps
> 
>    We believe these results can inform the technical discussion in the
>    IETF over how to use P4P iTracker mechanisms.  Should such a
>    mechanism be standardized, the use of ISP-provided P4P iTrackers
>    should probably be an opt-in feature for P2P users, or at least a
>    feature of which they are explicitly aware of and which has been
>    enabled by default in a particular P2P client.  In this way, P2P
>    users could choose to opt-in either explicitly or by their choice of
>    P2P client in order to choose to use the P4P iTracker to improve
>    performance, which benefits both the user and the ISP at the same
>    time.  Importantly in terms of privacy, the P4P iTracker makes
>    available only network topology information, and would not in its
>    current form enable an ISP, via the P4P iTracker, to determine what
>    P2P clients were downloading what content.
> 
> Spencer (nit): s/what/which/ seemed more natural to me in this sentence (but
> do the right thing)
> 
> 9.  IANA Considerations
> 
> Spencer (nit): we often include "Note to RFC Editor: please delete this
> section before publication" for null IANA sections, as Russ commented on one
> of my drafts earlier today :-)
> 
>    There are no IANA considerations in this document.
> 
> 10.  Acknowledgements
> 
>    The authors wish to acknowledge the hard work of all of the P4P
>    working group members, and specifically the focused efforts of the
>    teams at both Pando and Yale for the trial itself.  Finally, the
>    authors recognize and appreciate Peter Sevcik and John Bartlett, of
>    NetForecast, Inc., for their valued independent analysis of the trial
>    results.
> 
> 
> 11.1.  Normative References
> 
> Spencer (nit): as above, you can delete this reference
> 
>    [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
>               Requirement Levels", BCP 14, RFC 2119, March 1997.
> 
> 11.2.  Informative References
> 
> Spencer (nit): idnits says [SIGCOMM] isn't used as a reference in the
> document, so it can go away, too.
> 
>    [SIGCOMM]  Xie, H., Yang, Y., Krishnamurthy, A., Liu, Y., and A.
>               Silberschatz, "ACM SIGCOMM 2008 - P4P: Provider Portal for
>               Applications", Association for Computing Machinery SIGCOMM
>               2008 Proceedings, August 2008,
>               <http://ccr.sigcomm.org/online/files/p351-xieA.pdf>.
> 
> 
>