Invitation to subscribe to new DCTCP Evolution mailing list: (tcpPrague@ietf.org)
Bob Briscoe <ietf@bobbriscoe.net> Thu, 30 July 2015 11:47 UTC
Return-Path: <ietf@bobbriscoe.net>
X-Original-To: tsv-area@ietfa.amsl.com
Delivered-To: tsv-area@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B42421A8732; Thu, 30 Jul 2015 04:47:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_50=0.8, GB_I_INVITATION=-2, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=unavailable
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CPozGuCyiqjn; Thu, 30 Jul 2015 04:47:39 -0700 (PDT)
Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CC1321A6F2F; Thu, 30 Jul 2015 04:47:37 -0700 (PDT)
Received: from [81.174.189.118] (port=39150 helo=[192.168.0.11]) by server.dnsblock1.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.85) (envelope-from <ietf@bobbriscoe.net>) id 1ZKmJD-0006Mm-J2; Thu, 30 Jul 2015 12:47:36 +0100
Message-ID: <55BA0ED6.8060808@bobbriscoe.net>
Date: Thu, 30 Jul 2015 12:47:34 +0100
From: Bob Briscoe <ietf@bobbriscoe.net>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.8.0
MIME-Version: 1.0
To: tcpm IETF list <tcpm@ietf.org>, tsvwg IETF list <tsvwg@ietf.org>, tsv-area IETF list <tsv-area@ietf.org>, iccrg IRTF list <iccrg@irtf.org>
Subject: Invitation to subscribe to new DCTCP Evolution mailing list: (tcpPrague@ietf.org)
References: <55B77CFE.9090505@bobbriscoe.net>
In-Reply-To: <55B77CFE.9090505@bobbriscoe.net>
Content-Type: multipart/alternative; boundary="------------030901060405050102080506"
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - server.dnsblock1.com
X-AntiAbuse: Original Domain - ietf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - bobbriscoe.net
X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net
Archived-At: <http://mailarchive.ietf.org/arch/msg/tsv-area/_TJTlUOCYcDJIM3X4c7rYxLVFn4>
X-BeenThere: tsv-area@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: IETF Transport and Services Area Mailing List <tsv-area.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsv-area>, <mailto:tsv-area-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsv-area/>
List-Post: <mailto:tsv-area@ietf.org>
List-Help: <mailto:tsv-area-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsv-area>, <mailto:tsv-area-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 30 Jul 2015 11:47:46 -0000
tcpm, tsvwg, tsvarea, iccrg lists Recently, there have been developments (see URLs at end) that would make it possible to deploy scalable low-latency low-loss protocols like Data Center TCP alongside a mix of traffic, either in data centres and private networks, or even on the public Internet. One approach was demonstrated at the recent IETF in Prague, showing DCTCP giving ultra-low latency over a broadband Internet access while competing with a mix of Internet traffic on roughly equal terms. As a result of an ad hoc meeting ("Bar BoF" = Birds of a Feather) at the Prague IETF, we have formed a new mailing list. I'd like to invite you to join the list via: <https://www.ietf.org/mailman/listinfo/tcpprague> The idea is to ensure that those working on DCTCP implementations across platforms (Free BSD, Linux, Windows, ...) will converge on solutions that will interwork with each other and with existing traffic. Although it is under the IETF's umbrella, we hope and expect that discussion will be as much about implementation as writing standards. However, we get the benefit of the IETF's IPR disclosure rules <https://www.ietf.org/about/note-well.html>, and of course it fits the IETF's purpose of interoperability. The draft notes of the meeting are below. And below that, is the original announcement with some context and background URLs. You can catch up on any discussion you've missed using the list archives via the link above. If you want to respond about something most relevant to tcpprague, pls avoid cross-posting to all the other lists in this announcement. Cheers Bob Briscoe -------- Forwarded Message -------- Subject: Notes: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague Date: Tue, 28 Jul 2015 14:00:46 +0100 From: Bob Briscoe <ietf@bobbriscoe.net> To: TCP Prague IETF List <tcpPrague@ietf.org> Folks, These notes have taken a week, because I've only just put my machine back together after having to rebuild the hardware a little :| _*Notes: DCTCP Evolution Bar BoF*_ 6-7pm Tue 21 Jul 2015, Budapest room, The Hilton, Prague, CZ **Summary of Actions:* *Lars E: Set up tcpprague wiki page Bob B: Request tcpprague@ietf.org mailing list, via IETF process (requires Area Director approval) Bob B: Document Rationale - initiate a para on wiki. Lars E: fwd Dagstuhl invitee list to Bob Bob B: Set up list on wiki to assign people to invite those not in the room to join. * * 18:00 Introductions - name and interest ** ***Present: Marcelo Bagnulo Braun <marcelo@it.uc3m.es> Praveen Balasubramanian <pravb@microsoft.com> Martin Bekker <martin.becke@haw-hamburg.de> Bob Briscoe <ietf@bobbriscoe.net> Anna Brunstrom <anna.brunstrom@kau.se> Stuart Cheshire <cheshire@apple.com> Koen De Schepper <koen.de_schepper@alcatel-lucent.com> Fabien Duchen <fabien.duchene@uclouvain.be> Phil Eardley <philip.eardley@bt.com> Lars Eggert <lars@netapp.com> Michio Honda <michio@netapp.com> Per Hurtig <Per.Hurtig@kau.se> Jana Iyengar <jri@google.com> Naeem Khademi <naeem.khademi@gmail.com> Mirja Kuehlewind <mirja.kuehlewind@tik.ee.ethz.ch> Matt Mathis <mattmathis@google.com <mailto:mattmathis@google.com>> Andrew McGregor <andrewmcgr@google.com> Karen Nielsen <karen.nielsen@tieto.com> Tommy Pauly <tpauly@apple.com> Andreas Petlund <apetlund@ifi.uio.no <mailto:apetlund@ifi.uio.no>> Costin Raiciu <costin.raiciu@cs.pub.ro> Pasi Sarolahti <pasi.sarolahti@iki.fi> Richard Scheffenegger <rs@netapp.com> David Schinazi <dschinazi@apple.com> Randall Stewart <randall@lakerest.net> Dave Thaler <dthaler@microsoft.com> Brian Trammell <ietf@trammell.ch <mIlto:ietf@trammell.ch>> Michael Tuexen <Michael.Tuexen@lurchi.franken.de> Felix Weinrank <weinrank@fh-munster.de> Michael Welzl <michawe@ifi.uio.no> Alex Zimmermann <alexander.zimmermann@netapp.com> ** Scope and Agenda Bashing** *** [Non-italic text is from the materials pre-prepared by Koen De Schepper and Bob Briscoe. /Italic text summarises conversation in the room./] Meeting is covered by the standard IETF "Note Well" concerning intellectual property. *Scope*: * Evolving the e2e DCTCP protocol for use alongside existing traffic (whether in DCs, private nets or public Internet). * Primarily to get DCTCP /developers/ involved early (Windows, FreeBSD, Linux), so that whatever we decide to standardise can be implemented in parallel (Doing implementation and standardisation in series is not desirable, in whichever order). * Primarily an organisational meeting about creating a forum / community to do this work, using people's experience to know what will work best. *Not in Scope:*** * Network changes are not in scope unless they impact the list of changes needed to DCTCP * The in-network side of the solution (two approaches exist [DCttH <http://www.bobbriscoe.net/projects/latency/dctth_preprint.pdf>, Judd15 <https://www.usenix.org/system/files/conference/nsdi15/nsdi15-paper-judd.pdf>], others may follow).* ** Identifier of DCTCP-like traffic (please discuss by email, not in this meeting) /Lars E: Informational draft recording Microsoft's DCTCP should not be stalled by this, as it has value of its own. Unanimous agreement. / /Praveen S: Microsoft has offered a royalty free license for DCTCP IPR <https://datatracker.ietf.org/ipr/2319/>./ /Karen N: Is DCTCP over a non-TCP transport (e.g. SCTP) in scope//? Unanimous "Yes"// / /Outcome of discussion on the features of this DCTCP-like congestion control that define this work:// / 1. /Must use ECN, but unlike RFC3168 ECN, marking is not merely equivalent to drop,// //so ECN signals can be more plentiful and sooner than drop./ 2. /P//acket rate is proportional to 1/p//, where p is the ECN marking probability. // / /Matt M: 1/p makes congestion control scale with the bandwidth, //by making//the intensity of congestion control signals per RTT invariant//./ // /Stuart Ch: Apple is turning on ECN by default in clients. Currently in developer seeds but probably in the next releases. Packet loss is also not a mystery./ ** 18:15 List of /must-have/ changes before deployment alongside existing traffic.** *** /Matt M: Rather than a "MUST-have" list, produce a prioritised list, because where to draw the necessity line could depend on the use-case.// / The following list wasn't formally prioritised in the meeting, but items where some people questioned necessity are shifted down. 1. Fall back to Reno or Cubic behaviour on loss; /For how long?//quick consensus: 1 RTT, but needs further discussion. ECN response continues to operate in parallel./ 2. Negotiate altered feedback semantics, to convey the extent of ECN marking, not just its existence, and this feedback needs to be robust to loss [RFC-to-be 7560 <https://datatracker.ietf.org/doc/draft-ietf-tcpm-accecn-reqs/>]; /Mirja K, Richard S & Bob B plan to have spec of much simpler solution out soon. / 3. Use of a standardised packet identifier (if ECN-capable is not enough) /Identifier tbd. /*/- - - 8< - - - - - - - - highest line between "must-have for safety" and "would be nice for performance" - - - - - - 8< - - - -/* 4. Handle a window of less than 2 when the RTT is low, rather than increase the queue [TCP-sub-mss-cwnd <https://www.ietf.org/proceedings/93/slides/slides-93-iccrg-5.pdf>], like TCP Nice <http://www.cs.utexas.edu/users/dahlin/software/2002-nice.html>. /Michael W: Is this "must-have"? Quite a complicated step. // //Bob B: Yes, but, otherwise DCTCP will pollute ultra-low latency queues from the start./ 5. Average ECN feedback over its own RTT, not the hard-coded RTT suitable only for data-centres, perhaps reduce cwnd by seg-size/2 per ECN Echo, like Relentless TCP [Mathis09 <staff.psc.edu/mathis/papers/PFLDNet.pdf>]; /???: How bad would long-RTT flows be? More generally, how can we evaluate all this?/ /Bob B: With mixed RTTs, flows with RTT > a couple of ms will respond too quickly to bursts.//Whatever, it's already been implemented by Mohammad Alizadeh in Linux, and evaluated <http://simula.stanford.edu/%7Ealizade/Site/DCTCP_files/dctcp_analysis-full.pdf>, so this is easy./ 6. Heuristic testing for classic ECN bottlenecks //The idea would be to detect a 'classic' RFC316 bottleneck by whether appreciable delay growth accompanies the marking (originally suggested by Michael W). /Bob B: Complex and slow to detect, so it would have to learn and cache for new flows - suggest this should only be a must-have if measurements prove it to be a problem - i.e. if a significant proportion of classic ECN bottlenecks //exist// //Matt M: No need for this - rate mismatch //no worse than TCP already sees with RTT discrepancies./ /* - - - 8< - - - - - - - - lowest line between "must-have for safety" and "would be nice for performance" - - - - - - 8< - - - -*/ 7. /Costin R: //Faster-than-additive increase//(similar to Cubic) //A performance improvement, not "must-have", but would be nice to have while we're doing this./ 8. /[Not discussed in the meeting, but added by Bob B for the record]: Less drastic exit from slow-start, to match less drastic rate reduction per mark.// //Currently, because marking threshold is shallow, //slow start exits earlier than with drop, unnecessarily increasing completion time.// / / //Costin R: Any other way to evolve towards DCTCP over mixed networks, without separate queues in the network?// ////Bob B: To discuss on ML, and if we continue with the proposed approach, we must record the rationale on the WIki.// /// ** 18:30 Brainstorm to identify people not present who will be important to this.** *** Mohammad Alizadeh <alizadeh.mr@gmail.com> Grenville Armitage <garmitage@swin.edu.au> Fred Baker <fred@cisco.com> Stephen Bensley <sbens@microsoft.com> Daniel Borkmann <daniel.borkmann@alumni.ethz.ch> Yuchung Cheng <ycheng@google.com> Kenjiro Cho <kjc@iijlab.net> 邓灵莉/Lingli Deng <denglingli@chinamobile.com> Eric Dumazet <edumazet@gmail.com> Gorry Fairhurst <gorry@erg.abdn.ac.uk> Jamal Hadi Salim <hadi@mojatatu.com> Glenn Judd <glenn.judd@morganstanley.com> Midori Kato <katoon@sfc.wide.ad.jp> Kenneth Klette Jonassen <kennetkl@ifi.uio.no <mailto:kennetkl@ifi.uio.no>> (already subscribed) Sridharan, Murari <muraris@microsoft.com> Hiren Panchasara <hiren.panchasara@gmail.com> Hagen Pfeifer <hagen@jauu.net> Balaji Prabhakar <balaji@ee.stanford.edu> KK Ramakrishnan <kk@cs.ucr.edu> Lawrence Stewart <lstewart@netflix.com> Dave Taht <dave.taht@gmail.com> Florian Westphal <fw@strlen.de> /Agreed to cc to the following for awareness, but no need to invite to join the list:// /Stephen Hemminger <stephen@networkplumber.org> David Miller <davem@davemloft.net> /Missing types of organisations:// / * /Network operators (not so relevant for e2e protocol, but need to be motivated to deploy the network part)/ * /CDN//s/ /[Bob B adds: Subsequent to mtg, Erik Nygren tells me Xin Zhang leads Akamai's congestion control team. Also I noticed Hiren used to work at Limelight, so may have contacts]// //// //Lars E: Co-organising a Dagstuhl retreat around DCTCP. Will forward list of invitees to Bob to notify once the ML exists.// //Also Lars's list of FreeBSD and Linux devs.// /// ** 18:40 What is the best way to ensure the outputs from a number of separate developers all converge in parallel to standardisation?** ***/Common Test Suite// //Interop//events ////Plugfests// ////Serving paths (e.g. Google's) capable of serving this/ ** 18:50 Next steps: Actions to set up suitable MLs, tools, with timesales etc.** *** /Discussed pros and cons of hosting ML on ietf.org.// //General agreement: use ietf.org for ML - because the IPR Note Well is useful. // // /Name for ML? / //Matt M: TCP Prague (for an evolving protocol, a meaningless tag is best).// //Karen N: ecn-prague, because it's not just TCP?// // //Agreed: //tcpprague@ietf.org//// // //Actions:// //Bob B: ML - ask SpencerD/MartinS, following the documented process// //Lars E: Set up wiki page - for assigning people to send out invitations// /// ** End 19:05** *** Notes: Bob Briscoe, helped by Andrew McGregor 28 Jul 2015 -------- Forwarded Message -------- Subject: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague Date: Mon, 20 Jul 2015 22:46:14 +0100 From: Bob Briscoe <ietf@bobbriscoe.net> To: Mirja Kuehlewind <mirja.kuehlewind@tik.ee.ethz.ch>, EGGERT, Lars <lars@netapp.com>, Dave Thaler <dthaler@microsoft.com>, Praveen Balasubramanian <pravb@microsoft.com>, Alex Zimmermann <alexander.zimmermann@netapp.com>, Richard Scheffenegger <rs@netapp.com>, Fred Baker <fred@cisco.com>, Matt Mathis <matt.mathis@gmail.com>, Andrew McGregor <andrewmcgr@google.com>, Dave Taht <dave.taht@gmail.com>, Stuart Cheshire <cheshire@apple.com>, Michael WELZL <michawe@ifi.uio.no>, Andreas Petlund <andreas@petlund.no>, Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Anna Brunstrom <anna.brunstrom@kau.se> CC: De Schepper, Koen (Koen) <koen.de_schepper@alcatel-lucent.com> Folks, DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague Location: Unless I have emailed with a room location before then, pls meet at the IETF reception. Koen & I are trying to get together people in Prague who are involved in development of DCTCP across platforms (Windows, Free BSD, Linux, etc), and who are interested in evolving it for use on heterogeneous networks, e.g. * data centres with a mix of TCP flavours, not just DCTCP * private networks * the public Internet Pls fwd this invite to anyone in Prague who ought to be involved that I've missed (pls cc everyone else too). Sorry for short notice. One purpose of the session will be to build a community beyond the IETF, so I'd like us to compose an email to a wider set of people after the session, e.g.: Stephen Bensley <sbens@microsoft.com> <mailto:sbens@microsoft.com> Glenn Judd <glenn.judd@morganstanley.com> <mailto:glenn.judd@morganstanley.com> Daniel Borkmann <daniel.borkmann@alumni.ethz.ch> <mailto:daniel.borkmann@alumni.ethz.ch> Florian Westphal <fw@strlen.de> <mailto:fw@strlen.de> 邓 灵莉/Lingli Deng <denglingli@chinamobile.com> <mailto:denglingli@chinamobile.com> Mohammad Alizadeh <alizadeh.mr@gmail.com> <mailto:alizadeh.mr@gmail.com> Stephen Hemminger <stephen@networkplumber.org> <mailto:stephen@networkplumber.org> David S. Miller <davem@davemloft.net> <mailto:davem@davemloft.net> Sridharan, Murari <muraris@microsoft.com <mailto:muraris@microsoft.com>> Yuchung Cheng <ycheng@google.com <mailto:ycheng@google.com>> Koen & Bob PS. Below is some background, and some agenda ideas. Pls discuss, bash and add your own. > We've recently developed an AQM that allows DCTCP to co-exist with > Cubic/Reno etc. with zero config. Links below. > > We have to add some necessary mechanisms to DCTCP (listed below) so it > will be safe alongside other traffic. Two questions: > > Q1. We don't want to fork DCTCP. Does anyone think a fork optimised > for homogeneous DCTCP would be better? > > Q2. Anyone interested in helping? > We have an idea how to do each one, but sharing the load would be > great - there's Linux, FreeBSD, Windows, etc. to cover. > > List of the 4 essential 'safety' mods to DCTCP (copied from the IETF > Internet Draft linked below) and one that might need to be essential: > o fall back to Reno or Cubic behaviour on loss; > > o negotiate its altered feedback semantics, which conveys the extent > of ECN marking, not just its existence, and this feedback needs to > be robust to loss [I-D.ietf-tcpm-accecn-reqs]; > > o handle a window of less than 2 when the RTT is low, rather than > increase the queue [TCP-sub-mss-w]. > > o average ECN feedback over its own RTT, not the hard-coded RTT > suitable only for data-centres, perhaps like Relentless > TCP [Mathis09]; > > > o Use of a standardised packet identifier (if ECN-capable is not enough) > > > oHeuristic testing for classic ECN bottlenecks (optional?) > > > > > We're trying to move fast because if we can get on top of other > developments (e.g. Apple's recent release of ECN), it will mean less > messy classification code in the AQM. > So the links below are not on official sites yet. > > ‘Data Centre to the Home’: Ultra-Low Latency for All > <http://www.bobbriscoe.net/projects/latency/dctth_preprint.pdf> > > Highlights: > * 1ms 99%-ile queuing delay for all DCTCP traffic in thousands of > expts incl. high load, > over an e2e test network with real broadband equipment. > * DCTCP co-existence with Reno & Cubic, with no transport ID inspection. > * less ops per packet than RED > * Zero config > > IETF Draft to standardise those parts of the AQM relevant to > interop(not yet submitted to IETF): > <http://www.bobbriscoe.net/projects/latency/draft-briscoe-aqm-dualq-coupled-00.txt> > > Koen & Bob