[iccrg] Invitation to subscribe to new DCTCP Evolution mailing list: ([e-mail])

[Bob Briscoe <[e-mail]>]

tcpm, tsvwg, tsvarea, iccrg lists

Recently, there have been developments (see URLs at end) that would make 
it possible to deploy scalable low-latency low-loss protocols like Data 
Center TCP alongside a mix of traffic, either in data centres and 
private networks, or even on the public Internet. One approach was 
demonstrated at the recent IETF in Prague, showing DCTCP giving 
ultra-low latency over a broadband Internet access while competing with 
a mix of Internet traffic on roughly equal terms.

As a result of an ad hoc meeting ("Bar BoF" = Birds of a Feather) at the 
Prague IETF, we have formed a new mailing list.
I'd like to invite you to join the list via: 
<https://www.ietf.org/mailman/listinfo/tcpprague>

The idea is to ensure that those working on DCTCP implementations across 
platforms (Free BSD, Linux, Windows, ...) will converge on solutions 
that will interwork with each other and with existing traffic. Although 
it is under the IETF's umbrella, we hope and expect that discussion will 
be as much about implementation as writing standards. However, we get 
the benefit of the IETF's IPR disclosure rules 
<https://www.ietf.org/about/note-well.html>, and of course it fits the 
IETF's purpose of interoperability.


The draft notes of the meeting are below.
And below that, is the original announcement with some context and 
background URLs.
You can catch up on any discussion you've missed using the list archives 
via the link above.


If you want to respond about something most relevant to tcpprague, pls 
avoid cross-posting to all the other lists in this announcement.


Cheers



Bob Briscoe



-------- Forwarded Message --------
Subject: 	Notes: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague
Date: 	Tue, 28 Jul 2015 14:00:46 +0100
From: 	Bob Briscoe <[e-mail]>
To: 	TCP Prague IETF List <[e-mail]>



Folks,

These notes have taken a week, because I've only just put my machine 
back together after having to rebuild the hardware a little :|


_*Notes: DCTCP Evolution Bar BoF*_
6-7pm Tue 21 Jul 2015, Budapest room, The Hilton, Prague, CZ

**Summary of Actions:*
*Lars E: Set up tcpprague wiki page
Bob B: Request [e-mail] mailing list, via IETF process 
(requires Area Director approval)
Bob B: Document Rationale - initiate a para on wiki.
Lars E: fwd Dagstuhl invitee list to Bob
Bob B: Set up list on wiki to assign people to invite those not in the 
room to join.
*
* 18:00 Introductions - name and interest **
***Present:
Marcelo    Bagnulo Braun <[e-mail]>
Praveen    Balasubramanian <[e-mail]>
Martin    Bekker <[e-mail]>
Bob    Briscoe <[e-mail]>
Anna    Brunstrom <[e-mail]>
Stuart    Cheshire <[e-mail]>
Koen    De Schepper <[e-mail]>
Fabien    Duchen <[e-mail]>
Phil    Eardley <[e-mail]>
Lars    Eggert <[e-mail]>
Michio    Honda <[e-mail]>
Per    Hurtig <[e-mail]>
Jana    Iyengar <[e-mail]>
Naeem    Khademi <[e-mail]>
Mirja    Kuehlewind <[e-mail]>
Matt    Mathis    <[e-mail] <mailto:[e-mail]>>
Andrew    McGregor <[e-mail]>
Karen    Nielsen <[e-mail]>
Tommy    Pauly <[e-mail]>
Andreas Petlund <[e-mail] <mailto:[e-mail]>>
Costin    Raiciu <[e-mail]>
Pasi    Sarolahti <[e-mail]>
Richard    Scheffenegger <[e-mail]>
David    Schinazi <[e-mail]>
Randall    Stewart <[e-mail]>
Dave    Thaler <[e-mail]>
Brian    Trammell   <[e-mail] <mIlto:[e-mail]>>
Michael    Tuexen <[e-mail]>
Felix    Weinrank <[e-mail]>
Michael    Welzl <[e-mail]>
Alex    Zimmermann <[e-mail]>

** Scope and Agenda Bashing**
***
[Non-italic text is from the materials pre-prepared by Koen De Schepper 
and Bob Briscoe.
/Italic text summarises conversation in the room./]

Meeting is covered by the standard IETF "Note Well" concerning 
intellectual property.

*Scope*:
* Evolving the e2e DCTCP protocol for use alongside existing traffic 
(whether in DCs, private nets or public Internet).
* Primarily to get DCTCP /developers/ involved early (Windows, FreeBSD, 
Linux), so that whatever we decide to standardise can be implemented in 
parallel
   (Doing implementation and standardisation in series is not desirable, 
in whichever order).
* Primarily an organisational meeting about creating a forum / community 
to do this work, using people's experience to know what will work best.

*Not in Scope:***
* Network changes are not in scope unless they impact the list of 
changes needed to DCTCP
* The in-network side of the solution (two approaches exist [DCttH 
<http://www.bobbriscoe.net/projects/latency/dctth_preprint.pdf>, Judd15 
<https://www.usenix.org/system/files/conference/nsdi15/nsdi15-paper-judd.pdf>], 
others may follow).*
** Identifier of DCTCP-like traffic (please discuss by email, not in 
this meeting)

/Lars E: Informational draft recording Microsoft's DCTCP should not be 
stalled by this, as it has value of its own.
    Unanimous agreement.
/
/Praveen S: Microsoft has offered a royalty free license for DCTCP IPR 
<https://datatracker.ietf.org/ipr/2319/>./

/Karen N: Is DCTCP over a non-TCP transport (e.g. SCTP) in scope//?
    Unanimous "Yes"//
/
/Outcome of discussion on the features of this DCTCP-like congestion 
control that define this work://
/

 1. /Must use ECN, but unlike RFC3168 ECN, marking is not merely
    equivalent to drop,//
    //so ECN signals can be more plentiful and sooner than drop./
 2. /P//acket rate is proportional to 1/p//, where p is the ECN marking
    probability. //
    /

/Matt M: 1/p makes congestion control scale with the bandwidth, //by 
making//the intensity of congestion control signals per RTT invariant//./
//
/Stuart Ch: Apple is turning on ECN by default in clients. Currently in 
developer seeds but probably in the next releases. Packet loss is also 
not a mystery./

** 18:15 List of /must-have/ changes before deployment alongside 
existing traffic.**
***
/Matt M: Rather than a "MUST-have" list, produce a prioritised list, 
because where to draw the necessity line could depend on the use-case.//
/
The following list wasn't formally prioritised in the meeting, but items 
where some people questioned necessity are shifted down.

 1. Fall back to Reno or Cubic behaviour on loss;
    /For how long?//quick consensus: 1 RTT, but needs further
    discussion. ECN response continues to operate in parallel./
 2. Negotiate altered feedback semantics, to convey the extent of ECN
    marking, not just its existence, and this feedback needs to be
    robust to loss [RFC-to-be 7560
    <https://datatracker.ietf.org/doc/draft-ietf-tcpm-accecn-reqs/>];
    /Mirja K, Richard S & Bob B plan to have spec of much simpler
    solution out soon. /
 3. Use of a standardised packet identifier (if ECN-capable is not enough)
    /Identifier tbd.
    /*/- - - 8< - - - - - - - - highest line between "must-have for
    safety" and "would be nice for performance" - - - - - -  8< - - - -/*
 4. Handle a window of less than 2 when the RTT is low, rather than
    increase the queue [TCP-sub-mss-cwnd
    <https://www.ietf.org/proceedings/93/slides/slides-93-iccrg-5.pdf>],
    like TCP Nice
    <http://www.cs.utexas.edu/users/dahlin/software/2002-nice.html>.
    /Michael W: Is this "must-have"? Quite a complicated step. //
    //Bob B: Yes, but, otherwise DCTCP will pollute ultra-low latency
    queues from the start./
 5. Average ECN feedback over its own RTT, not the hard-coded RTT
    suitable only for data-centres, perhaps reduce cwnd by seg-size/2
    per ECN Echo, like Relentless TCP [Mathis09
    <staff.psc.edu/mathis/papers/PFLDNet.pdf>];
    /???: How bad would long-RTT flows be?  More generally, how can we
    evaluate all this?/
    /Bob B: With mixed RTTs, flows with RTT > a couple of ms will
    respond too quickly to bursts.//Whatever, it's already been
    implemented by Mohammad Alizadeh in Linux, and evaluated
    <http://simula.stanford.edu/%7Ealizade/Site/DCTCP_files/dctcp_analysis-full.pdf>,
    so this is easy./
 6. Heuristic testing for classic ECN bottlenecks
    //The idea would be to detect a 'classic' RFC316 bottleneck by
    whether appreciable delay growth accompanies the marking (originally
    suggested by Michael W).
    /Bob B: Complex and slow to detect, so it would have to learn and
    cache for new flows - suggest this should only be a must-have if
    measurements prove it to be a problem - i.e. if a significant
    proportion of classic ECN bottlenecks //exist//
    //Matt M: No need for this - rate mismatch //no worse than TCP
    already sees with RTT discrepancies./
    /* - - - 8< - - - - - - - - lowest line between "must-have for
    safety" and "would be nice for performance" - - - - - -  8< - - - -*/
 7. /Costin R: //Faster-than-additive increase//(similar to Cubic)
    //A performance improvement, not "must-have", but would be nice to
    have while we're doing this./
 8. /[Not discussed in the meeting, but added by Bob B for the record]:
    Less drastic exit from slow-start, to match less drastic rate
    reduction per mark.//
    //Currently, because marking threshold is shallow, //slow start
    exits earlier than with drop, unnecessarily increasing completion
    time.//
    /

/
//Costin R: Any other way to evolve towards DCTCP over mixed networks, 
without separate queues in the network?//
////Bob B: To discuss on ML, and if we continue with the proposed 
approach, we must record the rationale on the WIki.//
///
** 18:30 Brainstorm to identify people not present who will be important 
to this.**
***
Mohammad    Alizadeh <[e-mail]>
Grenville    Armitage <[e-mail]>
Fred    Baker <[e-mail]>
Stephen    Bensley <[e-mail]>
Daniel    Borkmann <[e-mail]>
Yuchung    Cheng <[e-mail]>
Kenjiro    Cho <[e-mail]>
邓灵莉/Lingli    Deng <[e-mail]>
Eric    Dumazet <[e-mail]>
Gorry    Fairhurst <[e-mail]>
Jamal    Hadi Salim <[e-mail]>
Glenn    Judd <[e-mail]>
Midori    Kato <[e-mail]>
Kenneth    Klette Jonassen    <[e-mail] 
<mailto:[e-mail]>> (already subscribed)
Sridharan,    Murari <[e-mail]>
Hiren    Panchasara <[e-mail]>
Hagen     Pfeifer <[e-mail]>
Balaji    Prabhakar <[e-mail]>
KK    Ramakrishnan <[e-mail]>
Lawrence    Stewart <[e-mail]>
Dave    Taht <[e-mail]>
Florian    Westphal <[e-mail]>

/Agreed to cc to the following for awareness, but no need to invite to 
join the list://
/Stephen    Hemminger <[e-mail]>
David    Miller <[e-mail]>

/Missing types of organisations://
/

  * /Network operators (not so relevant for e2e protocol, but need to be
    motivated to deploy the network part)/
  * /CDN//s/

/[Bob B adds: Subsequent to mtg, Erik Nygren tells me Xin Zhang leads 
Akamai's congestion control team. Also I noticed Hiren used to work at 
Limelight, so may have contacts]//
////
//Lars E: Co-organising a Dagstuhl retreat around DCTCP. Will forward 
list of invitees to Bob to notify once the ML exists.//
//Also Lars's list of FreeBSD and Linux devs.//
///
** 18:40 What is the best way to ensure the outputs from a number of 
separate developers all converge in parallel to standardisation?**
***/Common Test Suite//
//Interop//events
////Plugfests//
////Serving paths (e.g. Google's) capable of serving this/

** 18:50 Next steps: Actions to set up suitable MLs, tools, with 
timesales etc.**
***
/Discussed pros and cons of hosting ML on ietf.org.//
//General agreement: use ietf.org for ML - because the IPR Note Well is 
useful. //
//
/Name for ML? /
//Matt M: TCP Prague (for an evolving protocol, a meaningless tag is 
best).//
//Karen N: ecn-prague, because it's not just TCP?//
//
//Agreed: //[e-mail]////
//
//Actions://
//Bob B: ML - ask SpencerD/MartinS, following the documented process//
//Lars E: Set up wiki page - for assigning people to send out invitations//
///
** End 19:05**
***

Notes: Bob Briscoe, helped by Andrew McGregor
28 Jul 2015


-------- Forwarded Message --------
Subject: 	DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague
Date: 	Mon, 20 Jul 2015 22:46:14 +0100
From: 	Bob Briscoe <[e-mail]>
To: 	Mirja Kuehlewind <[e-mail]>, EGGERT, Lars 
<[e-mail]>, Dave Thaler <[e-mail]>, Praveen 
Balasubramanian <[e-mail]>, Alex Zimmermann 
<[e-mail]>, Richard Scheffenegger 
<[e-mail]>, Fred Baker <[e-mail]>, Matt Mathis 
<[e-mail]>, Andrew McGregor <[e-mail]>, Dave 
Taht <[e-mail]>, Stuart Cheshire <[e-mail]>, 
Michael WELZL <[e-mail]>, Andreas Petlund 
<[e-mail]>, Gorry Fairhurst <[e-mail]>, Anna 
Brunstrom <[e-mail]>
CC: 	De Schepper, Koen (Koen) <[e-mail]>



Folks,

DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague
Location: Unless I have emailed with a room location before then, pls 
meet at the IETF reception.

Koen & I are trying to get together people in Prague who are involved in 
development of DCTCP across platforms (Windows, Free BSD, Linux, etc), 
and who are interested in evolving it for use on heterogeneous networks, 
e.g.
* data centres with a mix of TCP flavours, not just DCTCP
* private networks
* the public Internet

Pls fwd this invite to anyone in Prague who ought to be involved that 
I've missed (pls cc everyone else too).

Sorry for short notice.

One purpose of the session will be to build a community beyond the IETF, 
so I'd like us to compose an email to a wider set of people after the 
session, e.g.:

Stephen Bensley <[e-mail]> <mailto:[e-mail]>
Glenn Judd <[e-mail]> 
<mailto:[e-mail]>
Daniel Borkmann <[e-mail]> 
<mailto:[e-mail]>
Florian Westphal <[e-mail]> <mailto:[e-mail]>
邓 灵莉/Lingli Deng <[e-mail]> 
<mailto:[e-mail]>
Mohammad Alizadeh <[e-mail]> <mailto:[e-mail]>
Stephen Hemminger <[e-mail]> 
<mailto:[e-mail]>
David S. Miller <[e-mail]> <mailto:[e-mail]>
Sridharan, Murari <[e-mail] <mailto:[e-mail]>>
Yuchung Cheng <[e-mail] <mailto:[e-mail]>>


Koen & Bob

PS. Below is some background, and some agenda ideas. Pls discuss, bash 
and add your own.


> We've recently developed an AQM that allows DCTCP to co-exist with 
> Cubic/Reno etc. with zero config. Links below.
>
> We have to add some necessary mechanisms to DCTCP (listed below) so it 
> will be safe alongside other traffic. Two questions:
>
> Q1. We don't want to fork DCTCP. Does anyone think a fork optimised 
> for homogeneous DCTCP would be better?
>
> Q2. Anyone interested in helping?
> We have an idea how to do each one, but sharing the load would be 
> great - there's Linux, FreeBSD, Windows, etc. to cover.
>
> List of the 4 essential 'safety' mods to DCTCP (copied from the IETF 
> Internet Draft linked below) and one that might need to be essential:
>     o  fall back to Reno or Cubic behaviour on loss;
>   
>     o  negotiate its altered feedback semantics, which conveys the extent
>        of ECN marking, not just its existence, and this feedback needs to
>        be robust to loss [I-D.ietf-tcpm-accecn-reqs];
>   
>     o  handle a window of less than 2 when the RTT is low, rather than
>        increase the queue [TCP-sub-mss-w].
>   
>     o  average ECN feedback over its own RTT, not the hard-coded RTT
>        suitable only for data-centres, perhaps like Relentless
>        TCP [Mathis09];
>
>
>     o  Use of a standardised packet identifier (if ECN-capable is not enough)
>
>
>     oHeuristic testing for classic ECN bottlenecks (optional?)
>
>
>
>
> We're trying to move fast because if we can get on top of other 
> developments (e.g. Apple's recent release of ECN), it will mean less 
> messy classification code in the AQM.
> So the links below are not on official sites yet.
>
> ‘Data Centre to the Home’: Ultra-Low Latency for All
> <http://www.bobbriscoe.net/projects/latency/dctth_preprint.pdf>
>
> Highlights:
> * 1ms 99%-ile queuing delay for all DCTCP traffic in thousands of 
> expts incl. high load,
>    over an e2e test network with real broadband equipment.
> * DCTCP co-existence with Reno & Cubic, with no transport ID inspection.
> * less ops per packet than RED
> * Zero config
>
> IETF Draft to standardise those parts of the AQM relevant to 
> interop(not yet submitted to IETF):
> <http://www.bobbriscoe.net/projects/latency/draft-briscoe-aqm-dualq-coupled-00.txt>
>
>

Koen & Bob