[tcpPrague] Notes: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague

[Bob Briscoe <[e-mail]>]

Folks,

These notes have taken a week, because I've only just put my machine 
back together after having to rebuild the hardware a little :|


_*Notes: DCTCP Evolution Bar BoF*_
6-7pm Tue 21 Jul 2015, Budapest room, The Hilton, Prague, CZ

**Summary of Actions:*
*Lars E: Set up tcpprague wiki page
Bob B: Request [e-mail] mailing list, via IETF process 
(requires Area Director approval)
Bob B: Document Rationale - initiate a para on wiki.
Lars E: fwd Dagstuhl invitee list to Bob
Bob B: Set up list on wiki to assign people to invite those not in the 
room to join.
*
* 18:00 Introductions - name and interest **
***Present:
Marcelo    Bagnulo Braun <[e-mail]>
Praveen    Balasubramanian <[e-mail]>
Martin    Bekker <[e-mail]>
Bob    Briscoe <[e-mail]>
Anna    Brunstrom <[e-mail]>
Stuart    Cheshire <[e-mail]>
Koen    De Schepper <[e-mail]>
Fabien    Duchen <[e-mail]>
Phil    Eardley <[e-mail]>
Lars    Eggert <[e-mail]>
Michio    Honda <[e-mail]>
Per    Hurtig <[e-mail]>
Jana    Iyengar <[e-mail]>
Naeem    Khademi <[e-mail]>
Mirja    Kuehlewind <[e-mail]>
Matt    Mathis    <[e-mail] <mailto:[e-mail]>>
Andrew    McGregor <[e-mail]>
Karen    Nielsen <[e-mail]>
Tommy    Pauly <[e-mail]>
Andreas Petlund <[e-mail] <mailto:[e-mail]>>
Costin    Raiciu <[e-mail]>
Pasi    Sarolahti <[e-mail]>
Richard    Scheffenegger <[e-mail]>
David    Schinazi <[e-mail]>
Randall    Stewart <[e-mail]>
Dave    Thaler <[e-mail]>
Brian    Trammell   <[e-mail] <mIlto:[e-mail]>>
Michael    Tuexen <[e-mail]>
Felix    Weinrank <[e-mail]>
Michael    Welzl <[e-mail]>
Alex    Zimmermann <[e-mail]>

** Scope and Agenda Bashing**
***
[Non-italic text is from the materials pre-prepared by Koen De Schepper 
and Bob Briscoe.
/Italic text summarises conversation in the room./]

Meeting is covered by the standard IETF "Note Well" concerning 
intellectual property.

*Scope*:
* Evolving the e2e DCTCP protocol for use alongside existing traffic 
(whether in DCs, private nets or public Internet).
* Primarily to get DCTCP /developers/ involved early (Windows, FreeBSD, 
Linux), so that whatever we decide to standardise can be implemented in 
parallel
   (Doing implementation and standardisation in series is not desirable, 
in whichever order).
* Primarily an organisational meeting about creating a forum / community 
to do this work, using people's experience to know what will work best.

*Not in Scope:***
* Network changes are not in scope unless they impact the list of 
changes needed to DCTCP
* The in-network side of the solution (two approaches exist [DCttH 
<http://www.bobbriscoe.net/projects/latency/dctth_preprint.pdf>, Judd15 
<https://www.usenix.org/system/files/conference/nsdi15/nsdi15-paper-judd.pdf>], 
others may follow).*
** Identifier of DCTCP-like traffic (please discuss by email, not in 
this meeting)

/Lars E: Informational draft recording Microsoft's DCTCP should not be 
stalled by this, as it has value of its own.
    Unanimous agreement.
/
/Praveen S: Microsoft has offered a royalty free license for DCTCP IPR 
<https://datatracker.ietf.org/ipr/2319/>./

/Karen N: Is DCTCP over a non-TCP transport (e.g. SCTP) in scope//?
    Unanimous "Yes"//
/
/Outcome of discussion on the features of this DCTCP-like congestion 
control that define this work://
/

 1. /Must use ECN, but unlike RFC3168 ECN, marking is not merely
    equivalent to drop,//
    //so ECN signals can be more plentiful and sooner than drop./
 2. /P//acket rate is proportional to 1/p//, where p is the ECN marking
    probability. //
    /

/Matt M: 1/p makes congestion control scale with the bandwidth, //by 
making//the intensity of congestion control signals per RTT invariant//./
////
/Stuart Ch: Apple is turning on ECN by default in clients. Currently in 
developer seeds but probably in the next releases. Packet loss is also 
not a mystery./

** 18:15 List of /must-have/ changes before deployment alongside 
existing traffic.**
***
/Matt M: Rather than a "MUST-have" list, produce a prioritised list, 
because where to draw the necessity line could depend on the use-case.//
/
The following list wasn't formally prioritised in the meeting, but items 
where some people questioned necessity are shifted down.

 1. Fall back to Reno or Cubic behaviour on loss;
    /For how long?//quick consensus: 1 RTT, but needs further
    discussion. ECN response continues to operate in parallel./
 2. Negotiate altered feedback semantics, to convey the extent of ECN
    marking, not just its existence, and this feedback needs to be
    robust to loss [RFC-to-be 7560
    <https://datatracker.ietf.org/doc/draft-ietf-tcpm-accecn-reqs/>];
    /Mirja K, Richard S & Bob B plan to have spec of much simpler
    solution out soon. /
 3. Use of a standardised packet identifier (if ECN-capable is not enough)
    /Identifier tbd.
    /*/- - - 8< - - - - - - - - highest line between "must-have for
    safety" and "would be nice for performance" - - - - - -  8< - - - -/*
 4. Handle a window of less than 2 when the RTT is low, rather than
    increase the queue [TCP-sub-mss-cwnd
    <https://www.ietf.org/proceedings/93/slides/slides-93-iccrg-5.pdf>],
    like TCP Nice
    <http://www.cs.utexas.edu/users/dahlin/software/2002-nice.html>.
    /Michael W: Is this "must-have"? Quite a complicated step. //
    //Bob B: Yes, but, otherwise DCTCP will pollute ultra-low latency
    queues from the start./
 5. Average ECN feedback over its own RTT, not the hard-coded RTT
    suitable only for data-centres, perhaps reduce cwnd by seg-size/2
    per ECN Echo, like Relentless TCP [Mathis09
    <staff.psc.edu/mathis/papers/PFLDNet.pdf>];
    /???: How bad would long-RTT flows be?  More generally, how can we
    evaluate all this?/
    /Bob B: With mixed RTTs, flows with RTT > a couple of ms will
    respond too quickly to bursts.//Whatever, it's already been
    implemented by Mohammad Alizadeh in Linux, and evaluated
    <http://simula.stanford.edu/%7Ealizade/Site/DCTCP_files/dctcp_analysis-full.pdf>,
    so this is easy./
 6. Heuristic testing for classic ECN bottlenecks
    //The idea would be to detect a 'classic' RFC316 bottleneck by
    whether appreciable delay growth accompanies the marking (originally
    suggested by Michael W).
    /Bob B: Complex and slow to detect, so it would have to learn and
    cache for new flows - suggest this should only be a must-have if
    measurements prove it to be a problem - i.e. if a significant
    proportion of classic ECN bottlenecks //exist//
    //Matt M: No need for this - rate mismatch //no worse than TCP
    already sees with RTT discrepancies./
    /* - - - 8< - - - - - - - - lowest line between "must-have for
    safety" and "would be nice for performance" - - - - - -  8< - - - -*/
 7. /Costin R: //Faster-than-additive increase//(similar to Cubic)
    //A performance improvement, not "must-have", but would be nice to
    have while we're doing this./
 8. /[Not discussed in the meeting, but added by Bob B for the record]:
    Less drastic exit from slow-start, to match less drastic rate
    reduction per mark.//
    //Currently, because marking threshold is shallow, //slow start
    exits earlier than with drop, unnecessarily increasing completion
    time.//
    /

/
//Costin R: Any other way to evolve towards DCTCP over mixed networks, 
without separate queues in the network?//
////Bob B: To discuss on ML, and if we continue with the proposed 
approach, we must record the rationale on the WIki.//
///
** 18:30 Brainstorm to identify people not present who will be important 
to this.**
***
Mohammad    Alizadeh    <[e-mail]>
Grenville    Armitage    <[e-mail]>
Fred    Baker    <[e-mail]>
Stephen    Bensley    <[e-mail]>
Daniel    Borkmann    <[e-mail]>
Yuchung    Cheng    <[e-mail]>
Kenjiro    Cho    <[e-mail]>
邓灵莉/Lingli    Deng    <[e-mail]>
Eric    Dumazet    <[e-mail]>
Gorry    Fairhurst <[e-mail]>
Jamal    Hadi Salim    <[e-mail]>
Glenn    Judd    <[e-mail]>
Midori    Kato    <[e-mail]>
Kenneth    Klette Jonassen    <[e-mail] 
<mailto:[e-mail]>> (already subscribed)
Sridharan,    Murari    <[e-mail]>
Hiren    Panchasara    <[e-mail]>
Hagen     Pfeifer    <[e-mail]>
Balaji    Prabhakar    <[e-mail]>
KK    Ramakrishnan    <[e-mail]>
Lawrence    Stewart    <[e-mail]>
Dave    Taht    <[e-mail]>
Florian    Westphal    <[e-mail]>

/Agreed to cc to the following for awareness, but no need to invite to 
join the list://
/Stephen    Hemminger    <[e-mail]>
David    Miller    <[e-mail]>

/Missing types of organisations://
/

  * /Network operators (not so relevant for e2e protocol, but need to be
    motivated to deploy the network part)/
  * /CDN//s/

/[Bob B adds: Subsequent to mtg, Erik Nygren tells me Xin Zhang leads 
Akamai's congestion control team. Also I noticed Hiren used to work at 
Limelight, so may have contacts]//
////
//Lars E: Co-organising a Dagstuhl retreat around DCTCP. Will forward 
list of invitees to Bob to notify once the ML exists.//
//Also Lars's list of FreeBSD and Linux devs.//
///
** 18:40 What is the best way to ensure the outputs from a number of 
separate developers all converge in parallel to standardisation?**
***/Common Test Suite//
//Interop//events
////Plugfests//
////Serving paths (e.g. Google's) capable of serving this/

** 18:50 Next steps: Actions to set up suitable MLs, tools, with 
timesales etc.**
***
/Discussed pros and cons of hosting ML on ietf.org.//
//General agreement: use ietf.org for ML - because the IPR Note Well is 
useful. //
//
/Name for ML? /
//Matt M: TCP Prague (for an evolving protocol, a meaningless tag is 
best).//
//Karen N: ecn-prague, because it's not just TCP?//
//
//Agreed: //[e-mail]////
//
//Actions://
//Bob B: ML - ask SpencerD/MartinS, following the documented process//
//Lars E: Set up wiki page - for assigning people to send out invitations//
///
** End 19:05**
***

Notes: Bob Briscoe, helped by Andrew McGregor
28 Jul 2015


=============================================================================================
Below I've pasted some of the discussion that went on between a growing 
group of people before the mailing list was formed.

On 21/07/15 13:44, Bob Briscoe wrote:
> Folks, [Adding Jana & Pasi (as tcpm co-chair)]
>
> I have booked the Budapest room on Lobby Level in the IETF Hotel 
> (Hilton Prague).
> Arrive 17:50 CET (*no longer 17:40*).
> I have the room from 18:00, but let's all get there to start on time.
>
> It may be a squeeze so arrive early. There are now 29 on the list - I 
> hope I have captured everyone, including those who were added, but 
> crossing postings dropped them off again.
>
> *Scope*: Evolving the e2e DCTCP protocol for use alongside existing 
> traffic (whether in DCs, private nets or public Internet).
>
> This session is primarily to get DCTCP /developers/ involved early 
> (Windows, FreeBSD, Linux), so that whatever we decide to standardise 
> can be implemented in parallel. Doing implementation and 
> standardisation in series is not desirable, in whichever order.
>
> So, it is primarily an organisational meeting about creating a forum / 
> community to do this work, using people's experience to know what will 
> work best.
>
> *Not in Scope:**
> ** Network changes are not in scope unless they impact the list of 
> changes needed to DCTCP that Koen and I posted in the original email.
> * I am only aware of two solutions in the network that enable DCTCP 
> co-existence (i) the one Koen and I link to; (ii) the one Glenn Judd 
> used in Morgan Stanley (Diffserv segregation or capacity). If people 
> think there are others, please keep today's discussion focused on 
> whether any different changes to DCTCP will be needed than those 
> listed above.
> * Please /do not/ describe in-network solutions in this Bar BoF.
>
> *Identifier**Out Of Scope for Today?
> *Please continue to discuss whether and what identifier will be needed 
> for different traffic treatment /by email/, but I suggest we do /not/ 
> discuss this in the session. We know there are at least three possible 
> solutions, and one will have to be picked during standardisation - so 
> that is not priority #1 task today.
>
> *Proposed Agenda (for bashing)**
> *
> * 18:00 Introductions - name and interest
> * 18:10 List of /must-have/ changes before deployment alongside 
> existing traffic.
> * 18:30 Brainstorm to identify people not present that will be 
> important to this.
> * 18:40 What is the best way to ensure the outputs from a number of 
> separate developers all converge in parallel to standardisation?
> * 18:50 Next steps: Actions to set up suitable MLs, tools, with 
> timesales etc.
> * End 19:00
>
> *List of must-have DCTCP changes* we think will be needed:
>
>     o   fall back to Reno or Cubic behaviour on loss;
>     o  negotiate its altered feedback semantics, which conveys the extent
>        of ECN marking, not just its existence, and this feedback needs to
>        be robust to loss [I-D.ietf-tcpm-accecn-reqs];
>     o  handle a window of less than 2 when the RTT is low, rather than
>        increase the queue [TCP-sub-mss-w].
>     o  average ECN feedback over its own RTT, not the hard-coded RTT
>        suitable only for data-centres, perhaps like Relentless
>        TCP [Mathis09];
>
>     o  Use of a standardised packet identifier (if ECN-capable is not enough)
>   *BUT no discussion of which identifier*!!!
>
>     oHeuristic testing for classic ECN bottlenecks (optional?)
>
>
>
> Bob
>
> On 21/07/15 12:40, Karen Elisabeth Egede Nielsen wrote:
>>
>> adding Michael
>>
>> *From:*Naeem Khademi [mailto:[e-mail] 
>> <mailto:[e-mail]>]
>> *Sent:* Tuesday, July 21, 2015 1:22 PM
>> *To:* De Schepper, Koen (Koen)
>> *Cc:* Matt Mathis; Michael Welzl; [e-mail] 
>> <mailto:[e-mail]>; [e-mail] 
>> <mailto:[e-mail]>; [e-mail] <mailto:[e-mail]>; 
>> Karen E. E. Nielsen; [e-mail] <mailto:[e-mail]>; Bob 
>> Briscoe; Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen 
>> Balasubramanian; Alex Zimmermann; Richard Scheffenegger; Fred Baker; 
>> Andrew McGregor; Dave Taht; Stuart Cheshire; Andreas Petlund; Anna 
>> Brunstrom; Per Hurtig; Tommy Pauly; David Schinazi
>> *Subject:* Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague
>>
>> On Tue, Jul 21, 2015 at 1:00 PM, De Schepper, Koen (Koen) 
>> <[e-mail] 
>> <mailto:[e-mail]>> wrote:
>>
>> Hi Naeem,
>>
>> >> I'd be cautious to speak of "Scalable TCP" as a separate item from 
>> "Scalable ECN"
>>
>> >> as scalable TCP will *necessarily* need ECN (with instantaneous 
>> making and no
>>
>> >> averaging on the AQM side) to work.
>>
>> Well, this seems to be a common misunderstanding. A scalable TCP 
>> (like DCTCP) needs only
>>
>> a DON’T think twice ECN (which I call a scalable ECN). So if you mark 
>> with p and drop with p² it “works”.
>>
>> All other things you refer to are optimizations for having real low 
>> queuing latency as well (which are only possible with scalable TCP).
>>
>> To me getting a low latency translates into: low AQM thresholds + 
>> small TCP sawteeth + ECN  (to avoid losses with small sawteeth) => 
>> can be achieved with DCTCP or a beta_{ecn} that is large enough. All 
>> other issues are only about backward compatibility with legacy TCP 
>> (fairness, convergence, etc if ever needed).
>>
>> Cheers,
>>
>> Naeem
>>
>>     On a single queue with an ARED, curvyRED, PIE or CoDel controller
>>     it “works” (meaning it does not cause starvation, gives fairness,
>>     less wobbling flow throughput and throughput independent marking
>>     feedback). If unavoidable, even smoothing and delays are allowed,
>>     with some impact on latency of course, but it “works”.
>>
>>     This is good news, as single queue routers just need 2 random
>>     values for drop (making it curvyRED) and one for marking. This
>>     might be a very simple upgrade for current hardware. 2 random
>>     variables can be “produced” simply, for example by remembering
>>     the previous one.
>>
>>     >> is it about improving DCTCP for data center (which its
>>     original design was intended
>>
>>     >> for)? or improving it for the use in the general Internet?
>>
>>     That is a good question. I guess it depends on whether the
>>     “improvement” will also improve pure datacenter deployment (then
>>     it is a no-brainer), or otherwise if coexistence is required in
>>     the data center.
>>
>>     Regards,
>>
>>     Koen.
>>
>>     *From:*Naeem Khademi [mailto:[e-mail]
>>     <mailto:[e-mail]>]
>>     *Sent:* dinsdag 21 juli 2015 12:35
>>     *To:* De Schepper, Koen (Koen)
>>     *Cc:* Matt Mathis; Michael Welzl; [e-mail]
>>     <mailto:[e-mail]>; [e-mail]
>>     <mailto:[e-mail]>; [e-mail] <mailto:[e-mail]>;
>>     Karen E. E. Nielsen; [e-mail] <mailto:[e-mail]>;
>>     Bob Briscoe; Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen
>>     Balasubramanian; Alex Zimmermann; Richard Scheffenegger; Fred
>>     Baker; Andrew McGregor; Dave Taht; Stuart Cheshire; Andreas
>>     Petlund; Anna Brunstrom
>>
>>
>>     *Subject:* Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40,
>>     Prague
>>
>>     On Tue, Jul 21, 2015 at 12:16 PM, De Schepper, Koen (Koen)
>>     <[e-mail]
>>     <mailto:[e-mail]>> wrote:
>>
>>     Hi All,
>>
>>     The meeting is indeed to discuss improvements of DCTCP. Hopefully
>>     it does not get completely hijacked by the deployment story ;-).
>>     But it is an important topic to work on and to take into account
>>     for some of the other safety improvements. I see there is a lot
>>     of interest on this topic, so enough volunteers to work this ;-).
>>
>>     Also for people who want to experience the power of using a
>>     scalable TCP and a scalable ECN (L4S) compared with that of the
>>     classic ones, the bitsNbytes demo is up and running on location.
>>     Give me a call or sms ([phone]) if you want to feel it. If
>>     people have an application on a client and server doing DCTCP, we
>>     could even do interop testing. We still have a few slots in our
>>     switches (only wired for now).
>>
>>     Thanks Koen for the info.
>>
>>     Just a minor side note: I'd be cautious to speak of "Scalable
>>     TCP" as a separate item from "Scalable ECN" as scalable TCP will
>>     *necessarily* need ECN (with instantaneous making and no
>>     averaging on the AQM side) to work. Apart than than DCTCP has
>>     been around for a while, and would have perhaps been a great
>>     thing if legacy traffic (loss-based or legacy ECN) had never
>>     existed in the first place and Internet had evolved in a way that
>>     we only had DCTCP today. Now that we know this is not the case,
>>     it would perhaps be of significant importance to discuss the
>>     deployment path and backward compatibility in great details IMHO.
>>
>>     Another question about BarBof: is it about improving DCTCP for
>>     data center (which its original design was intended for)? or
>>     improving it for the use in the general Internet? from my
>>     understanding these are two separate issues and both should be
>>     addressed but better to be on the same page perhaps?
>>
>>     Cheers,
>>
>>     Naeem
>>
>>         Regards,
>>
>>         Koen.
>>
>>         *From:*Matt Mathis [mailto:[e-mail]]
>>         *Sent:* dinsdag 21 juli 2015 8:51
>>         *To:* Michael Welzl
>>         *Cc:* [e-mail] <mailto:[e-mail]>;
>>         [e-mail] <mailto:[e-mail]>; [e-mail]
>>         <mailto:[e-mail]>; Karen E. E. Nielsen; [e-mail]
>>         <mailto:[e-mail]>; De Schepper, Koen (Koen); Bob
>>         Briscoe; Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen
>>         Balasubramanian; Alex Zimmermann; Richard Scheffenegger; Fred
>>         Baker; Andrew McGregor; Dave Taht; Stuart Cheshire; Andreas
>>         Petlund; Anna Brunstrom; Naeem Khademi
>>
>>
>>         *Subject:* Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015,
>>         17:40, Prague
>>
>>         I don't think it matters....
>>
>>         Given that TCP friendliness is but an illusion, I don't
>>         believe that differing ECN interpretations matter as much as
>>         people think.
>>
>>         In the core any congestion causes egregious unfairness, no
>>         matter what the network and stacks try to do.  Strive to
>>         eliminate congestion in the core.
>>
>>         Near the network edges, there is an extremely powerful
>>         workaround that is already in play: have enough capacity and
>>         avoid uncontrolled background traffic when there is something
>>         important in the foreground.
>>
>>         These are powerful enough to bridge most of any transition....
>>
>>         The danger does not come from deploying multiple versions of
>>         ECN at the edges, but deploying ECN in "substitute for loss"
>>         mode in the network.  Network devices that independently mark
>>         and drop are not grossly incompatible with either edge
>>         behavior (although admittedly not optimal).
>>
>>         Thanks,
>>
>>         --MM--
>>         Matt Mathis Home/Cell: 650-417-3029
>>         -------------------------------------------
>>         Evil is defined by mortals who think they know "The Truth"
>>         and use force to apply it to others.
>>
>>         On Mon, Jul 20, 2015 at 11:42 PM, Michael Welzl
>>         <[e-mail] <mailto:[e-mail]>> wrote:
>>
>>         +1, and cc the people that have been newly added because they
>>         missed this thread. Read the bottom email first, then the one
>>         above, which is Koen's answer to me with Gorry's answer to
>>         that inline.
>>
>>         Cheers,
>>         Michael
>>
>>
>>
>>         > On 21. jul. 2015, at 08.29, [e-mail]
>>         <mailto:[e-mail]> wrote:
>>         >
>>         > Hi all,
>>         >
>>         > So  I thought this was a Bar-BOF to discuss how to fix DCTP
>>         and take on a
>>         > road map to get this made robust. I would have been
>>         interested in this.
>>         > However, as I read on I now perceive the BOF as a trick to
>>         introduce your
>>         > own new idea for how to replace the operation of AQM 
>>         within routers -
>>         > something quite different.
>>         >
>>         >> Hi Michael,
>>         >>
>>         >> According to me it means that there is still the
>>         possibility to hijack the
>>         >> Classic ECN bits and to use them for Scalable ECN.
>>         >>
>>         > I think this is still craziness.
>>         >
>>         >> It might be simpler
>>         >> today to contain Classic ECN in a controlled environment
>>         (with diffserv
>>         >> codepoint?) than future Scalable ECN. I suppose that if
>>         there is Classic
>>         >> ECN deployment today, it is already in a very controlled
>>         environment, and
>>         >> we hope that it will get deprecated soon as the advantages
>>         of scalable ECN
>>         >> and congestion control are huge compared to the Classic
>>         ones...
>>         >>
>>         > This we have talked about - my take this is that this a
>>         plain ridiculous
>>         > position.
>>         >
>>         >> I guess the three options are ignore Classic ECN, contain
>>         Scalable ECN, or
>>         >> contain Classic ECN. In the last 2 cases, we need to
>>         define that
>>         >> container.
>>         >>
>>         >> Regards,
>>         >> Koen.
>>         >>
>>         > Gorry
>>         >
>>         >>> -----Original Message-----
>>         >>> From: Michael Welzl [mailto:[e-mail]
>>         <mailto:[e-mail]>]
>>         >>> Sent: maandag 20 juli 2015 23:58
>>         >>> To: Bob Briscoe
>>         >>> Cc: Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen
>>         >>> Balasubramanian; Alex Zimmermann; Richard Scheffenegger;
>>         Fred Baker;
>>         >>> Matt Mathis; Andrew McGregor; Dave Taht; Stuart Cheshire;
>>         Andreas
>>         >>> Petlund; Gorry Fairhurst; Anna Brunstrom; De Schepper,
>>         Koen (Koen);
>>         >>> Naeem Khademi
>>         >>> Subject: Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015,
>>         17:40, Prague
>>         >>>
>>         >>> [ + Naeem Khademi, who I think is also interested ]
>>         >>>
>>         >>>
>>         >>> Hi,
>>         >>>
>>
>>         >>> I’m reacting to:
>>         >>>
>>         >>>> PS. Below is some background, and some agenda ideas. Pls
>>         discuss,
>>         >>> bash and add your own.
>>         >>>
>>         >>> when I say:
>>         >>>
>>         >>> This is about something that’s not compatible at all
>>         with today’s
>>         >>> ECN.
>>         >>> That is, senders that use ECN and react to it like the
>>         spec so far says
>>         >>> will be beat to death by DCTCP with this scheme, unless
>>         we have a way
>>         >>> to classify this as being as something else, and queue
>>         this traffic
>>         >>> separately. Luckily, in AQM today, you said at the mic
>>         that you’re
>>         >>> considering usage of ECT(1) and probably a DSCP.
>>         >>>
>>         >>> But then I don’t understand this:
>>         >>>
>>         >>>>> We're trying to move fast because if we can get on top
>>         of other
>>         >>> developments (e.g. Apple's recent release of ECN), it
>>         will mean less
>>         >>> messy classification code in the AQM.
>>         >>>
>>         >>> You say “get on top of other developments†and you
>>         want to avoid
>>         >>> “less
>>         >>> messy classification code†. What exactly do you mean?
>>         >>>
>>         >>> Cheers,
>>         >>> Michael
>>         >>
>>         >>
>>         >
>>
>
>

-------- Forwarded Message --------
Subject: 	DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague
Date: 	Mon, 20 Jul 2015 22:46:14 +0100
From: 	Bob Briscoe <[e-mail]>
To: 	Mirja Kuehlewind <[e-mail]>, EGGERT, Lars 
<[e-mail]>, Dave Thaler <[e-mail]>, Praveen 
Balasubramanian <[e-mail]>, Alex Zimmermann 
<[e-mail]>, Richard Scheffenegger 
<[e-mail]>, Fred Baker <[e-mail]>, Matt Mathis 
<[e-mail]>, Andrew McGregor <[e-mail]>, Dave 
Taht <[e-mail]>, Stuart Cheshire <[e-mail]>, 
Michael WELZL <[e-mail]>, Andreas Petlund 
<[e-mail]>, Gorry Fairhurst <[e-mail]>, Anna 
Brunstrom <[e-mail]>
CC: 	De Schepper, Koen (Koen) <[e-mail]>



Folks,

DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague
Location: Unless I have emailed with a room location before then, pls 
meet at the IETF reception.

Koen & I are trying to get together people in Prague who are involved in 
development of DCTCP across platforms (Windows, Free BSD, Linux, etc), 
and who are interested in evolving it for use on heterogeneous networks, 
e.g.
* data centres with a mix of TCP flavours, not just DCTCP
* private networks
* the public Internet

Pls fwd this invite to anyone in Prague who ought to be involved that 
I've missed (pls cc everyone else too).

Sorry for short notice.

One purpose of the session will be to build a community beyond the IETF, 
so I'd like us to compose an email to a wider set of people after the 
session, e.g.:

Stephen Bensley <[e-mail]> <mailto:[e-mail]>
Glenn Judd <[e-mail]> 
<mailto:[e-mail]>
Daniel Borkmann <[e-mail]> 
<mailto:[e-mail]>
Florian Westphal <[e-mail]> <mailto:[e-mail]>
邓灵莉/Lingli Deng <[e-mail]> 
<mailto:[e-mail]>
Mohammad Alizadeh <[e-mail]> <mailto:[e-mail]>
Stephen Hemminger <[e-mail]> 
<mailto:[e-mail]>
David S. Miller <[e-mail]> <mailto:[e-mail]>
Sridharan, Murari <[e-mail] <mailto:[e-mail]>>
Yuchung Cheng <[e-mail] <mailto:[e-mail]>>


Koen & Bob

PS. Below is some background, and some agenda ideas. Pls discuss, bash 
and add your own.


> We've recently developed an AQM that allows DCTCP to co-exist with 
> Cubic/Reno etc. with zero config. Links below.
>
> We have to add some necessary mechanisms to DCTCP (listed below) so it 
> will be safe alongside other traffic. Two questions:
>
> Q1. We don't want to fork DCTCP. Does anyone think a fork optimised 
> for homogeneous DCTCP would be better?
>
> Q2. Anyone interested in helping?
> We have an idea how to do each one, but sharing the load would be 
> great - there's Linux, FreeBSD, Windows, etc. to cover.
>
> List of the 4 essential 'safety' mods to DCTCP (copied from the IETF 
> Internet Draft linked below) and one that might need to be essential:
>     o  fall back to Reno or Cubic behaviour on loss;
>   
>     o  negotiate its altered feedback semantics, which conveys the extent
>        of ECN marking, not just its existence, and this feedback needs to
>        be robust to loss [I-D.ietf-tcpm-accecn-reqs];
>   
>     o  handle a window of less than 2 when the RTT is low, rather than
>        increase the queue [TCP-sub-mss-w].
>   
>     o  average ECN feedback over its own RTT, not the hard-coded RTT
>        suitable only for data-centres, perhaps like Relentless
>        TCP [Mathis09];
>
>
>     o  Use of a standardised packet identifier (if ECN-capable is not enough)
>
>
>     oHeuristic testing for classic ECN bottlenecks (optional?)
>
>
>
>
> We're trying to move fast because if we can get on top of other 
> developments (e.g. Apple's recent release of ECN), it will mean less 
> messy classification code in the AQM.
> So the links below are not on official sites yet.
>
> ‘Data Centre to the Home’: Ultra-Low Latency for All
> <http://www.bobbriscoe.net/projects/latency/dctth_preprint.pdf>
>
> Highlights:
> * 1ms 99%-ile queuing delay for all DCTCP traffic in thousands of 
> expts incl. high load,
>    over an e2e test network with real broadband equipment.
> * DCTCP co-existence with Reno & Cubic, with no transport ID inspection.
> * less ops per packet than RED
> * Zero config
>
> IETF Draft to standardise those parts of the AQM relevant to 
> interop(not yet submitted to IETF):
> <http://www.bobbriscoe.net/projects/latency/draft-briscoe-aqm-dualq-coupled-00.txt>
>
>

Koen & Bob