slow-start related performance bugs in TCP

[John Heidemann <[e-mail]>]

Dan Coolidge wrote:
> This is all well and good, but thus far a bit too nebulous for my liking. I
> would ask of anybody who is planning on attending the BOF and who has a
> specific TCP implementation problem to discuss, that they bring it up, in
> advance, on this mailing list.

We would like to discuss a couple of performance problems in many TCP
implementations:

Slow-start/Delayed-ACK interactions
	We have found two interactions between the slow-start and
	delayed ACK algorithms that force TCP to stall until
	a delayed ACK triggers (which takes an average of 100ms
	in BSD-based TCP stacks).  For short connections
	(common in HTTP traffic) the effects of this stall
	can be significant.

	The first interaction occurs when the congestion window (cwnd)
	is at 1x MSS (as at the start of a connection, or after an idle
	period as described below).  The sender transmits the single
	packet allowed by cwnd, but the receiver delays ACKing this
	packet because immediate ACKs are only required after two
	full packets.

	The second interaction occurs when the cwnd is at 2x MSS.
	If the sender transmits a less-than full-size segment
	the receiver will refuse to ACK immediately
	because delayed ACKs require receipt of two *full-MSS*
	segments for an immediate ACK.

Slow-start restart inconsistencies
	Current TCPs restart slow-start in two ways that are not
	always optimal:  when the connection has been idle for a long
	time and when the back-end of the window jumps by a large
	amount.  TCP initiates slow-start to avoid sending a large
	number of back-to-back packets since the ACK clock has become
	disconnected.

	There are two problems here:  first, many implementations fail to
	slow-start after connection idleness (thus sending large
	packet trains).  4.4BSD (and derivatives) and Linux close the
	window (to 1x MSS), many other TCP stacks don't (including at least
	SunOS).  Is there consensus on what should be done here?

	Second, we believe that slow-starting is overly conservative.
	We are currently experimenting with rate-based pacing
	to re-start the	ACK clock.  Rather than send back-to-back
	packets or slow-start, we use the old connection statistics
	to pace outgoing packets at a rate conservatively below
	the prior rate, but faster than slow-start.  This approach
	provides better throughput while avoiding 
	overly aggressive transmission.

We have encountered each of these problems in the context of P-HTTP
(see ``Performance Interactions Between P-HTTP and TCP
Implementations'' at
<URL:http://www.isi.edu/~johnh/PAPERS/Heidemann96b.html>).  We are
investigating these problems as a part of TCP del Rey, an effort at
ISI to tune TCP for 5-20KB transactions common for web and distributed
object systems.

In addition to the work described here we also plan on examining TCP
control-block sharing issues raised by T/TCP and concurrent TCP
connections between pairs of hosts (see
draft-touch-tcp-interdep-00.txt for details of this work).

Are these problems implementation or research issues?  
Let's look at the parts:

    - rate-based pacing:  research issue

    - TCP control block sharing:  research issue

    - cwnd on connection idle:  I think that clarification of what
	should be done here is an important implementation issue.
	Jacobson introduced the problem in his 1990 revision of
	``Congestion Avoidance and Control'' but it hasn't been
	nailed down by an RFC one way or the other.

    - slow-start/delayed-ACK interactions:  These seem like small
	performance bugs that may not have been anticipated in the
	specification.  I'd like to see them resolved.

Comments?

   -John Heidemann, Joe Touch, Katia Obraczka, Ted Faber