Re: [tcpm] Summary of mt comments/questions WGLC for draft-ietf-tcpm-fastopen-05

["Scharf, Michael (Michael)" <[e-mail]>]

Hi Yuchung,

I guess one of the key points here is that in corner cases the cached MSS value might be outdated. For instance, this could happen if a load balancer sits in front of two middleboxes that are (mis-) configured with two different MSS clamping values.

And legacy middlebox may not expect data in the SYN and behave different to what one could expect. I am not sure whether this is a real-world problem. But, indeed, it seems like an area where further experimentation could provide better insight (including the potential outcome that this is not a problem). I strongly agree that this should be added to the experimentation section.

Thanks

Michael


From: Yuchung Cheng [mailto:[e-mail]]
Sent: Tuesday, December 03, 2013 1:32 AM
To: Scharf, Michael (Michael)
Cc: Gorry Fairhurst; [e-mail]; [e-mail]
Subject: Re: [tcpm] Summary of mt comments/questions WGLC for draft-ietf-tcpm-fastopen-05



On Mon, Dec 2, 2013 at 5:49 AM, Scharf, Michael (Michael) <[e-mail]<mailto:[e-mail]>> wrote:
Some comments, with chair hat off...

a)+b):

I have no strong opinion on whether the connection initiator (client) should cache the RTT, or not. It seems partly orthogonal to TFO. If the RTO expires spuriously, the document recommends to retransmit without data in SYN, i.e., the amount of bytes sent into the network is the same as for regular TCP. From a congestion control perspective, the result is more or less the same.

However, after having followed this discussion, I wonder why the document does not comment at all about RTT estimation at the connection acceptor (server). With TFO, the connection acceptor (server) is allowed to send a full IW of data back to the connection initiator (client), following the SYN-ACK. However, at that point in time, the connection acceptor (server) has no recent RTT measurement, right? This is different to the traditional TCP handshake, in which
correct
an RTT measurement is available before data is sent into the network.  The fact that TFO allows sending of a full IW of data before an RTT estimation is available should be mentioned as a warning sign, IMHO. For instance, it will make it more difficult to apply pacing.
I can add that point about RTT estimation.


c)+d):

In almost all existing use of regular TCP, the endpoints negotiate the MSS before actually sending any data, because adding data to SYNs doesn't make a lot of sense without TFO. Therefore, "MSS clamping" in middleboxes usually works for regular TCP, and it may avoid PMTUD procedures in many cases. With TFO, the situation can be more
I need more information about the MSS clamping feature.

My understanding (which can be wrong) of MSS clamping is that the middle-box is transparently reducing the value in the MSS option in the SYN packet. If so the connection that negotiate the cookie, should receive and cache the reduced value, which will be used by the later Fast Open connection.

It is possible the middle-box lowers the MSS prior the to SYN-data. In that case it is equivalent to sending a SYN-data, or a data packet on a new path that has lower MTU. In that case, the remedy is ICMP error or ultimately a timeout. When recurring SYN-data timeouts happen, then the client should disable Fast Open (temporarily) due to the negative caching suggested in the draft.

complex. In addition to the case of MSS<536 bytes raised by Gorry, it is also not clear to me what a middlebox should do if it receives a TFO SYN with data, and if this segment is larger than the "MSS clamping" value of that middlebox. This corner case can occur e.g. after path routing changes. Therefore, I agree with Gorry that the document should at least mention that MSS issues may require further experimentation.
As replied earlier, I will document SYN-data / middlebox issue as the "area of experimentation" in the previous email.


Thanks

Michael


From: tcpm [mailto:[e-mail]<mailto:[e-mail]>] On Behalf Of Yuchung Cheng
Sent: Sunday, December 01, 2013 7:08 PM
To: Gorry Fairhurst
Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Re: [tcpm] Summary of mt comments/questions WGLC for draft-ietf-tcpm-fastopen-05



On Sat, Nov 30, 2013 at 9:07 AM, Gorry Fairhurst <[e-mail]<mailto:[e-mail]>> wrote:

Sorry for the delay (I was unwell), so this email is to restart.
I think we got a long way through discussing things, so I've
summarised below all the points as I understand from my side,
so we and others know where we have arrived and can fix others.

I support the experimentation with TFO, although I would argue
we do need to write more about how the proposed changes in this
ID will modify STD behaviour and explicitly say how to "negatively"
cache path info that indicates TFO should not be used.

I would like to see documented the cases where this could
@@potentially@@ be worse than STD mechanisms. The ID already
has some good documentation, but I think it needs a little
more detail especially the set of potential issues that we
expect to be mitigated by a cache at both the client and server
- especially when and why do we need to revoke the cookie
to prevent potentially bad pathologies from being
repeated in a short time between a pair of clients and servers
that keep starting up new connections.

---

The points below hopefully help us see what we can agree on:

a) Case: cached RTT greater than minRTO
   The current text says "RECOMMEND that the client caches the MSS
   and RTT to the server to enhance performance."

   I think ideally we should also explicitly say if the RTT is
   not cached TCP SHOULD disable use of TFO (make the cookie
   invalid) when the RTT>MinRTO.

   ... & note something about  "this prevents a premature RTO
   for flows with a long RTT and also provides an acceptable RTT
   for pacing" - I feel the current text focuses on performance
   improvement for a low RTT, but does not explain why it is
   important for large RTTs.

b) Case: cached RTT less than minRTO and path change
   We could just note that this ID does not change the behaviour
   that a path change can result in an invalid RTT/RTO value,
   and that normal TCP behaviour is expected to recover.
no on a and b.

The agreed change is to remove RTT caching in TFO spec.
TFO will benefit higher RTT client more. Do I need to explain such a simple logic further?

min-RTO is not defined in IETF RFCs, nor do I see your new logic improves TFO b/c ultimately cached RTT can be out-dated. I don't see how tweaking TFO with historical RTT values can make it safer.


c) Case: receiver-advertised MSS less than 536 bytes.

   This is an unusual corner-case and is not one that I think we need
   to engineer for. However, I do think we need to note that an IPv4
   receiver advertised MSS less than 536 would result in
   transmission of an unexpected large segment to the receiver.
   (I.e. say we noted this, but don't change the recommendation).
Why is that specific to TFO?


d) Case: receiver-advertised MSS greater than 1460 B,
   i.e. allowing TFO with large segments.

   I think we should explicitly say MUST NOT use an MSS that
   results in a packet size greater than 1500B.
No. if you send a packet too big you risk it getting lost and wish you get some ICMP warnings. Why is that specific to TFO?


   - AND I think we should caution that we do not have current
   experience of the effect of this behaviour. We could note
   it as an area for further experimentation? I'm not sure what
   we agreed?
ok. Sending SYN-data is indeed a new behavior and an area for further experimentation. I think that's why TFO-draft is experimental. I could add more words about this in the "areas for experimentation".



e) Case: A sender overwhelms a path with limited capacity.
   This is the corner case where a SYN may fail. The issue is most
   severe for links that suffer major overload (think low-speed
   large multiplexing).

   First SORRY - I'd like to avoid "slightly" and "significantly"
   wording  - we discussed on the thread - instead what matters
   to me really is:

   This draft updates the congestion-control behaviour while the
   sender is waiting acknowledgement for a SYN. This will result in
   more data (up to one IW of segments) being sent before the sender
   receives the ACK for the SYN. This could result in additional
   congestion on links that have a high loss rate where the initial
   ACK would have been lost, which in standard behaviour would have
   resulted in a lower initial congestion window.
I can merge the text above into the draft.


   - personally I see the effect as important to document,
   but is OK for experimentation with an IW of 3 (see (g)).

f) The cookie-less experimental method will likely also need a negative
   cache to disable inappropriate TFO usage.

   I'd like to see an explicit note for the future experimentation
   something a bit like this would be good:

  "Even if cookie-less methods are found, it is expected that a
   future method will still need a way to detect conditions where
   TFO should not be used due to path properties and some
   equivalent method may then be needed to disable TFO."
ok.


g) I am unsure that allowing TFO to use IW=10 is safe!

Best wishes,

Gorry