Re: [tcpm] More TCP option space on SYNs

[Joe Touch <[e-mail]>]

Hi, Martin,

On 7/8/2014 3:39 PM, Martin Duke wrote:
> Gentlemen,
>
> I think we're throwing out the idea of extended space on SYN a little
> too quickly.

Bob, Ted Faber, and I are working on a draft to do that which uses 
multiple segments, as has been noted on the list.

 > From the EDO draft:
>
> A new TCP option cannot extend the Data Offset of a TCP initial SYN.
>     All TCP segments, including the initial SYN, may include user data
>     in the payload data [RFC793  <http://tools.ietf.org/html/rfc793>], and this can be useful for some
>
>     proposed features such as TCP Fast Open [Ch14  <http://tools.ietf.org/html/draft-touch-tcpm-tcp-edo-02#ref-Ch14>]. Legacy endpoints
>     that ignore the new option would process the payload contents as
>     user data and send an ACK. Once ACK'd, this data cannot be removed
>     from the user stream.
>
> The TFO option issue seems addressable by specifying that if a SYN
> contains both EDO and TFO Cookie, the TFO Cookie MUST be entirely in the
> extended option space.* This eliminates the possibility that a server
> with TFO but not EDO will misinterpret extended options as data.

You have not addressed servers that support neither TFO nor EDO. They 
will silently ignore EDO and interpret the TFO cookie as user data - 
which cannot be undone. The only solution is to abort the connection, 
and that's thus not considered a viable backward-compatible approach.

Approaches that are not backward compatible are trivial, but the cost of 
aborting undesired connections to legacy servers is considered unacceptable.

> TFO aside, I hesitate to throw out EDO with SYN for the benefit of
> listening TCP servers that accept unvalidated data appended to the SYN.
> Even if technically permitted, this is obviously unsafe behavior in the
> open internet -- or else no one would have invented TFO.

You're confusing TFO with the validity of data in the initial SYN. TFO 
lets that data go to the application immediately, without needing to 
wait for the completion of the three-way handshake. However, data in the 
initial SYN has always been valid - it just waits for the handshake to 
complete to be passed to the user.

Although many existing servers simply discard data in the initial SYN 
(to reduce the potential for a DOS attack), that's not a requirement, 
and there are lots of devices that aren't "servers" to the rest of the 
world but act like servers in the context of a TCP connection. Data in 
the initial SYN cannot be 'undone' - even if it's resent with different 
content (i.e., using the same sequence number), the receiver has no 
obligation to overwrite it.

 > Before
> discarding the architecturally simple solution I would prefer to see (1)
> evidence that real servers actually accept and deliver unvalidated data
> attached to a SYN packet,

The process works the other way around; the only way to deprecate an 
existing feature is to have evidence it's very widely NOT used and thus 
that deprecation would not cause harm. That's very difficult to do - 
it's not a matter of what the few main OS'es do, it depends on a lot of 
legacy equipment.

> and (2) an argument that this is behavior that
> the standard should support.

See above; whether it should or not, you don't know that it doesn't.

Joe

> -Martin
>
>
> * In principle, the TFO Cookie need only partially be in the
> extendedspace, but this would offer less protection to servers that aren't
> properly validating their TFO cookies.
>
>
>
> On Sat, May 31, 2014 at 3:34 PM, Bob Briscoe <[e-mail]
> <mailto:[e-mail]>> wrote:
>
>     Joe,
>
>
>     At 20:20 31/05/2014, Joe Touch wrote:
>
>         Hi, Bob,
>
>         On 5/31/2014 11:19 AM, Bob Briscoe wrote:
>
>             Joe,
>
>             Thx for consolidating this thread. I've given it a new
>             subject line.
>
>             1) You've silently made an important alteration to the proposed
>             protocol. You've put the extra-options directly in the TCP
>             option space
>             of the C-SYN, not within the payload.
>
>
>         I hadn't assumed that; I was describing only which SYN carried
>         which options, not where they were. As you do, I was assuming
>         the C-SYN options were in the data space.
>
>
>     OK.
>
>
>
>         To be clear, in my variant - using data outside the sync'd
>         connection (with ACK clear), the option space is similarly in
>         the data portion of the 'extra' segment.
>
>
>     Yup, that was understood already.
>
>
>
>             Yes, your altered proposal is cleaner. However, don't
>             imagine I didn't
>             think of this. I did and I deliberately didn't do it this
>             way.We have a
>             choice:
>                       clean and vulnerable vs. messy but robust.
>
>             I'm not wedded to using port 80 and http headers, but this
>             is perhaps
>             the most pragmatic approach.
>
>
>         That's fraught with a whole slew of other problems, notably the
>         middleboxes that currently try to translate, proxy, or otherwise
>         modify anything they see as port 80.
>
>         Further, that would limit the number of pending connections.
>
>             It will be really unorthodox to define such
>             a protocol I know. We would have to say something like
>
>                       "The dst port of the C-SYN MUST be 80, and the
>             payload MUST start
>                       with the constant magic_token, where
>                       magic_token = 'PUT / HTTP/1.1<CRLF>Connection :
>             DSO<CRLF><CRLF>'
>                       "
>
>             I'm sorry if even thinking about this makes you feel dirty :|
>
>             Other suggestions for inner protocols are welcome, including
>             tunnelled
>             protocols, as long as middleboxes widely forward them, given
>             their dst
>             port.
>
>
>         I don't see what this gets us other than potential interactions.
>         There's no reason to use HTTP to format TCP options, and it
>         would necessitate new encodings and new processing for existing
>         options that might end up in the extended space.
>
>
>     To be clear, I would put the general requirement as :
>
>     "       Pick a port number, then make sure the app-layer headers at
>     the start of the TCP payload appear valid for that port number, so
>     that middleboxes will forward it, then include the extra TCP options
>     as the body of this faked up app-layer protocol.
>     "
>
>     Then the dilemma becomes: pick a number, any number as long as it's
>     80. The alternative is often to be blocked. Particularly if you
>     connect from inside a company that has bought one of those myopic
>     Web gateways, or you connect to a cellular network, or... you just
>     connect to the Internet via a box from a vendor beginning with a
>     letter in the alphabet,... or a digit.
>
>
>
>             2) The main problem with your notation is it doesn't say
>             /where/ the
>             info is placed.
>
>
>         EDO says the options are at the head of the data segment, just
>         with a longer pointer - the same way as current options.
>
>         For extending the SYN using a separate segment - whether a C-SYN
>         or unsync'd data segment - the data similarly comes at the head
>         of the data, just that there's no user data permitted after it.
>
>
>     I understood all this. I just meant that the notation you introduced
>     in your email didn't have the facility to specify at which layer
>     fields were placed.
>
>
>
>             I've added notation as follows:
>             TCP(base header [TCP options [APP(header[payload])]])
>
>             And for the record I've made the if-else logic clearer.
>
>             Where I've made more than clarifying edits inline, I've
>             described them
>             and tagged them with [BB].
>
>             At 21:34 30/05/2014, Joe Touch wrote:
>
>                 Hi, Bob,
>
>                 Let's get back to the core, in a simpler fashion, so
>                 other can follow it.
>
>                 I stand by my "there's no way to extend the space in the
>                 initial SYN",
>                 but you've convinced me there *might* be a way to
>                 provide extended
>                 space that can occur during the first phase of the TWHS.
>                 I think the
>                 dual-SYN approach still isn't viable, but I've outlined
>                 an alternative
>                 below that's similar but doesn't have the same baggage, IMO.
>
>                 Again, I'm still concerned by what midboxes might do to
>                 this...
>
>                 What do others think??
>
>                 Joe
>
>                 For quick review, here's what I understand:
>
>                                  dso = dual-syn option
>                                          dso-D = data
>                                          dso-C = control
>                                  conn_id = identifier to link the two
>                 SYNs together
>                                  extra_opt = options that didn't fit in
>                 legacy SYN
>                                  fit_opt = options that do fit in the
>                 legacy SYN
>
>                       new client endpoint sends
>                               TCP(port A SYN [dso-D(conn_id) + fit_opt] )
>                               TCP(port B SYN [dso-C [APP(headers [conn_id +
>             extra_opt] ) ] ] )
>
>
>         IMO, the conn_ID in the C-SYN should similarly be in the regular
>         option space, but I don't think it matters.
>
>             [BB]: i/APP(headers...)/
>
>                                       if (legacy server endpoint) {
>             sends back two
>             connections:
>                                               TCP(port A SYN-ACK [fit_opt] )
>                                               TCP(port B SYN-ACK [??] )
>
>                                                  (it's interpretation of
>                 extra_opt)
>
>
>         I'm assuming EDO after the SYN, which means the SYN-ACK has as
>         much space as it wants for options, as would every other segment.
>
>                                                       new client
>             endpoint responds:
>                                                       TCP(port A ACK)
>             (established)
>                                                       TCP(port B RST)
>
>                                          Notes about legacy servers:
>                                                  - they do twice the
>                 work on SYNs
>                                                  - they might keep twice
>                 the state
>                                                  (if not using cookies)
>                                                  - they might clean
>                 state if the RST
>                                                  is received, but that
>                 state might
>                                                  persist indefinitely
>                 (until the next
>                                                  connection, depending
>                 on timeouts, etc.)
>
>                                          -----
>
>                                       } elif (new server endpoint) {
>             sends back one
>             connection:
>                                               TCP(port A SYN-ACK [edo +
>             fit_opt +
>             extra_opt] )
>
>             [BB]: s/dso-d/edo/
>
>
>         Sure. At this point we're basically in EDO-land.
>
>                                                       new client
>             endpoint responds:
>                                                       TCP(port A ACK)
>             (established)
>
>
>                                          Notes:
>                                                  - can stall when dso-D
>                 SYN arrives
>                                                  before dso-C SYN, up to
>                 some limit
>                                                  - twice the work on
>                 SYNs (or more)
>
>                                       }
>
>                 Here's what I was assuming, though admittedly it's not
>                 documented (yet):
>
>                          - no significant impact on TCP connection rate for
>                          legacy servers
>
>                          - no significant impact on TCP connection rate for
>                          legacy clients
>
>                          - impact dominated by processing the extended
>                 option space
>                          for extended clients
>
>                          - impact dominated by processing the extended
>                 option space
>                          for extended servers
>
>                          - compatible with typical TCP processing
>                 optimizations,
>                          notably SYN cookies
>                                  you did provide a potential way forward
>                 for these
>
>                          - capable of successfully traversing typical NATs
>
>                 Your approach has the following properties:
>
>
>             The 3 bullets below are not useful ways to describe
>             performance impact.
>             They selectively describe whichever gives the most
>             pessimistic picture
>             out of:
>             a) either the instantaneous performance change at the moment
>             of connection
>             b) or the worst-case long-run performance impact
>
>             They don't describe the average long-run performance impact,
>             which is
>             important for sizing machines.
>
>             Worse, the instantaneous performance impact is only
>             significant when a
>             machine's SYN processing time is large relative to the e2e
>             delay, which
>             would be a highly unusual scenario on public networks (even
>             in scenarios
>             such as intra-data-centre, it's hard to reduce e2e delay to
>             approach SYN
>             processing time, but you could for intra-machine connections).
>
>
>         SYN processing is a known load - and potential overload - on
>         machines. That's why it's reasonable to focus on it.
>
>
>     But we should primarily think in terms of SYN cookie processing for
>     legacy hosts, and the equivalent for upgraded hosts. We should
>     design thia as the norm, not the exception. Possibly the /only/ way
>     to implement this new form of SYN processing.
>
>     Then, for the dual-SYN scheme, the upgraded host doesn't have a
>     normal SYN cookie processing load with a C-SYN, so we can no longer
>     rely on known-load formulae.
>
>
>
>                          - halves the server connection rate for updated
>                 servers
>                          from legacy clients when this option is in use
>
>
>             Eh? The long-run server connection rate will be fractionally
>             decreased
>             due to updated clients using extra options (which is your
>             third case
>             below), but the instantaneous server connection rate seen by
>             a legacy
>             client is unchanged, because it only sends one SYN.
>
>
>         Agreed. I was doing too many of these at once...
>
>                          - lowers (to some extent, if not halves) the client
>                          connection rate of updated clients to all servers
>                          when this option is in use
>
>                          - halves (roughly) the server rate for all servers
>                          when this option is in use
>
>
>             Nope. All long-run server rates are reduced by 1/(1+e),
>             where e is the
>             fraction of connections using extra options.
>
>
>         the rate for connections when this option is in use is halved -
>         I didn't see that the sentence above could be parsed the other
>         way. the overall rate depends on the fraction of connections
>         using the option.
>
>                 It also:
>
>                          - doubles the number of SYNs in the network
>
>
>             Nope. The number of SYNs in the network is inflated by e
>             where e is the
>             fraction of connections using extra options.
>
>
>         You keep bringing this up - I don't like a solution that should
>         be used only a little. If it works, we should assume it works
>         everywhere all the time.
>
>
>     It doesn't make sense to predict that all connections will always
>     use MSS, timestamp, SACK, window-scale, MPTCP, TCP-AO, new option X,
>     new option y, etc etc.
>
>     For a start, less than 1% of connections contain enough packets to
>     make MPTCP worth starting.
>
>     The word option means optional.
>
>
>                          - susceptible to lack of fate-sharing problems,
>                 e.g.,
>                          if the two SYNs experience different firewall
>                 configurations
>
>
>             Nope. It's fairer to say it's potentially susceptible to
>             second-order
>             fate-sharing problems like your firewall example (the
>             first-order fate
>             sharing problems have been addressed).
>
>
>         I have no idea why you think firewalls are a second-order
>         fate-sharing, or what that even means. Fate sharing means fate
>         is shared. When fate is not shared by the SYNs, this method has
>         problems. There's no mechanism to recover the missing D-SYN or
>         C-SYN.
>
>
>     What I mean by first-order fate-sharing problems is those obvious
>     cases (reordering, drop) that we have addressed by resynthesising
>     fate sharing.
>
>     There is certainly a possibility that two SYNs between the same
>     endpoints will pass through different firewalls, but it is small in
>     the Internet as a whole. There is a further possibility that these
>     two firewalls will be configured differently, but it is small. There
>     is a further possibility that the differences in configuration will
>     be significant enough to distinguish the behaviour wrt the two SYNs,
>     but it is small. When three small probabilities are multiplied
>     together, you get a really small probability. This is what I call a
>     second-order fate-sharing problem.
>
>     You must surely see that it is inconsistent to say:
>     * "this method has problems" about the DSO idea, based on such a
>     corner case
>     * "no problem with fate-sharing" about the ASO idea, when all you
>     mean is that the FBP will follow the same path as the SYN, and they
>     will therefore likely pass through the same devices.
>
>     In the ASO proposal, splitting the options on a SYN over two
>     packets, one of which is considered invalid by TCP, will cause the
>     FBP to be discarded by numerous TCP normalisers and stateful
>     firewalls. That is a fate sharing problem of the first order. Likely
>     to be on a much larger scale than the 'second-order' firewall
>     problem above.
>
>
>
>                          - reduces the space available for fit_opt due
>                 to the need
>                          for the conn_id even in the fall-back D-SYN,
>                 which means
>                          less option space in the SYNs for fall-back
>                 connections
>
>
>             Yup.
>
>
>                          the conn_id which may need to be very large
>                 because it
>                          needs to be unique per source port and source
>                 IP address
>                          because that information is lost during NAT
>                 translation
>
>
>             Given many NATs will typically make the src IPs of both SYNs
>             the same, I
>             suggest a larger conn_id should be a fall-back option for
>             the client,
>             not a default.
>
>
>         Multiple NATs are quite common - carrier grade NATs in the net,
>         and user NATs at the home WIFI point.
>
>         Any multiple-NAT, or any path through even a single
>         carrier-grade NAT will end up potentially un-doing or over-doing
>         source IP overlap, creating false positives and false negatives
>         that break this mechanism.
>
>         That's really the crux - fate sharing and the need to bind the
>         two SYNs - that make this untenable, IMO.
>
>             Even if the src IPs of both SYNs are different once they
>             reach the
>             server, the high end bits will invariably be the same.
>
>
>         You can't know this - the two SYNs could have gone through
>         different numbers of NATs or different NATs entirely, so the
>         bits could change arbitrarily.
>
>
>     That's what 'invariably' means. I'm not claiming this works 100%. I
>     am saying I expect it to work sufficiently to be worth trying it out
>     to see if I'm right. If it doesn't work, on some paths, the
>     fall-back has been described - the client has to rely on the TCP
>     options it can fit in a single SYN.
>
>
>
>             So the max size
>             of the contents of the DSO TCP option can be 6B, and the
>             server can take
>             the rest of conn_id from the higher bits of the src IP addr
>             of each SYN.
>             This is a variant of the idea in
>             <draft-wing-nat-reveal-option>__.
>
>
>         FWIW, you're now in the land where we would have to start
>         explaining the privacy and tracking implications of those bits.
>         I'd really like to avoid that.
>
>
>     Nope. I think you're reading this the wrong way round. I'm not
>     revealing private IDs in TCP option fields. I'm reducing the size of
>     the (random) conn_id in cases where the /public/ src port or the
>     upper bits of the /public/ src IP have sufficient entropy to be used
>     to construct the rest of the conn_id.
>
>     I was acknowledging Dan Wing 'cos he gave us the idea. But it's a
>     /variant/ that doesn't reveal anything private.
>
>     I'm going offline, possibly until Tue now.
>
>     Regards
>
>
>     Bob
>
>
>
>             In fact, the server doesn't even need a small conn_id for
>             clients that
>             know they are not behind a NAT and that want more option
>             space in the
>             D-SYN - then the server could use the src port & src IP for
>             the conn_id.
>
>             To summarise, these options could be distinguished by the
>             length field
>             of the dual-syn option.
>             Length = 2B                             => conn_id = src
>             netaddr + src port
>             Length = 6B = 2B +4B conn_id_short      => conn_id = src
>             netaddr +
>             conn_id_short
>             Length = 8B = 2B +6B conn_id_long       => conn_id = hsb(src
>             netaddr) +
>             conn_id_long
>
>             Given there have been numerous other attempts to reveal a
>             connection ID
>             that is preserved through middleboxes [RFC6967], rather than
>             defining a
>             dual-syn option that carries a conn_id, we might want to
>             design a TCP
>             connection ID option with a flag to say whether it is also
>             part of a
>             dual SYN pair or not.
>
>             Where
>             * hsb(src netaddr) is the netaddr with the lowest 16 bits
>             truncated
>             * src netaddr is the network address (IPv4, IPv6, or any
>             other network
>             protocol)
>
>             To reduce latency, a host could use the default
>             short_conn_id for all
>             connections at first, then:
>             - if it finds that DSO persistently doesn't work it falls
>             back to the
>             long_conn_id for all connections
>             - it occasionally tests the short and zero options to see if
>             it can use
>             shorter DSO options.
>
>
>                          - requires the ISNs to be related (see RFC6528
>                 - if there's
>                          a rule to generate it, there will be code to
>                 validate that
>                          rule, and eventually a BCP to encourage that
>                 validation -
>                          typically from the same RFC author)
>
>
>             Eh? The ISNs can and should be independent. To be robust against
>             middleboxes that rewrite sequence numbers, we must not
>             required ISNs to
>             be related.
>
>
>                 I agree that you have proposed potentially viable ways
>                 to deal with
>                 the SYN cookie, and that RST state is not an issue.
>
>
>             A feature that I think it's fair to add:
>                       - Good chance of passing through app-layer
>             middleboxes that
>             forward
>                       unrecognised TCP options unchanged, but not those
>             that discard
>             them.
>
>
>                 However, there are too many problems with this, IMO, to
>                 call it viable.
>
>
>             Once your over-pessimistic analyses of the performance
>             impact are
>             corrected, and my ideas to reduce the size of the conn_id
>             are taken into
>             account, it's a different story.
>
>             But it's up to the WG to decide whether this is worth taking
>             further.
>             Not just you or I.
>
>
>         Agreed.
>
>                 Here's another trick that might clean up the above a little:
>
>
>             <snip - I'll respond separately to your later updates on
>             this ASO idea,
>             with ACK=0>
>
>             Cheers
>
>
>             Bob
>
>
>             ____________________________________________________________________
>             Bob Briscoe,                                                  BT
>
>
>         ____________________________________________________________________
>         Bob Briscoe,                                                  BT
>
>
>     _________________________________________________
>     tcpm mailing list
>     [e-mail] <mailto:[e-mail]>
>     https://www.ietf.org/mailman/__listinfo/tcpm
>     <https://www.ietf.org/mailman/listinfo/tcpm>
>
>