Re: [TLS] TLS 1.3 and TCP interactions

[David Benjamin <[e-mail]>]

On Fri, May 29, 2020 at 7:28 PM Nico Williams <[e-mail]> wrote:

> On Fri, May 29, 2020 at 06:35:58PM -0400, Watson Ladd wrote:
> > In my experience the issues are thorniest when dealing with blocking
> > sockets. Libraries using nonblocking sockets have to signal to the
> > application that they want IO to happen during the handshake, and can
> > use that same mechanism at later times, particularly for rekeying.
> > Libraries with blocking behavior are unfortunately in a difficult
> > position if they imitate a POSIX API and have no means to drive I/O.
> >
> > One possible dirty trick is to set nonblocking on an owned socket and
> > translate the blocking call into a select or poll based loop that
> > issues both writes and reads until enough is read or written. Note
> > that the real corner case is unanticipated needs to read from the
> > socket: the library has control when it needs to write.
>

The signaling for nonblocking sockets is somewhat interesting. I don't
think it's as straightforward as that because these non-blocking APIs and
how callers use them becomes sequential anyway. Consider the
NewSessionTicket deadlock issue and a TLS implementation which wishes to
send it immediately after the handshake, rather than deferring to the
first write. Signaling to the application that I/O needs to happen during
handshake, read, and write may look something like...

enum tls_io_result {
    tls_io_success,
    tls_io_error,
    tls_io_want_read,
    tls_io_want_write,
};

tls_io_result tls_handshake(...);
tls_io_result tls_read(...)
tls_io_result tls_write(...)

An HTTP/1.1 server may then:

1. tls_handshake(). On tls_io_want_*, pop back to the select() loop and
retry until tls_io_success.
2. tls_read("GET HTTP/1.1 ..."). On tls_io_want_*, pop back to the select()
loop and retry until tls_io_success.
3. tls_write("HTTP/1.1 200 OK ...")). On tls_io_want_*, pop back to the
select() loop and retry until tls_io_success.

Conversely, an HTTP/1.1 client may tls_handshake(), tls_write(), and
tls_read(), in that order. It likewise would only advance to the next
TLS-level function when the previous one succeeded. You generally do
not read or write before the connection is established, and HTTP/1.1,
unlike HTTP/2, never reads and writes concurrently.

Now the TLS library implements TLS 1.3 and wishes the server to send
NewSessionTicket as soon as possible. It may think to send it right at the
end of tls_handshake(). That gets it out independent of the caller's I/O
patterns. If NewSessionTicket hits EWOULDBLOCK, it then returns
tls_io_want_write out of tls_handshake(). But now we have our deadlock.
This HTTP server will not proceed to tls_read() and thus not consume the
(potentially large) HTTP request until *after* NewSessionTicket is written.
Conversely, this client will not consume the NewSessionTicket via
tls_read() until *after* it has finished writing the HTTP request.

Deferring to the first write makes it all work out, but that is because it
removes the "out-of-turn" write altogether.


> Indeed.
>
> Another is to start a worker thread to do all (async) I/O on the
> connection and use inter-thread communications primitives on the
> blocking I/O API side.  Because the worker thread needs to do async I/O
> anyways, it might as well service multiple connections to reduce the
> amount of resources needed for the whole thing.
>

Right, what is happening here, blocking or non-blocking, is that a
non-deferred NewSessionTicket must not be in sequence with the handshake or
TLS read. A deferred one can be in sequence with TLS write since you're
"naturally" blocking on transport write anyway.

If your core TLS protocol implementation is at the same layer as the
component that can drive new sequences, that all works fine. If not, you
need the upper layer to drive that background work, which means this is
part of the TLS protocol implementation's API contract. (Often the layers
above have only a basic understanding of the TLS protocol, since the point
of the TLS library is to abstract that.)

Or, in the case of NewSessionTicket, just defer them to TLS writes because
it's so much simpler. :-)

David