Re: [TLS] TLS 1.3 and TCP interactions

David Benjamin <> Sat, 30 May 2020 15:27 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 658D03A08D9 for <>; Sat, 30 May 2020 08:27:48 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -9.25
X-Spam-Status: No, score=-9.25 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_SPF_WL=-7.5] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id Y2jPNqlqkEmJ for <>; Sat, 30 May 2020 08:27:46 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::1035]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id B4E2C3A08D5 for <>; Sat, 30 May 2020 08:27:46 -0700 (PDT)
Received: by with SMTP id s88so2205632pjb.5 for <>; Sat, 30 May 2020 08:27:46 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=xcNRMtw6A1QOx9R/ytQlkNTFt6pMHFKtcNmAICPox/s=; b=fB08vrZaTBRpPSJ0Fkj6fnmZYuBm+fKudVVXBshMO7BGF9r71ti0QKNAf9E0n1RSlA PfFQUeT95rI8cTDcEVjgYHNXqwXF8o8DylxLiHLA+Gy7Xlbjs2Q/99kKUNPG0iaa8nIx l7ytERw5w8FTSc8oWh53CLkssrf1+GTm78J0c=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xcNRMtw6A1QOx9R/ytQlkNTFt6pMHFKtcNmAICPox/s=; b=ibPdjJWb54Wweq6Pnjd+veCy3Sm/Q/ifXIThiMzXkZeMxsVpVgTpByy3o0vYCHviWI 0Oed4eHdR7UTQqV+lZ03HmMf5gaCY05tvlnk5PHb8gT5plxwRLY6jDoQeZ1u8Ur7V0hc av8DY3M0hISpM7DDl7Jp0XzvcPGCx1rd5gPkEqOQ0ca/A848NG2Sj5UC/SUE4/JuhjIU z2anGEIeJuGExUiTnzYrlUtSCboyXjIxqFSSpvmc/l+YHeyhKT00+DH1JX6sFrqBx0DR J8wWl+KB6T9B2xuvnCJLNiB0Np66oYOVDPRHrAmKnJlyVHJUE/U1AabTBL1yd1l0yknK SVgg==
X-Gm-Message-State: AOAM533AnRDUSDnCTzP7nlhFbyk6O41riYfWVCgRvQ6ubUPbez5WogBe YKne89pRK2+gZY532xSlnjV5IAjHyxtil1oV8UL9
X-Google-Smtp-Source: ABdhPJy0SdoW53+9CNf3WwC7Odxiq0xOhvUE9TwTiwd9wJ3HtqX55BYjPXWNfK9HsgrL1ePVd/ZKZjSDEFtlh/MtsTI=
X-Received: by 2002:a17:902:a515:: with SMTP id s21mr13497401plq.334.1590852464721; Sat, 30 May 2020 08:27:44 -0700 (PDT)
MIME-Version: 1.0
References: <> <> <20200529232821.GO18021@localhost>
In-Reply-To: <20200529232821.GO18021@localhost>
From: David Benjamin <>
Date: Sat, 30 May 2020 11:27:28 -0400
Message-ID: <>
To: Nico Williams <>
Cc: Watson Ladd <>, "<>" <>
Content-Type: multipart/alternative; boundary="0000000000000d1cb805a6df3474"
Archived-At: <>
Subject: Re: [TLS] TLS 1.3 and TCP interactions
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "This is the mailing list for the Transport Layer Security working group of the IETF." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sat, 30 May 2020 15:27:48 -0000

On Fri, May 29, 2020 at 7:28 PM Nico Williams <> wrote:

> On Fri, May 29, 2020 at 06:35:58PM -0400, Watson Ladd wrote:
> > In my experience the issues are thorniest when dealing with blocking
> > sockets. Libraries using nonblocking sockets have to signal to the
> > application that they want IO to happen during the handshake, and can
> > use that same mechanism at later times, particularly for rekeying.
> > Libraries with blocking behavior are unfortunately in a difficult
> > position if they imitate a POSIX API and have no means to drive I/O.
> >
> > One possible dirty trick is to set nonblocking on an owned socket and
> > translate the blocking call into a select or poll based loop that
> > issues both writes and reads until enough is read or written. Note
> > that the real corner case is unanticipated needs to read from the
> > socket: the library has control when it needs to write.

The signaling for nonblocking sockets is somewhat interesting. I don't
think it's as straightforward as that because these non-blocking APIs and
how callers use them becomes sequential anyway. Consider the
NewSessionTicket deadlock issue and a TLS implementation which wishes to
send it immediately after the handshake, rather than deferring to the
first write. Signaling to the application that I/O needs to happen during
handshake, read, and write may look something like...

enum tls_io_result {

tls_io_result tls_handshake(...);
tls_io_result tls_read(...)
tls_io_result tls_write(...)

An HTTP/1.1 server may then:

1. tls_handshake(). On tls_io_want_*, pop back to the select() loop and
retry until tls_io_success.
2. tls_read("GET HTTP/1.1 ..."). On tls_io_want_*, pop back to the select()
loop and retry until tls_io_success.
3. tls_write("HTTP/1.1 200 OK ...")). On tls_io_want_*, pop back to the
select() loop and retry until tls_io_success.

Conversely, an HTTP/1.1 client may tls_handshake(), tls_write(), and
tls_read(), in that order. It likewise would only advance to the next
TLS-level function when the previous one succeeded. You generally do
not read or write before the connection is established, and HTTP/1.1,
unlike HTTP/2, never reads and writes concurrently.

Now the TLS library implements TLS 1.3 and wishes the server to send
NewSessionTicket as soon as possible. It may think to send it right at the
end of tls_handshake(). That gets it out independent of the caller's I/O
patterns. If NewSessionTicket hits EWOULDBLOCK, it then returns
tls_io_want_write out of tls_handshake(). But now we have our deadlock.
This HTTP server will not proceed to tls_read() and thus not consume the
(potentially large) HTTP request until *after* NewSessionTicket is written.
Conversely, this client will not consume the NewSessionTicket via
tls_read() until *after* it has finished writing the HTTP request.

Deferring to the first write makes it all work out, but that is because it
removes the "out-of-turn" write altogether.

> Indeed.
> Another is to start a worker thread to do all (async) I/O on the
> connection and use inter-thread communications primitives on the
> blocking I/O API side.  Because the worker thread needs to do async I/O
> anyways, it might as well service multiple connections to reduce the
> amount of resources needed for the whole thing.

Right, what is happening here, blocking or non-blocking, is that a
non-deferred NewSessionTicket must not be in sequence with the handshake or
TLS read. A deferred one can be in sequence with TLS write since you're
"naturally" blocking on transport write anyway.

If your core TLS protocol implementation is at the same layer as the
component that can drive new sequences, that all works fine. If not, you
need the upper layer to drive that background work, which means this is
part of the TLS protocol implementation's API contract. (Often the layers
above have only a basic understanding of the TLS protocol, since the point
of the TLS library is to abstract that.)

Or, in the case of NewSessionTicket, just defer them to TLS writes because
it's so much simpler. :-)