Re: [TLS] Update on TLS 1.3 Middlebox Issues

Ilari Liusvaara <> Sun, 08 October 2017 08:09 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 538D2134DF1 for <>; Sun, 8 Oct 2017 01:09:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 6dfnCmiYC0cT for <>; Sun, 8 Oct 2017 01:09:30 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id AC49C134DA9 for <>; Sun, 8 Oct 2017 01:09:29 -0700 (PDT)
Received: from localhost (localhost []) by (Postfix) with ESMTP id 666B6B518C; Sun, 8 Oct 2017 11:09:26 +0300 (EEST)
X-Virus-Scanned: Debian amavisd-new at
Received: from ([IPv6:::ffff:]) by localhost ( [::ffff:]) (amavisd-new, port 10024) with ESMTP id GIRtoYq1GlPn; Sun, 8 Oct 2017 11:09:25 +0300 (EEST)
Received: from LK-Perkele-VII ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPSA id 82FF32313; Sun, 8 Oct 2017 11:09:22 +0300 (EEST)
Date: Sun, 8 Oct 2017 11:09:22 +0300
From: Ilari Liusvaara <>
To: Jeffrey Walton <>
Cc: "Salz, Rich" <>, "<>" <>
Message-ID: <20171008080922.pqhnj6rw26vo42sy@LK-Perkele-VII>
References: <> <> <> <> <> <> <> <> <>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <>
User-Agent: NeoMutt/20170609 (1.8.3)
Archived-At: <>
Subject: Re: [TLS] Update on TLS 1.3 Middlebox Issues
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "This is the mailing list for the Transport Layer Security working group of the IETF." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 08 Oct 2017 08:09:32 -0000

On Sat, Oct 07, 2017 at 11:33:33AM -0400, Jeffrey Walton wrote:
> On Sat, Oct 7, 2017 at 11:25 AM, Salz, Rich <>; wrote:
> >
> >
> >> I suggest we not have this debate now. We'll have a lot more data towards
> >> the end of the month and we can have an informed discussion then.
> >
> > That is what I am asking for.  More information so that the entire WG can
> > make an informed decision.  And I was only laying out an option that does
> > not seem to have been considered before.
> The group (or the IETF) might also consider a policy to answer Ilari
> Liusvaara's question, "What you think is acceptable failure rate?"
> That is a governance issue. It should probably be [nearly] written in
> stone and applied equally to all problems and decisions.

Unfortunately, things are actually more complicated than that.

I suspect that none of the figures "minimal", 1.5% nor 3.4% are
actually accurate, due to "survivorship bias" (survive to be tested).
This is both due to large variance in results, and Google and FB
disagreeing on impact of the record type hack.

I think Attributing the differences to survivorship bias (or other
similar statistical bias) makes much more sense than attributing the
differences to random chance (I presume the sample sizes are large
enough to easily resolve even 0.1% differences) or a testing mistake.

If asked to guess which result is the closest to the true value, I
would guesss Google's (which is also the largest value). But I do
not have any idea even which direction the true value is (often in
studies it is rather easy to guess the direction of the true value,
even if one can not guess the correction magnitude).

The nasty issue in testing is that there are several classes of
connections, with quite widely varying properties:

1) Residential wired
2) Wireless Mobile
3) Enterprise (includes some schools)
4) Satellite (most probably of minimal use).

I would expect that with residential wired, the failure rates are
minimal, whereas with Enterprise, the failure rates would be pretty
substantial. With wireless mobile in the middle. I have no idea how
satellite would stack up.

I suspect the main factor in differences was proportion of enterprise
networks that were tested. Which would imply that the enterprise
failure rate is much higher than 3.4%. And even getting the failure
rate reduced to 1%, the enterprise failure rate would still be
substantially higher than 1%.

And analyzing individual middleboxes to determine the kind of
intolerance is only useful for guiding what kind of modifications to
test on the field, since estimating any useful statistics from
just the devices or software behavior is virtually impossible.