Re: [TLS] Update on TLS 1.3 Middlebox Issues

[Ilari Liusvaara <[e-mail]>]

On Sat, Oct 07, 2017 at 11:33:33AM -0400, Jeffrey Walton wrote:
> On Sat, Oct 7, 2017 at 11:25 AM, Salz, Rich <[e-mail]> wrote:
> >
> >
> >> I suggest we not have this debate now. We'll have a lot more data towards
> >> the end of the month and we can have an informed discussion then.
> >
> 
> > That is what I am asking for.  More information so that the entire WG can
> > make an informed decision.  And I was only laying out an option that does
> > not seem to have been considered before.
> 
> The group (or the IETF) might also consider a policy to answer Ilari
> Liusvaara's question, "What you think is acceptable failure rate?"
> 
> That is a governance issue. It should probably be [nearly] written in
> stone and applied equally to all problems and decisions.
> 

Unfortunately, things are actually more complicated than that.

I suspect that none of the figures "minimal", 1.5% nor 3.4% are
actually accurate, due to "survivorship bias" (survive to be tested).
This is both due to large variance in results, and Google and FB
disagreeing on impact of the record type hack.

I think Attributing the differences to survivorship bias (or other
similar statistical bias) makes much more sense than attributing the
differences to random chance (I presume the sample sizes are large
enough to easily resolve even 0.1% differences) or a testing mistake.

If asked to guess which result is the closest to the true value, I
would guesss Google's (which is also the largest value). But I do
not have any idea even which direction the true value is (often in
studies it is rather easy to guess the direction of the true value,
even if one can not guess the correction magnitude).

The nasty issue in testing is that there are several classes of
connections, with quite widely varying properties:

1) Residential wired
2) Wireless Mobile
3) Enterprise (includes some schools)
4) Satellite (most probably of minimal use).

I would expect that with residential wired, the failure rates are
minimal, whereas with Enterprise, the failure rates would be pretty
substantial. With wireless mobile in the middle. I have no idea how
satellite would stack up.

I suspect the main factor in differences was proportion of enterprise
networks that were tested. Which would imply that the enterprise
failure rate is much higher than 3.4%. And even getting the failure
rate reduced to 1%, the enterprise failure rate would still be
substantially higher than 1%.


And analyzing individual middleboxes to determine the kind of
intolerance is only useful for guiding what kind of modifications to
test on the field, since estimating any useful statistics from
just the devices or software behavior is virtually impossible.


-Ilari