Re: [dane] SMTP STARTTLS stripping in the wild

Viktor Dukhovni <ietf-dane@dukhovni.org> Fri, 14 November 2014 02:27 UTC

Return-Path: <ietf-dane@dukhovni.org>
X-Original-To: dane@ietfa.amsl.com
Delivered-To: dane@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 047671A019B for <dane@ietfa.amsl.com>; Thu, 13 Nov 2014 18:27:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZlRJb20Im2xj for <dane@ietfa.amsl.com>; Thu, 13 Nov 2014 18:27:17 -0800 (PST)
Received: from mournblade.imrryr.org (mournblade.imrryr.org [38.117.134.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C1C061A017C for <dane@ietf.org>; Thu, 13 Nov 2014 18:27:17 -0800 (PST)
Received: by mournblade.imrryr.org (Postfix, from userid 1034) id 741A8284AD6; Fri, 14 Nov 2014 02:27:16 +0000 (UTC)
Date: Fri, 14 Nov 2014 02:27:16 +0000
From: Viktor Dukhovni <ietf-dane@dukhovni.org>
To: dane@ietf.org
Message-ID: <20141114022716.GZ22498@mournblade.imrryr.org>
References: <alpine.LFD.2.10.1411131457140.25815@bofh.nohats.ca> <20141114004313.8557.qmail@ary.lan> <20141114005253.GW22498@mournblade.imrryr.org> <2CCB417DB294C9CC79C32DC1@96B2F16665FF96BAE59E9B90>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2CCB417DB294C9CC79C32DC1@96B2F16665FF96BAE59E9B90>
User-Agent: Mutt/1.5.23 (2014-03-12)
Archived-At: http://mailarchive.ietf.org/arch/msg/dane/CBYhAHcvOwSYaPmfuLG2Qn90apE
Subject: Re: [dane] SMTP STARTTLS stripping in the wild
X-BeenThere: dane@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: dane@ietf.org
List-Id: DNS-based Authentication of Named Entities <dane.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/dane>, <mailto:dane-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/dane/>
List-Post: <mailto:dane@ietf.org>
List-Help: <mailto:dane-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/dane>, <mailto:dane-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Nov 2014 02:27:20 -0000

On Thu, Nov 13, 2014 at 03:57:05PM -1000, Chris Newman wrote:

> > I am far more curious about what happened to Google's outbound
> > TLS mail stats around Oct 14th:
> > 
> >     https://www.google.com/transparencyreport/saferemail/
> > 
> > but this seems likely to remain a mystery.  Perhaps some large peer
> > botched their TLS settings?
> 
> My speculation:
> 
> There's no specification for SMTP relay fallback from STARTTLS to cleartext.

Correct, Sendmail defers the mail, while Postfix falls back to
cleartext (eventually).

> So if a provider turns on fully RFC 3207 client-side STARTTLS for relay

Google is the sender in this case.  The drop in TLS traffic started
on Oct 7th, fell sharply on the 11th and ended on Oct 15th.  Outbound
TLS support at Google is far from new.

> and that
> software doesn't also implement the unspecified fallback, it will stop the flow
> of mail to peers that advertise STARTTLS but implement it incorrectly.

So I think you're saying that Google may have disabled its previous
STARTTLS to cleartext fallback, and after a few days found that a
lot of mail got deferred.  And that point either selectively or
globally re-enabled fallback?

I'd have hoped they would notice a lot sooner, the drop is from
75% to 50% of mail using TLS.  If the missing TLS mail is queued,
that would rapidly lead to enourmous mail backlogs.  It seems far
more likely that some remote peer broke their TLS, or Google tried
some fancy TLS settings, and ran into interoperability issues.  In
either case Google likely then delivered in cleartext.

> Once the queue buildup is noticed, the client-side site has to:
>
> 1. turn off client-side STARTTLS

Note what happened at Google, since the TLS rate did not go to zero.

> 2. upgrade to a software version implementing the unspecified protocol, if
> available.
> 3. manually configure no-STARTTLS for each endpoint that has a broken STARTTLS
> implementation.
> 4. bounce mail to legitimate recipients with a bogus STARTTLS implementation.

They probably had cleartext fallback all along, and I think deferred
mail at that volume would have been noticed rather quickly.

> Incidentally, I suspect 1 is a common service provider response to this
> situation.

That is rather plausible.

-- 
	Viktor.