[TLS] TLS 1.3 / RSA-PSS and unusual key sizes

Hanno Böck <hanno@hboeck.de> Fri, 27 January 2017 13:30 UTC

Return-Path: <hanno@hboeck.de>
X-Original-To: tls@ietfa.amsl.com
Delivered-To: tls@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A9377129574 for <tls@ietfa.amsl.com>; Fri, 27 Jan 2017 05:30:10 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level:
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wN5WZOv9ukSr for <tls@ietfa.amsl.com>; Fri, 27 Jan 2017 05:30:08 -0800 (PST)
Received: from zucker.schokokeks.org (zucker.schokokeks.org [178.63.68.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B3820129571 for <tls@ietf.org>; Fri, 27 Jan 2017 05:30:08 -0800 (PST)
Received: from pc1 ([2001:2012:127:3e00:b3bf:56a1:a140:6086]) (AUTH: LOGIN hanno-default@schokokeks.org, TLS: TLSv1/SSLv3, 256bits, ECDHE-RSA-AES256-GCM-SHA384) by zucker.schokokeks.org with ESMTPSA; Fri, 27 Jan 2017 14:30:05 +0100 id 00000000000000FC.00000000588B4B5E.00006568
Date: Fri, 27 Jan 2017 14:30:03 +0100
From: Hanno Böck <hanno@hboeck.de>
To: tls@ietf.org
Message-ID: <20170127143003.36c20329@pc1>
X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.31; x86_64-pc-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/tls/GNZrg_LnBMx2mKuOniwDTFdoIIc>
Subject: [TLS] TLS 1.3 / RSA-PSS and unusual key sizes
X-BeenThere: tls@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "This is the mailing list for the Transport Layer Security working group of the IETF." <tls.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tls>, <mailto:tls-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tls/>
List-Post: <mailto:tls@ietf.org>
List-Help: <mailto:tls-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tls>, <mailto:tls-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 27 Jan 2017 13:30:10 -0000

Hi,

I wanted to warn people about a potential source of bugs with the
deployment of RSA-PSS in TLS 1.3.

Usually the RSA key modulus is a multiple of 8 (2048, 4096 etc.).
However there's no rule that RSA keys can't have other sizes.

Implementing PSS with support for arbitrary key sizes is a bit more
complicated than implementing it for multiples of 8. I wrote the PSS
implementation of NSS as a summer of code project a couple of years ago
and I remember that my first implementation completely failed to
consider this. (The fix for that never got merged afair, I informed NSS
developers about this.)

Back then I also reported a bug in OpenSSL:
https://rt.openssl.org/Ticket/Display.html?id=2315&user=guest&pass=guest

Long story short: It's not unlikely that there are more PSS
implementations having problems with this.
So I strongly recommend that all implementors of TLS 1.3 test their
implementations for key sizes from n*8+1 to N*8+7.

Such keys are rare, but they do exist in the wild. If implementations
failing on that get shipped widely we may see random unexplained errors
when people start migrating to TLS 1.3 in masses.

I had actually considered proposing to change TLS 1.3 in a way that
such keys would be simply forbidden. But I did a check on the censys
data and there were too many of them in the wild, so I thought it
wasn't a feasible idea.

-- 
Hanno Böck
https://hboeck.de/

mail/jabber: hanno@hboeck.de
GPG: FE73757FA60E4E21B937579FA5880072BBB51E42