[TLS] Interop failure due to wrong encoding of ServerDHParams

Florian Weimer <fweimer@redhat.com> Tue, 07 April 2015 12:04 UTC

Return-Path: <fweimer@redhat.com>
X-Original-To: tls@ietfa.amsl.com
Delivered-To: tls@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 964AF1B3527 for <tls@ietfa.amsl.com>; Tue, 7 Apr 2015 05:04:41 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.912
X-Spam-Level:
X-Spam-Status: No, score=-6.912 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8T7dPP11j8M2 for <tls@ietfa.amsl.com>; Tue, 7 Apr 2015 05:04:40 -0700 (PDT)
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 249281B3518 for <tls@ietf.org>; Tue, 7 Apr 2015 05:04:40 -0700 (PDT)
Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (Postfix) with ESMTPS id 9E36638591E for <tls@ietf.org>; Tue, 7 Apr 2015 12:04:39 +0000 (UTC)
Received: from oldenburg.str.redhat.com (oldenburg.str.redhat.com [10.33.200.60]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t37C4cC2018560 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <tls@ietf.org>; Tue, 7 Apr 2015 08:04:39 -0400
Message-ID: <5523C7D5.8090703@redhat.com>
Date: Tue, 07 Apr 2015 14:04:37 +0200
From: Florian Weimer <fweimer@redhat.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0
MIME-Version: 1.0
To: "tls@ietf.org" <tls@ietf.org>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
Archived-At: <http://mailarchive.ietf.org/arch/msg/tls/ZLNB8IFP2R5tWfZu7fpq4HLuG9U>
Subject: [TLS] Interop failure due to wrong encoding of ServerDHParams
X-BeenThere: tls@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "This is the mailing list for the Transport Layer Security working group of the IETF." <tls.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tls>, <mailto:tls-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/tls/>
List-Post: <mailto:tls@ietf.org>
List-Help: <mailto:tls-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tls>, <mailto:tls-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Apr 2015 12:04:41 -0000

I've encountered a TLS 1.2 server implementation which sometimes sends a
ServerDHParams structure as part of the ServerKeyExchange message which
indicates a length of 127 bytes for the dh_Ys parameter, but the
parameter actually sent contains 128 bytes.  As a result, parsing the
ServerKeyExchange message fails, leading to a handshake failure.  (Maybe
this happens if the most significant byte of dh_Ys is zero.)

In this situation, the OpenSSL client logs this error message:

error:1408D172:SSL routines:SSL3_GET_KEY_EXCHANGE:wrong signature
type:s3_clnt.c:1863:

Does this ring a bell?  I don't know any details about the server
implementation, unfortunately.

-- 
Florian Weimer / Red Hat Product Security