Re: [TLS] TLS 1.2 - is it allowed to strip the leading zero byte(s) in RSA signature in ServerKeyExchange?

David Benjamin <davidben@chromium.org> Wed, 12 February 2020 06:53 UTC

Return-Path: <davidben@google.com>
X-Original-To: tls@ietfa.amsl.com
Delivered-To: tls@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B2BF812006B for <tls@ietfa.amsl.com>; Tue, 11 Feb 2020 22:53:47 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.249
X-Spam-Level:
X-Spam-Status: No, score=-9.249 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.25, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_SPF_WL=-7.5] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=chromium.org
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UXu6aZihqtNq for <tls@ietfa.amsl.com>; Tue, 11 Feb 2020 22:53:46 -0800 (PST)
Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9E9C9120052 for <tls@ietf.org>; Tue, 11 Feb 2020 22:53:46 -0800 (PST)
Received: by mail-pj1-x1033.google.com with SMTP id d5so456103pjz.5 for <tls@ietf.org>; Tue, 11 Feb 2020 22:53:46 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=AYKB5yo59nWTNao5HreJF309H0IcjQWmgSjPNGf4Y9s=; b=d48+yfDRygKAgP5o/v7RPCIhP8ItaxbQovfWQrtgDmDR+KDXdAeC2sWNZ+zWYbEMs6 eAjexjKqFB9TCtv3pRAJq73L7nBX+XCR6moAIRVb7ERTieO8tuibjd2zNJ4rYUb3FU0m 9kGcNnJZ1Ubc1Yec4jBWVylIyC592A1A9yit0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=AYKB5yo59nWTNao5HreJF309H0IcjQWmgSjPNGf4Y9s=; b=ez4jwwIq5eTS1S0CqdlheeAbtvgLPcKdbQqRlhwMWFtqfxULLaUQdgW4F0kWnu61gl EMS46qnjhAiejCHjLF3vmXutYfgu54mKOqo1L53jnR2QdGlPIwdHAshUANciG1hK+qhi 5NswgB8Vfnyx1h8y5WCmRBbdXi07intCcx8xqSkuLdggf1LxA2d1ai+fkUncF0SPXMoD T/YDhWGAEgiw9QRioAkB4Fm1X6e1671JGhmOUUYqsm8vqH0qKG/ruIschoxSVheFiwGM ULZZCJI/2YFoSSrv/XkupSN19WEveoOEt3JKRfXLlO+/7j34bQIPb6QBIttxUIOPD+ov dfvA==
X-Gm-Message-State: APjAAAXE+KzHdqoIwfR5yckjn0XU1cHgH9FWos/wFbsvK35gBLwi+BOs yoa+NivQFsQRf2BkNv6XSkpzPsYKJKFg+67j1t1Z
X-Google-Smtp-Source: APXvYqx0bl7mqXbYNXdkVRcd93/HhFIvka0PJaqyIT/MpgoMQOGnX3tJqgvqVr+JWAtKgQ+QkH62tWTLyQPPMQahJKk=
X-Received: by 2002:a17:902:694b:: with SMTP id k11mr6926202plt.334.1581490425884; Tue, 11 Feb 2020 22:53:45 -0800 (PST)
MIME-Version: 1.0
References: <CAG5P2e_yELKn_ypt2cVAHtoBeNVrpKLqLwuZprq0bm=h4odrHA@mail.gmail.com>
In-Reply-To: <CAG5P2e_yELKn_ypt2cVAHtoBeNVrpKLqLwuZprq0bm=h4odrHA@mail.gmail.com>
From: David Benjamin <davidben@chromium.org>
Date: Wed, 12 Feb 2020 01:53:29 -0500
Message-ID: <CAF8qwaB0G-PLGSRPxfMoyEEEc8t2D6k_oX9wrZqpPpwS6FPnnA@mail.gmail.com>
To: M K Saravanan <mksarav@gmail.com>
Cc: "<tls@ietf.org>" <tls@ietf.org>
Content-Type: multipart/alternative; boundary="0000000000000d569c059e5b6f49"
Archived-At: <https://mailarchive.ietf.org/arch/msg/tls/BVW9JT1cOQqpq_1zKA9sE4ebT-M>
Subject: Re: [TLS] TLS 1.2 - is it allowed to strip the leading zero byte(s) in RSA signature in ServerKeyExchange?
X-BeenThere: tls@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "This is the mailing list for the Transport Layer Security working group of the IETF." <tls.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tls>, <mailto:tls-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tls/>
List-Post: <mailto:tls@ietf.org>
List-Help: <mailto:tls-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tls>, <mailto:tls-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Feb 2020 06:53:48 -0000

The signature is invalid. The client is correct to reject it, and the
server is incorrect to produce it.

RFC5246 cites PKCS1 (then RFC3447, now RFC8017). Both versions spell out
the signing and verifying operations explicitly. The signing operation must
produce a fixed-width output and the verification operation must reject
incorrectly-sized inputs:
https://tools.ietf.org/html/rfc3447#section-8.2.1
https://tools.ietf.org/html/rfc3447#section-8.2.2
https://tools.ietf.org/html/rfc8017#section-8.2.1
https://tools.ietf.org/html/rfc8017#section-8.2.2


On Wed, Feb 12, 2020 at 1:27 AM M K Saravanan <mksarav@gmail.com>; wrote:

> Hi,
>
> I recently encountered the below issue:
>
> TLS1.2
> ECDHE_RSA
> server certificate: 2048-bit RSA (= 256 bytes)
> ServerKeyExchange hash/sign algorithm: rsa_pkcs1_sha1
>
> The server was sending the ServerKeyExchange with 255 byte as length for
> the RSA signature (i.e. the leading zero was stripped) instead of 256 like
> this:
>
> ====================
> Handshake Protocol: Server Key Exchange
>     Handshake Type: Server Key Exchange (12)
>     Length: 328
>     EC Diffie-Hellman Server Params
>         Curve Type: named_curve (0x03)
>         Named Curve: secp256r1 (0x0017)
>         Pubkey Length: 65
>         Pubkey: 042206562efea8bd47bf014a9e650c42f27078643c553671…
>         Signature Algorithm: rsa_pkcs1_sha1 (0x0201)
>         Signature Length: 255
>         Signature: d1bf915eca2ec0bcdda6f90a398fe5378d2028a22574d213…
> ====================
>
> Is this allowed?  i.e. stripping the leading zero of the RSA signature and
> marking the length as 255?   It is not clear to me from the RFC5246 whether
> it is allowed or not.
>
> (client was failing to verify the signature due to this).
>
> with regards,
> Saravanan
>
> _______________________________________________
> TLS mailing list
> TLS@ietf.org
> https://www.ietf.org/mailman/listinfo/tls
>