[TLS] Regarding the length of encrypted premaster secret.

nilesh <nilesh.tayade@netscout.com> Tue, 11 October 2011 11:17 UTC

Return-Path: <Nilesh.Tayade@netscout.com>
X-Original-To: tls@ietfa.amsl.com
Delivered-To: tls@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 54B5421F8D52 for <tls@ietfa.amsl.com>; Tue, 11 Oct 2011 04:17:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.74
X-Spam-Level:
X-Spam-Status: No, score=-4.74 tagged_above=-999 required=5 tests=[BAYES_20=-0.74, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ETPtbUcdeob1 for <tls@ietfa.amsl.com>; Tue, 11 Oct 2011 04:17:17 -0700 (PDT)
Received: from exprod6ob106.obsmtp.com (exprod6ob106.obsmtp.com [64.18.1.190]) by ietfa.amsl.com (Postfix) with SMTP id A89E921F8CE2 for <tls@ietf.org>; Tue, 11 Oct 2011 04:17:17 -0700 (PDT)
Received: from nsmailfe2k3.netscout.com ([12.187.89.175]) (using TLSv1) by exprod6ob106.postini.com ([64.18.5.12]) with SMTP; Tue, 11 Oct 2011 04:17:16 PDT
Received: from nsmail.netscout.com ([192.168.39.90]) by nsmailfe2k3.netscout.com with Microsoft SMTPSVC(6.0.3790.3959); Tue, 11 Oct 2011 07:17:08 -0400
Received: from [172.16.32.29] ([172.16.32.29]) by nsmail.netscout.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 11 Oct 2011 07:17:08 -0400
Message-ID: <4E9425B3.8070109@netscout.com>
Date: Tue, 11 Oct 2011 16:47:07 +0530
From: nilesh <nilesh.tayade@netscout.com>
Organization: NetScout
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.21) Gecko/20110831 Thunderbird/3.1.13
MIME-Version: 1.0
To: tls@ietf.org
Content-Type: multipart/mixed; boundary="------------050608040300010606010308"
X-OriginalArrivalTime: 11 Oct 2011 11:17:08.0446 (UTC) FILETIME=[508CDFE0:01CC8807]
X-TM-AS-Product-Ver: SMEX-10.0.0.4152-6.800.1017-18442.004
X-TM-AS-Result: No--17.772500-8.000000-31
X-TM-AS-User-Approved-Sender: No
X-TM-AS-User-Blocked-Sender: No
Subject: [TLS] Regarding the length of encrypted premaster secret.
X-BeenThere: tls@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "This is the mailing list for the Transport Layer Security working group of the IETF." <tls.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tls>, <mailto:tls-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/tls>
List-Post: <mailto:tls@ietf.org>
List-Help: <mailto:tls-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tls>, <mailto:tls-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 11 Oct 2011 11:17:18 -0000

Hi,

I have setup an apache server with TLS/SSL support. And I could capture 
some https packets.
Also I have written a dummy code to decrypt using AES algorithm.

However, in the current trace file, the size of encrypted premaster sent 
from client is 130bytes.
I tried first getting a premaster from the encrypted premaster and it 
failed (my program is correct as
I verified it on different trace, also the server private key is correct 
as wireshark could decrypt using this key).
If I remove initial 2bytes of encrypted premaster secret (make it 
128bytes) - then I am able to decrypt it without failure.
But the master_secret and key block generated are not correct then (the 
application data get decrypted to junk -
again, I verified my code to generate master_secret and key block with 
different trace).

<snip>
rsa = RSA_new();
fp = fopen("server.key", "rb"); // error handling is done.
PEM_read_RSAPrivateKey(fp, &rsa, NULL, NULL);
RSA_check_key(rsa);
// Below API fails with '-1' rc - when 130bytes encrypted_premaster is used.
RSA_private_decrypt(pre_master_len, encrypted_premaster, pre_master, 
rsa, RSA_PKCS1_PADDING);
</snip>

Could someone please advise if I missed anything? Should the encrypted 
premaster be always 128bytes?
I have attached the trace file and key file with the email.

-- 
Thanks,
Nilesh