Re: [TLS] BoringSSL's TLS test suite

Henrick Hellström <henrick@streamsec.se> Sun, 25 September 2016 21:07 UTC

Return-Path: <henrick@streamsec.se>
X-Original-To: tls@ietfa.amsl.com
Delivered-To: tls@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4080212B028 for <tls@ietfa.amsl.com>; Sun, 25 Sep 2016 14:07:15 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LD2iY31iu-Ng for <tls@ietfa.amsl.com>; Sun, 25 Sep 2016 14:07:12 -0700 (PDT)
Received: from vsp6.ballou.se (vsp6.ballou.se [91.189.40.85]) by ietfa.amsl.com (Postfix) with SMTP id 4415F12B01F for <tls@ietf.org>; Sun, 25 Sep 2016 14:07:11 -0700 (PDT)
X-Halon-Scanned: 7f4a955dd6f4c149d51110a345668da22aa82d04
Received: from nmail1.ballou.se (unknown [10.0.0.116]) by vsp6.ballou.se (Halon Mail Gateway) with ESMTP for <tls@ietf.org>; Sun, 25 Sep 2016 23:07:08 +0200 (CEST)
Received: from [192.168.0.190] (c-999671d5.06-134-73746f39.cust.bredbandsbolaget.se [213.113.150.153]) (Authenticated sender: henrick@streamsec.se) by nmail1.ballou.se (Postfix) with ESMTPSA id A0AF3C9405 for <tls@ietf.org>; Sun, 25 Sep 2016 23:07:08 +0200 (CEST)
References: <CAF8qwaBQkVy+wcK1-NFctBepV7TW93YmmPnxS2WoJ6F6=v-aEg@mail.gmail.com>
To: tls@ietf.org
From: Henrick Hellström <henrick@streamsec.se>
Message-ID: <c70c6db3-5d1c-d2db-1e37-f8849166786e@streamsec.se>
Date: Sun, 25 Sep 2016 23:06:18 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0
MIME-Version: 1.0
In-Reply-To: <CAF8qwaBQkVy+wcK1-NFctBepV7TW93YmmPnxS2WoJ6F6=v-aEg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Transfer-Encoding: 8bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/tls/Lu9z6jueg2djunf2VXF-kwiIJCU>
Subject: Re: [TLS] BoringSSL's TLS test suite
X-BeenThere: tls@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
Reply-To: henrick@streamsec.se
List-Id: "This is the mailing list for the Transport Layer Security working group of the IETF." <tls.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tls>, <mailto:tls-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tls/>
List-Post: <mailto:tls@ietf.org>
List-Help: <mailto:tls-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tls>, <mailto:tls-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 25 Sep 2016 21:07:15 -0000

Have you noticed that BoringSSL seems to abort handshakes with an 
illegal_parameter alert, if the server certificate uses the standard 
compliant (albeit highly unusual) DER encoding of NULL OPTIONAL as the 
empty string, instead of the non-standard but ubiquitous 0x05 0x00 encoding?

Is this just a regression bug in BoringSSL, or is it an intentional 
restriction of the TLS protocol that should be propagated to other 
implementations as well?

On 2016-08-16 20:08, David Benjamin wrote:
> Hi folks,
>
> BoringSSL has developed a test harness[1] that consists of a fork of
> Go’s crypto/tls package (recently dubbed “BoGo" at the Berlin hackathon)
> plus a test runner that allows BoGo to be run against the TLS stack
> under test. BoGo can be configured to behave in a number of unexpected
> ways that violate the TLS standard, thus enabling the testing of many
> scenarios that would be otherwise difficult to obtain with a standard
> stack. We (David Benjamin and Eric Rescorla) have been getting it to
> work with NSS and wanted to let others know in case they might find it
> useful.
>
> This system was initially designed to work with BoringSSL, but in
> principle can be used with any stack. The portability is still a little
> rough, and we'll likely make changes as we get more experience here, but
> it has already been used to test NSS[2] and OpenSSL[3]. We've written up
> some notes at [4].
>
> The test suite should be fairly extensive for DTLS and TLS 1.2 (giving
> around 75% line coverage in BoringSSL’s TLS code at last count). It
> tests TLS 1.3 as well, though those tests are still in progress. They
> track BoringSSL’s in-progress TLS 1.3 implementation.
>
> David and Eric
>
> [1] https://boringssl.googlesource.com/boringssl/+/master/ssl/test/
> [2] https://hg.mozilla.org/projects/nss/file/tip/external_tests/nss_bogo_shim
> [3] https://github.com/google/openssl-tests
> [4] https://boringssl.googlesource.com/boringssl/+/master/ssl/test/PORTING.md
>
>
> _______________________________________________
> TLS mailing list
> TLS@ietf.org
> https://www.ietf.org/mailman/listinfo/tls
>