[TLS] BoringSSL's TLS test suite

David Benjamin <davidben@chromium.org> Tue, 16 August 2016 18:08 UTC

Return-Path: <davidben@google.com>
X-Original-To: tls@ietfa.amsl.com
Delivered-To: tls@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1066E12D12F for <tls@ietfa.amsl.com>; Tue, 16 Aug 2016 11:08:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.946
X-Spam-Level:
X-Spam-Status: No, score=-3.946 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=chromium.org
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m8Je0IglvAzv for <tls@ietfa.amsl.com>; Tue, 16 Aug 2016 11:08:49 -0700 (PDT)
Received: from mail-io0-x233.google.com (mail-io0-x233.google.com [IPv6:2607:f8b0:4001:c06::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 057B012D0E0 for <tls@ietf.org>; Tue, 16 Aug 2016 11:08:48 -0700 (PDT)
Received: by mail-io0-x233.google.com with SMTP id b62so114732922iod.3 for <tls@ietf.org>; Tue, 16 Aug 2016 11:08:48 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:from:date:message-id:subject:to; bh=O73I0Q8PkQ9t4JKw9PhxxAeboIKg+Pq6iTUJqYWk5sc=; b=QfzwDsOnV/xO2Ys9q+lkQ/jRXLmffbDvtmy5GKTJKwEOezbB4ZmlIvE8fJ0U3hslR4 UArMUY+HxfnlgTO9VDqdqqKni1XoHAXmKTOOd5ODMDav/q5k2BnguMMimupwt2zCkM3X LMFMtz/OWG9tVYtbHvVLdOTR1QghZ3uzENXyA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=O73I0Q8PkQ9t4JKw9PhxxAeboIKg+Pq6iTUJqYWk5sc=; b=UwgLdwtwZmZF0+8p4gq+SbKyxLYIampWJe53VCjh+O9r/202Jcvb1LAi2jsIgQseXv 7XdBI9OdN1IlH4GweZrt9UiyeOWt+Jmn1Hu2lp8cPVAr4YsO7C3GR2zAj2y7RY+FKbOY o8XY9+PvVCO5jlBc5ebVITFGbse1/6W4DlN2KgqMY+apxrNZEa51vn66f+hzBzRduE5s twxFb/JZW6wfJdANhScJoblMewoztMiqxnCk8kJJckUmLmfGG4UFKYMsrkKcYd9grRfm S03A8UJ9clhWMu2rLinOtuHlTHcX8/UWZcAonZnTkPq2sdIJrSYExSIjHuw1g7tjtDp6 fk7g==
X-Gm-Message-State: AEkooutmg4BN6B1kmEi2UMuS+SNLrT6wchI3PQJEO9m9SgLJDOvNStMBVlftHFyiztrsh1qyLEsD/WKel9+xD3gq
X-Received: by 10.107.52.198 with SMTP id b189mr44412799ioa.6.1471370927814; Tue, 16 Aug 2016 11:08:47 -0700 (PDT)
MIME-Version: 1.0
From: David Benjamin <davidben@chromium.org>
Date: Tue, 16 Aug 2016 18:08:36 +0000
Message-ID: <CAF8qwaBQkVy+wcK1-NFctBepV7TW93YmmPnxS2WoJ6F6=v-aEg@mail.gmail.com>
To: "tls@ietf.org" <tls@ietf.org>, EKR <ekr@rtfm.com>
Content-Type: multipart/alternative; boundary="001a11440f7e7caacd053a343d84"
Archived-At: <https://mailarchive.ietf.org/arch/msg/tls/Czdoo_rGuZjf7JbFnWNxTN1hOyU>
Subject: [TLS] BoringSSL's TLS test suite
X-BeenThere: tls@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "This is the mailing list for the Transport Layer Security working group of the IETF." <tls.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tls>, <mailto:tls-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tls/>
List-Post: <mailto:tls@ietf.org>
List-Help: <mailto:tls-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tls>, <mailto:tls-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Aug 2016 18:08:51 -0000

Hi folks,

BoringSSL has developed a test harness[1] that consists of a fork of Go’s
crypto/tls package (recently dubbed “BoGo" at the Berlin hackathon) plus a
test runner that allows BoGo to be run against the TLS stack under test.
BoGo can be configured to behave in a number of unexpected ways that
violate the TLS standard, thus enabling the testing of many scenarios that
would be otherwise difficult to obtain with a standard stack. We (David
Benjamin and Eric Rescorla) have been getting it to work with NSS and
wanted to let others know in case they might find it useful.

This system was initially designed to work with BoringSSL, but in principle
can be used with any stack. The portability is still a little rough, and
we'll likely make changes as we get more experience here, but it has
already been used to test NSS[2] and OpenSSL[3]. We've written up some
notes at [4].

The test suite should be fairly extensive for DTLS and TLS 1.2 (giving
around 75% line coverage in BoringSSL’s TLS code at last count). It tests
TLS 1.3 as well, though those tests are still in progress. They track
BoringSSL’s in-progress TLS 1.3 implementation.

David and Eric

[1] https://boringssl.googlesource.com/boringssl/+/master/ssl/test/
[2]
https://hg.mozilla.org/projects/nss/file/tip/external_tests/nss_bogo_shim
[3] https://github.com/google/openssl-tests
[4]
https://boringssl.googlesource.com/boringssl/+/master/ssl/test/PORTING.md