[jose] DARE: PKCS#7, Blockchain in JSON/JOSE
Phillip Hallam-Baker <phill@hallambaker.com> Wed, 21 August 2019 20:30 UTC
Return-Path: <hallam@gmail.com>
X-Original-To: jose@ietfa.amsl.com
Delivered-To: jose@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C16551200A4; Wed, 21 Aug 2019 13:30:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 1.099
X-Spam-Level: *
X-Spam-Status: No, score=1.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, TO_NO_BRKTS_PCNT=2.499, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id INInJyQ7jscN; Wed, 21 Aug 2019 13:30:57 -0700 (PDT)
Received: from mail-oi1-f193.google.com (mail-oi1-f193.google.com [209.85.167.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6C135120098; Wed, 21 Aug 2019 13:30:57 -0700 (PDT)
Received: by mail-oi1-f193.google.com with SMTP id t24so2612391oij.13; Wed, 21 Aug 2019 13:30:57 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=vWOcvxvMhRmDkEngm8f9l8v1JURhhJVlWZUUCaSteL4=; b=NdsK4y5X1Lfqrg/Jm9RuUxkPPf8mkWMG/BdCSQuag9PZNG+iWZjqFF41MyTQOungG6 RxeTDAco8aD+EV3pxelD7XeMeKJkVbaMoytBqiPIVQ0y0XYugXmJ8qZNSoVfu42iKWwr FTmeHzkcDVfEmMeeNGkpw/v6DzGdy8Jc6DqEXNCooPjZPMQNB5ZWjYOpqHZilExIXXYT uFEDbPIFfPC6yw/TX1BANtPtPuVkSKoWTsHjQKWhKNlO4XWRIOZf3TU7qJ+Wx9nIjANf siKFy/fnbgaqChE1NUYhfOiQyH6txYfMrcHp8G63cpUWlKbmQx4QERhW+qR3Py2SbvtA GyoA==
X-Gm-Message-State: APjAAAVSEjl+sWqcJUDABeiI2KUF9PUt54N+fL3VdAubHBs52SOUn3Kj 440n1trL2Y4v5CWrlgezL9fPcYrtCe1OKqZzL6Y8VEVh
X-Google-Smtp-Source: APXvYqx6eITPpSWAxj1PI5NHqzpMdcQIY0tdjI8WpZ7qWE/lQ+6lNWEaXeMU4GPx9vK9Ep/tkr6yDytqBCUJyMARIyY=
X-Received: by 2002:aca:2818:: with SMTP id 24mr1442749oix.100.1566419456454; Wed, 21 Aug 2019 13:30:56 -0700 (PDT)
MIME-Version: 1.0
From: Phillip Hallam-Baker <phill@hallambaker.com>
Date: Wed, 21 Aug 2019 16:30:44 -0400
Message-ID: <CAMm+LwjFREuvR13sSacU4-UnsScmWatKXV-jzzKbw-_sUQ4nJA@mail.gmail.com>
To: mathmesh@ietf.org, jose@ietf.org
Content-Type: multipart/alternative; boundary="0000000000004575a80590a67335"
Archived-At: <https://mailarchive.ietf.org/arch/msg/jose/SQU0shdEqaBmXqmVZXsbfbsCByA>
Subject: [jose] DARE: PKCS#7, Blockchain in JSON/JOSE
X-BeenThere: jose@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Javascript Object Signing and Encryption <jose.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/jose>, <mailto:jose-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/jose/>
List-Post: <mailto:jose@ietf.org>
List-Help: <mailto:jose-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/jose>, <mailto:jose-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Aug 2019 20:31:00 -0000
[cc's to the JOSE list since this is extending JOSE] UDF is one of the Mesh technologies that could be used independently to build other systems. DARE is another. The spec is available from the IETF tools or in HTML form as: http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html This format is used within the Mesh protocols to support authentication at the Message and presentation layers. It is also used as the basis for the Mesh persistence stores such as an end-to-end encrypted password catalog. Using append only logs authenticated by means of a Merkle tree as the basis for the persistence model turns out to be very powerful. Devices, Passwords, Tasks are added to or removed from a catalog by adding entries to a log. The same approach is used to implement message spools, the approach bearing some similarity to ideas used in the IBM CICS system (or at least my interpretation of said system as described in a conversation mediated by many pints of Marstons) The same approach could be applied to many other applications. For test purposes only, I developed a ZIP archive type format out of it which seems to be functional. I am also planning to use this format to encrypt the logs kept by Mesh Services. The incremental encryption scheme avoids the need to perform a key agreement for every incremental update which would slow down reading and writing greatly and bloat the log size. A process need only perform a key exchange when it starts writing to the log and refresh it as policy dictates (e.g hourly). Also two different processes can write to the same log simultaneously, encrypting their entries under different key agreements. The starting point for the design was the observation that the Mesh needs a PKCS#7/CMS type cryptographic envelope and a Merkle Tree type authentication mechanism. As I worked on the two independently, I realized that instead of there being two systems, these needed to be designed to work together so that a series of envelopes could be concatenated in an append only log to make a stack or container. The result is DARE which applies the work of JSON Signature and Encryption but with some restrictions. I am a big fan of the Rails part of Ruby on Rails. Sticking to one way to identify things and applying it consistently is very powerful. DARE requires that every key be identified by its UDF fingerprint and only its UDF fingerprint. Any other information (certificates, SAML assertions, the key value etc. is merely provided to assist). So first design decision, why JSON? 1) No matter how much argument is made, a solid majority of IETF-ers despise ASN.1 and will refuse to do any new project based on ASN.1 2) JSON is clearly the favorite data serialization at present and has clear advantages over XML while allowing for a document-friendly text encoding. 3) The main drawback to JSON is the lack of a binary encoding. This is not actually much of a problem for the Mesh as the data items are actually quite small. So the decision made was to make use of JSON and address the encoding efficiency issue by proposing an OPTIONAL extension to JSON encoding that drops in length/ prefixed data sequences for binary and character data: http://mathmesh.com/Documents/draft-hallambaker-jsonbcd.html The reason I prefer extending JSON to moving to CBOR is that extending JSON means that an implementation can use a single decoder for JSON and JSON-B. That greatly simplifies the design of the code and removes the need to negotiate encodings and the number of different code paths to test. Another, secondary issue is that I wanted to be able to encode streams of data without buffering. This meant I had to extend any of the options on offer io The second big concern was how to support separate encryption of a content body and related content data in a secure and flexible fashion. Consider the S/MIME problem of encrypting the subject line on a message with a powerpoint document of 3MB. It is obvious we want to encrypt the subject line for security. It is also obvious that we don't want to have to decrypt the whole document just to read the subject line. And for proper hygiene, I want each piece of data encrypted under a completely different key. The approach I took was to make use of KDF to generate encryption keys, MAC keys and IVs. One concern that was raised is that it is frequently desirable to erase information such as a file from a container. Overwriting a 3MB powerpoint file with zeros means modifying a large chunk of the file and does not provide atomicity. Most O/S will guarantee that a 32KB file update is atomic but no more. To overcome this issue, I introduced the option of adding in a nonce as a mix-in to the KDF. If a process needs to delete an entry, they only need to erase the nonce and the whole encrypted file becomes unrecoverable. This in turn lead to the idea of incremental encryption. If I have a sequence of envelopes, I can use a single key exchange to generate a master key which is then used together with a unique nonce per as input to the KDF used to encrypt each envelope. The envelope format is designed to support single pass, unbuffered encoding with optional encryption and authentication. Each envelope has a header (required), a body and an optional trailer. The container format is almost just a sequence of enveloped but with one important twist. Each envelope begins and ends with a length indicator. And the final length indicator is written in reverse (another break with CBOR). This approach makes it possible to read containers in either the forward or reverse direction with equal efficiency. This is very useful because 90% of the time, the most interesting material in an append only log file is to be found at or near the end. The spec proposes that processes could periodically add indexes to a container allowing O(1) retrieval of specific records by index. This has not (yet) been implemented. While the DARE format was originally designed to allow the use of the meta-cryptography techniques I am using in the Mesh, it does not actually introduce any new crypto. This is because the split key techniques I use affect the implementation of decryption. The encryption approach is unchanged.
- [jose] DARE: PKCS#7, Blockchain in JSON/JOSE Phillip Hallam-Baker