Re: [openpgp] [Cfrg] streamable AEAD construct for stored data?

[Daniel Kahn Gillmor <[e-mail]>]

On Fri 2015-10-30 21:30:44 +0900, Watson Ladd wrote:
>> >  b) it doesn't seem to compose as well with asymmetric signatures as one
>> >     might like: a signature over the whole material can't itself be
>> >     verified until one full pass through the data; and a signature over
>> >     just the symmetric key would prove nothing, since anyone getting the
>> >     symmetric key could forge an arbitrary valid, decryptable stream.
>> >     Is there an intermediate approach that would combine an asymmetric
>> >     signature with a chunkable authenticated encryption such that a
>> >     decryptor could stream one pass and be certain of its origin (at
>> >     least up until truncation, if (a) can't be resolved)?
>> >
>> > Thoughts, pointers, or suggestions would be much appreciated.
>
> Use authenticated encryption so no signatures are required. Detached
> signature verification is used for large public messages already: no
> streaming needed.

OK, i am worried that i might have sidetracked the dicsussion with this
side-note here.  We don't have a solution for (b) above today, and as
Watson notes, there are use patterns that suggest people might be OK
with doing two-pass signature verification before processing large
files.

So i'd like to ask folks to put this side-note aside, please, and try to
re-focus on the main original question:

> AGL describes this problem here:
> 
>  https://www.imperialviolet.org/2014/06/27/streamingencryption.html
> 
> and he roughly outlines a generic construction of a composable/chunkable
> approach using AEAD:
> 
>> Ideally such a scheme would take an AEAD and produce something very
>> like an AEAD in that it takes a key, nonce and additional data, but
>> can safely work in a streaming fashion. I don't think it need be very
>> complex: take 64 bits of the nonce from the underlying AEAD as the
>> chunk number, always start with chunk number zero and feed the
>> additional data into chunk zero with a zero byte prefix. Prefix each
>> chunk ciphertext with a 16 bit length and set the MSB to indicate the
>> last chunk and authenticate that indication by setting the additional
>> data to a single, 1 byte. The major worry might be that for many
>> underlying AEADs, taking 64 bits of the nonce for the chunk counter
>> leaves one with very little (or none!) left.
> 
> Two examples of projects that take something like this approach are
> miniLock and Tahoe-LAFS:
> 
>  https://github.com/kaepora/miniLock
>  https://tahoe-lafs.org/trac/tahoe-lafs/browser/docs/specifications/file-encoding.rst
> 
> I'm unaware of any formalization of this approach, though.  Does anyone
> know of one?  If OpenPGP were to adopt AGL's construct, are there
> specific risks to be aware of?

  --dkg