Re: [openpgp] AEAD Chunk Size

["Neal H. Walfield" <[e-mail]>]

At Thu, 28 Mar 2019 14:27:27 -0700,
Jon Callas wrote:
> > For me, using an unbounded amount of memory is not an option for a
> > component processing OpenPGP data if we want to build robust systems
> > on top.
> 
> Okay, prior to this working group, when there was running code
> without a consensus rough or not, this problem existed. Even with
> compression, PGP 2 ran on DOS machines with a max of 640K of
> RAM. There are many similarly constrained systems that run OpenPGP
> implementations.

Until now, OpenPGP didn't require buffering data.  A decrypted AEAD
chunk MUST only be released when it has been authenticated.  In the
current proposal, AEAD chunks are potentially unbounded (well, up to 4
exabytes...) in size.  No one can decrypt such chunks without
cheating, i.e., releasing unauthenticated plaintext.

> > Therefore, we need to process OpenPGP data in bounded space.  Since
> > there can be no relation between encrypted and decrypted message size
> > due to compression, the only option I see is to provide a streaming
> > API, which let's us process data in constant space.
> 
> I’m not quite sure what you mean when you say “bounded space”
> because one interpretation of that is obviously false. OpenPGP has
> always supported being able to process messages where the encryptor
> does not know the size ahead of time. That’s why we have
> indeterminate lengths and chunking.

[snip]
>  For example, RFC 4880 allows a partial body length (a chunk) to be
>  2^30 octets, and that could be irritating to handle.

indeterminate lengths and partial body chunking don't prohibit the
implementation from releasing the text early; there is no requirement
to buffer the whole chunk.

> One of (perhaps unstated) goals of OpenPGP is that it allow for
> highly constrained implementations. This was a huge consideration in
> both 2440 and 4880.

We are arguing within this tradition.  We want highly constrained
systems to be able to process (nearly) all OpenPGP messages in a
*conforming* manner.  AEAD mandates buffering whole chunks, because
unauthenticated plaintext MUST NOT be released.  By limiting chunks to
some kilobytes in size, highly constrained systems will be able to
process these messages without releasing unauthenticated plaintext.

> Efail is primarily a problem with MIME encoding and layering violations. It works just as much with S/MIME as OpenPGP. Perhaps I’m missing something, but I don’t see how Efail is relevant to resource bounds.

EFAIL is also about ciphertext modification.  If an implementation
never releases unauthenticated plaintext, then ciphertext modification
is not possible.  Highly constrained systems can do this if the chunk
size is limited to something they can handle.

> > Therefore, we need to use chunking and authenticate message prefixes.
> > We need to use chunks that are reasonably small, and this size should
> > preferably not be configurable.  We should consider performance
> > constraints and pick one suitable size.  Configurable chunk sizes
> > bring complexity and increase the attack surface, as was pointed out
> > in this thread.
> > 
> 
> I’m with you on a lot of this, but I don’t know what you mean by “configurable”? Do you mean that there should be one chunk size only? If so, what size do you propose? 32 Meg or thereabouts (2^25 is in that ball park)? If so, would that mean that all messages smaller than your chunk size would be a single chunk? 

Configurable means that the chunk size is not fixed by the standard.
There are two proposals right now: we either fix the chunk size to
16kb (support by Justus, Sebastian and I) or to 256kb (preferred by
Bart from Proton Mail, since that is what they are using in practice).

> > I appreciate the desire to protect against truncation.  But,
> > truncation is pretty common when we transmit data, so I'd argue that
> > application developers are more likely to expect and gracefully deal
> > with truncated data than with ciphertext being manipulated or the PGP
> > implementation consuming unbounded amounts of memory.
> 
> Does this mean that you think that message truncation is an error that OpenPGP doesn’t need to guard against?
> 
> That’s the way I interpret the first line in the paragraph above (“I appreciate … But,…”). If so, that’s counter to the long-standing consensus of the working group. It’s the whole reason we have MDCs and the reason why they were aggressively pushed in the implementations and non-MDC packets browbeaten into doing MDCs. See the non-normative discussion in section 5.13 of 4880.

Either you stream or you protect against truncation.  I don't see how
you can do both.  That doesn't mean you can't detect truncation and
signal that it occurred.  Indeed, that is a property that MDC and
signatures provide.  But, then it is up to the application to recover.
(Perhaps the application, with or without the help of the OpenPGP
implementation, simply buffers the whole message or a sufficiently
large prefix.  But, that is built on top.  We want OpenPGP to remain
streamable.)

> > I have implemented AEAD in Sequoia, and I have evaluated the
> > implementations in GnuPG and RNP.  Every implementation is either
> > unsafe, not robust, or does not implement the proposal.
> 
> Tell us more. What problems did you find?

Sequoia is written in Rust, and in Rust out of memory errors are
fatal.  Handling a 4 exabyte chunk would result in an out of memory
error, and termination of the process.

Both GnuPG and RNP process unauthenticated plaintext.  They release
the plaintext for further processing before it is authenticated by the
AEAD tag.

> In my reading of this, I think I have identified two points you’re making.
> 
> (1) It’s possible for a chunk to be larger than reasonable processing resources.
> (2) It’s possible for a long stream to have an error in the last chunk that signals an error wayyyyyy in the past.
> 
> Handling (1) is reasonably easy. Return an error. This situation exists today. It’s possible to make partial bodies of a gigabyte each, and an implementation may not be able to handle that. Return an error.

I don't understand why partial bodies are relevant here.  They are
quite easy to implement using a small buffer even given large partial
body chunks.

The issue with returning an error in the face of large AEAD chunks is
that it is completely gratuitous.  If the sender had only used a small
chunk size, it would have been possible to authenticate the
ciphertext!

What is the advantage of large AEAD chunks (serious question!)?  It
seems their only effect is to encourage the release of unauthenticated
plaintext.

> Handling (2) is also easy, you return an error. This might be unsatisfying, because the error might be in the past, and lots of stuff already handled. Is this your objection?

Supporting stream means allowing for truncation.  We want streaming,
and we are prepared to handle truncation should it occur.


Thanks,

:) Neal