Re: [openpgp] AEAD Chunk Size

[Justus Winter <[e-mail]>]

Hello,

I'm interested in building robust systems.  To do that, I need to
consider memory consumption of both my code, and the code that I'm
using, say in libraries.  I consider that important in general, but
even more so if the application in question is handling sensitive
data, and transporting it over untrusted networks, therefore it is
necessarily exposed to potentially malicious data.

In the context of processing OpenPGP data, currently there is no
relation between the size of the encrypted message and the size of the
decrypted message.  This is due to compression.  Recently, we
discussed compression on this list, and it does not look like we will
be phasing it out anytime soon.

https://mailarchive.ietf.org/arch/browse/openpgp/?index=dFS9kySMJCcOlR9Svys9QwP7I7Y&gbt=1

For me, using an unbounded amount of memory is not an option for a
component processing OpenPGP data if we want to build robust systems
on top.

Therefore, we need to process OpenPGP data in bounded space.  Since
there can be no relation between encrypted and decrypted message size
due to compression, the only option I see is to provide a streaming
API, which let's us process data in constant space.

[Now, when I say constant space, implementations could still decide to
use, say, 30 megabytes of buffer space.  Then, most emails will fit
into this buffer, and we can detect truncated messages before we hand
out one byte to the downstream application.  This is what we do in
Sequoia.  Note, however, that the consumer decides how much data to
buffer before releasing the first data, and not the producer.  If we
decide to even allow 128 megabyte chunks, than the producer can
*force* the consumer to allocate 128 megabytes, or either not process
the message or do it unsafely.]

Now, as efail demonstrated, we need to protect against ciphertext
modifications, and we need to do it in a way that does not bring back
the problems with requiring unbounded space that we're trying to
address with streaming in the first place.

Therefore, we need to use chunking and authenticate message prefixes.
We need to use chunks that are reasonably small, and this size should
preferably not be configurable.  We should consider performance
constraints and pick one suitable size.  Configurable chunk sizes
bring complexity and increase the attack surface, as was pointed out
in this thread.

The only argument for a configurable chunk size that came out of this
thread is to be able to fit the entire message into one chunk.

I appreciate the desire to protect against truncation.  But,
truncation is pretty common when we transmit data, so I'd argue that
application developers are more likely to expect and gracefully deal
with truncated data than with ciphertext being manipulated or the PGP
implementation consuming unbounded amounts of memory.

Now, you may say that even if the PGP implementation doesn't buffer
the plaintext, the downstream consumer must buffer it in order to
detect truncation.  But that is not always true.  As pointed out in
this thread, you can use some kind of transaction scheme to only
commit data once it has been confirmed to be not truncated.


I have implemented AEAD in Sequoia, and I have evaluated the
implementations in GnuPG and RNP.  Every implementation is either
unsafe, not robust, or does not implement the proposal.

What is proposed in RFC4880-bis06 can not be implemented safely.  If the
working group produces a standard that cannot be implemented safely, I
consider that a grave failure of the standardization effort.


Thanks,
Justus