Re: [openpgp] AEAD Chunk Size

Justus Winter <> Thu, 28 March 2019 12:30 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 8181A1202CF for <>; Thu, 28 Mar 2019 05:30:23 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 6atYIequZWtw for <>; Thu, 28 Mar 2019 05:30:20 -0700 (PDT)
Received: from ( [IPv6:2a00:1450:4864:20::331]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 0E567120296 for <>; Thu, 28 Mar 2019 05:30:20 -0700 (PDT)
Received: by with SMTP id h18so3813469wml.1 for <>; Thu, 28 Mar 2019 05:30:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=from:to:subject:in-reply-to:references:date:message-id:mime-version; bh=wzrs/ehZD/tYEXXL5kD2J53tgqUk9IuVpzTJulMRnKY=; b=e9LgP3vNp3QDKvefpodCxPpVPxxjTg/USk2z8yxqjxoDnWqMLiQ5g0vHHmYqC+sJK2 YPdbmTvCa/nxMbqdR28gSzBEK1gy7lZMxJ2M1xGB31js4RRi9ibQ/bBRTQxeIbWLbE9C pDICbu3wYGf17xRbka5YUy1afkA8kYXV68B6QtrDy6GyTIZgjyiROPJumny9owH8ukTt R703OXVqCbOazBDazM6wTq3W/OP4gvQ+fU7klKg2fbYa4w8Wsox7JxoWCAzeOFOMHWYX 0S196wJSwT1HlzeksNHWn0KAsCND93H3dztyzcHoCH4yuwc/iw/6oYo/J7+X/54UH3KY 6lNA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:from:to:subject:in-reply-to:references:date :message-id:mime-version; bh=wzrs/ehZD/tYEXXL5kD2J53tgqUk9IuVpzTJulMRnKY=; b=UZacmdA9y6VuWyTxvuONJqTjEx1qznzBA9jQpDbwF9b7054PMwhPad/1apRNh7X4PX LHnr8zXNSlQMIAw/9CxcRTuDt8rxFcsDQ8BRFa6OKyBpvLvvmWzl/fLSyE4kYv0oiWRk sVZvD377IJTEkMXjjLwnCEGefW65X6DZC+2uEhDG+eDn2phZWzmRShimc9vZF5QV1WTw l6k1RQn4AecoKpcHCFC3BOll1DrzOfgJkxamctkvIYVaJa3Ji5CFgA7tyCi4H4C78eTK lKoMAuzRuFEIkmox85mlanysZWpH4pn+rBMgfS1jGx+Xgf+k3mMG690E/Wid4GY2QrBP toAw==
X-Gm-Message-State: APjAAAXl2crHTazcsvYeHP71m7gk281bTLL2Aqq1nmnRbTspx0owYxqG Amj3AudbOEjsahbvvxQKpiunUH+9
X-Google-Smtp-Source: APXvYqxLCL+CaI+1tylFaBQ6pS+CXnT+CqlRGHhNJFq74qRxUjFbMuYiPZfapaLy9qNZD5MLq2DwnA==
X-Received: by 2002:a1c:f00a:: with SMTP id a10mr23451719wmb.100.1553776218392; Thu, 28 Mar 2019 05:30:18 -0700 (PDT)
Received: from localhost ( []) by with ESMTPSA id c10sm32681140wrt.65.2019. for <> (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 28 Mar 2019 05:30:17 -0700 (PDT)
From: Justus Winter <>
In-Reply-To: <>
References: <>
Date: Thu, 28 Mar 2019 13:30:16 +0100
Message-ID: <>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature"
Archived-At: <>
Subject: Re: [openpgp] AEAD Chunk Size
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Ongoing discussion of OpenPGP issues." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 28 Mar 2019 12:30:23 -0000


I'm interested in building robust systems.  To do that, I need to
consider memory consumption of both my code, and the code that I'm
using, say in libraries.  I consider that important in general, but
even more so if the application in question is handling sensitive
data, and transporting it over untrusted networks, therefore it is
necessarily exposed to potentially malicious data.

In the context of processing OpenPGP data, currently there is no
relation between the size of the encrypted message and the size of the
decrypted message.  This is due to compression.  Recently, we
discussed compression on this list, and it does not look like we will
be phasing it out anytime soon.

For me, using an unbounded amount of memory is not an option for a
component processing OpenPGP data if we want to build robust systems
on top.

Therefore, we need to process OpenPGP data in bounded space.  Since
there can be no relation between encrypted and decrypted message size
due to compression, the only option I see is to provide a streaming
API, which let's us process data in constant space.

[Now, when I say constant space, implementations could still decide to
use, say, 30 megabytes of buffer space.  Then, most emails will fit
into this buffer, and we can detect truncated messages before we hand
out one byte to the downstream application.  This is what we do in
Sequoia.  Note, however, that the consumer decides how much data to
buffer before releasing the first data, and not the producer.  If we
decide to even allow 128 megabyte chunks, than the producer can
*force* the consumer to allocate 128 megabytes, or either not process
the message or do it unsafely.]

Now, as efail demonstrated, we need to protect against ciphertext
modifications, and we need to do it in a way that does not bring back
the problems with requiring unbounded space that we're trying to
address with streaming in the first place.

Therefore, we need to use chunking and authenticate message prefixes.
We need to use chunks that are reasonably small, and this size should
preferably not be configurable.  We should consider performance
constraints and pick one suitable size.  Configurable chunk sizes
bring complexity and increase the attack surface, as was pointed out
in this thread.

The only argument for a configurable chunk size that came out of this
thread is to be able to fit the entire message into one chunk.

I appreciate the desire to protect against truncation.  But,
truncation is pretty common when we transmit data, so I'd argue that
application developers are more likely to expect and gracefully deal
with truncated data than with ciphertext being manipulated or the PGP
implementation consuming unbounded amounts of memory.

Now, you may say that even if the PGP implementation doesn't buffer
the plaintext, the downstream consumer must buffer it in order to
detect truncation.  But that is not always true.  As pointed out in
this thread, you can use some kind of transaction scheme to only
commit data once it has been confirmed to be not truncated.

I have implemented AEAD in Sequoia, and I have evaluated the
implementations in GnuPG and RNP.  Every implementation is either
unsafe, not robust, or does not implement the proposal.

What is proposed in RFC4880-bis06 can not be implemented safely.  If the
working group produces a standard that cannot be implemented safely, I
consider that a grave failure of the standardization effort.