Re: [openpgp] Deprecating compression support

Jon Callas <> Wed, 20 March 2019 20:57 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id C346C124BF6 for <>; Wed, 20 Mar 2019 13:57:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.85
X-Spam-Status: No, score=-1.85 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, KHOP_DYNAMIC=0.85, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id adN0F98Nf4mE for <>; Wed, 20 Mar 2019 13:57:24 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id C100A131288 for <>; Wed, 20 Mar 2019 13:57:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=04042017; t=1553115432; bh=Mc3z0RZ0zN2eo2TGwhmjqSKruLTWh+lNdBV9CLI1K+E=; h=Content-Type:Mime-Version:Subject:From:Date:Message-Id:To; b=ZI8YRipsfp4/PJaHGU94n/DoNcjgYXcPJeOOJspWAR9jNPRQV51Xv49uqtvGG+UbR 4u+GhJtMbDuZ4jSjZC+Hvvw+NVXveG3SouSSzRz8IJLmr7SaI4+vni52MXNnvAViKt 8bjIVn4L2HBxPGap8sj92cWO83NqC3sPXfdzCsBWQbKsdzOyklNMMjcjWFTQpbxChZ vQndsAv0hEvqmIUkglwhV5rRbVtelP7TDxprB1+j3BLUWgSxgAAU1fgFhcdbrA0nu0 dcF75wJD0szlq5WI89jjTfeJPHVEXVXj5gCCJ3/+ChRoSaXm1RvW0BtXgQOW3iDYcU 9kSSe9/q3KizA==
Received: from [] ( []) by (Postfix) with ESMTPSA id 13F3D1800E5; Wed, 20 Mar 2019 20:57:11 +0000 (UTC)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\))
From: Jon Callas <>
In-Reply-To: <>
Date: Wed, 20 Mar 2019 13:57:11 -0700
Cc: Jon Callas <>
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <>
X-Mailer: Apple Mail (2.3445.102.3)
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-03-20_12:, , signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1812120000 definitions=main-1903200151
Archived-At: <>
Subject: Re: [openpgp] Deprecating compression support
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Ongoing discussion of OpenPGP issues." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 20 Mar 2019 20:57:28 -0000

As mentioned in my previous note, I feel like I need to comment on the security rationales, rather than the software engineering rationale. Here’s what was given before. Andre Heinecke also had very good comments.

> On Mar 18, 2019, at 5:07 AM, Justus Winter <> wrote:
> I propose to deprecate compression support in OpenPGP.  The reasons
> for this are:
>  - Compression makes it impossible to reason about the size of a
>    decrypted message, requiring the use of a streaming interface even
>    for seemingly small messages, e.g. emails.  Experience has shown
>    that downstream users struggle with the correct use of streaming
>    interfaces.

What experience? References?

I have a raised eyebrow on this reason because the vast majority of every PGP message created in the last 28 years had compression on. I don’t know of any problems. I don’t know of any downstream uses struggling with streaming interfaces. Moreover, this is a criticism of streaming. 

Moreover, in the common case (an email), there are many plaintext transformations that go on. For example, there’s quoted-printable, hexadecimal expansion, base64 encoding, and lots of other things that go on in the plaintext that make it hard to know the length of the final plaintext. You’re right that it’s hard to know the final length of a message, but removing compression doesn’t solve the problem, nor does it make it appreciatively better. 

Are you suggesting that OpenPGP only allow definite lengths?

To me, this seems like a non sequitur even if you back it up with evidence. 

>  - Compression allows the construction of quines.

So what? How is this a problem? Text editors allow the construction of quines, too. 

This is an assertion, but it doesn’t say how it’s a problem, nor does it state how removing compression would remove the problem. 

My suggestion — moving compression to an upper layer — also allows quines, since it allows someone to encrypt a bit string that’s been compressed, and therefore might have a quine in it.

>  - Compression interacts badly with encryption, see e.g. CRIME,
>    BREACH, and hiding of EFAIL-style CFB gadgets [0].

This isn’t strictly correct.

There are a number of attacks on interactive encryption protocols that use differences in different compressed plaintext to learn something about the internal structure of the plaintext. This is obviously bad.

However, *static* encryption, like OpenPGP doesn’t have this problem.

Here’s a challenge I give.

Create two plaintexts, P and P’ where P’ = compress(P). Pick any compression function and any plaintext. Now, encrypt them both, so we have E_1 = encrypt(P) and E_2 = encrypt(P’). Show that there is an advantage to an attacker for recovering P’ from E_2 over recovering P from E_1.

I assert that if you can, then your cipher is flawed and you need to replace it. There is nothing magical about compressed plaintext that makes it easier to recover.

With regards to EFAIL, that is a huge problem that is far more to do with MIME encoding than anything else. It applies to S/MIME as well as OpenPGP, and S/MIME typically does CBC mode as opposed to CFB and typically does not compress. Thus the connection is tenuous, because EFAIL applies not only to compressed data with CFB mode, but uncompressed data with CBC.

>  - The downstream application is in a better position to decide whether
>    and how to compress data that is then encrypted using OpenPGP.

I kinda agree with you, in this one. I’d say upstream rather than downstream because I think that the generator of encrypted objects is upstream, and the thing that processes the output is downstream. But I agree with what I think you’re saying.

Nonetheless, this is a feature that’s been there since Day One, and there are many systems that rely on OpenPGP compressing. In that case, the other application has made a decision and this proposal would override their decision on the grounds that they should be allowed to make a decision.

This is why while I like deprecating compression, I think we need to move with care.

>  - Compression make the standard more complex, and enlarges the
>    trusted computing base of implementations.

I don’t disagree here, and this is essentially my rationale. However, to be fair, this is a part of the system that is used on nearly every encryption and there haven't been many (or any?) problems. Removing something is also risky and introduces other chances for errors, too.

This is why taking action *now* by just stopping to use compression is a very good idea. It shows that there won’t be compatibility issues, and the more that the actual use goes down, the more than an argument to remove makes sense. 

The worst thing possible would be that no-compression gets tied into things like V5 keys (which we need) and AEAD modes (which we also need) is that it would blunt adoption of the good new technologies by adding on and tying things together. This is a classic problem that many systems have, that the goodness is blunted because it drags in something about which gentlepersons can disagree. Okay, enough soapbox.

I think we should start moving away from default compression, starting now. I think we should move slowly with the standard and collect data about use, which would support removal in a future update.