Re: [openpgp] Disabling compression in OpenPGP

[ianG <[e-mail]>]

On 19/03/2014 22:04 pm, Gregory Maxwell wrote:
> On Wed, Mar 19, 2014 at 2:41 PM, Peter Todd <[e-mail]> wrote:
>> After realizing that I needed to account for the size of the "Version:"
>> string I got the exact same size as your secret for ballot.1, so I'm
>> guessing that was your vote.
>>
>> Am I right?
> 
> Indeed, you are.
> https://people.xiph.org/~greg/openpgp_testpubkey.secret.asc  password:
> qwertyqwerty
> 
> On Wed, Mar 19, 2014 at 2:37 PM, Jon Callas <[e-mail]> wrote:
>> But it proves nothing here. It's a really, really interesting exception case. We're talking about a default. We're also talking about the protocol definition, and not implementation software.
> 
> It's a very highly surprising failure mode which leaks information
> about the plaintext by encoding it into the size, one which baffels
> otherwise expert users of the sort who would post to the openpgp list
> to exclaim "What's being leaked by compression? Really, I don't get
> it."


Clearly, compression leaks info if there is some understanding of the
structure of the message.  Sure.  But OpenPGP is typically used to ship
lots of large documents around, and people are accustomed to getting the
compression bounty for free.

So there is a sense that in order to protect the very few who might fall
for this case, we'll end up disadvantaging a wider population who might
actually suffer more.


> Even thoughtful users who take care to make their messages constant
> length will have the implicit compression sneak up and bite them.  Of
> course, you could compress and pad up to the original size…


The assumption here is that constant length is somehow more secure than
inconstancy.

If for example the voting procedure were this:

Send N messages, 1 for each candidate, with 0 for the ones you do not
vote for (we need reliability here...) and 1 + the name for the one
person you vote for (redundancy is good...) ...

Then you get the same problem;  *length leaks some info*.  This seems to
be with or without compression, so is this just a case of choosing ones
pathology?

There is no fundamental way to overcome the flaw of length in a packet
protocol, which is what OpenPGP is.  (In a stream protocol one can
overcome it by insisting on transmitting X bps forever;  this is what
ZKS did, and setting it at some ridiculously high number like 256kbps
lead the mode to be rather unpopular.)

> Voting isn't the only case where compression leaks data about the
> plain-text, it's just one where I know that it cause and actual
> compromise, with actual expert users, in actual practice.
> 
> There are a great many people who use gpg to encrypt a small number of
> control messages (e.g. for updating route registries and such) where
> compression can leak information about the content.


I see this in my work all the time.  Error packets are 10 bytes long,
success is like 500 bytes.  This is leaking info ... my only solution is
to make all packets like 1024 bytes, but some of my packets are longer.
 How far do I go?  Answer:  I've got better things to worry about right
now.  YMMV.


>> Compression is on by default because it improves security. It meets that goal. It is, however, a default. Defaults can be changed.
> 
> It does not improve security in any practical sense. The presence of
> compression harms security severely and in a very practical way in at
> least some applications. No cipher should be used where the entropy of
> the input is believed to be security relevant, and if one were used,
> the compression in gpg still leaves non-trivial known plaintext in
> cases where the input begins with known plaintext (in addition to
> additional compression headers giving more known plaintext even when
> the message is otherwise completely unknown).
> 
> (It's possible to design compression systems where the output is
> indistinguishable, e.g. arithmetic coders bijective termination, but
> none of the supported compressors are designed this way... they will
> allow you to detect an incorrect symmetric key with overwhelming
> probability with just a dozen bytes of decrypted output or less)


That;s interesting.  Maybe what we need to do is look at more
pathological cases and come up with a sort of survey of with/without?


>> Moreover, there's a way to work around the issue in the existing standard. Make the vote-submission key not support compression. Poof, it works.
> 
> Yea, sure, don't aim the gun at your foot and your foot will not be
> shot.  It's true, but it's not always useful.  What do you care about
> a specification and software? real engineers encrypt their messages by
> hand with pen and paper.
> 
> Part of the purpose of the specification and software is that doing
> everything 'manually' is too costly and difficult, the specification
> hopefully embodies the collected wisdom of careful consideration by
> the best available experts.


Well, it is true that PGP was conceived in a different world to today,
and carries those traditions which are now 20 years old.  But poking at
just one of those assumptions may not reward, because there are 10
others sitting and wobbling.


>> However, there are still many people who consider default compression a feature and *rely* on it for their system.
> 
> Care to elaborate on how they rely on it? That seems highly suspect to me.
> 
> Compression is a useful feature, it's nice that often the base64
> encoded encrypted versions are not a ton longer than ascii text... but
> if we're going through the trouble of encrypting something then the
> encryption really ought to not invisibly leak data about the
> plaintext. When something is a general tool used for many applications
> then we ought to be assuming a sum-of-all-attacks and not just a very
> narrow threat model.
> 
> Obviously a size side channel still exists, and might be worth
> reducing (e.g. by quantizing message size to a multiple of 512 bytes)
> but at least that side channel is highly visible to technical users.


So one of those traditions was that every byte saved was important, as
back in those days people still had 9600 baud to worry about.  These
days less so.  But if we're going to look at that, there are a whole
bunch of other things that also merit attention.



iang