Re: [openpgp] AEAD Chunk Size
Tobias Mueller <muelli@cryptobitch.de> Sun, 17 March 2019 20:01 UTC
Return-Path: <muelli@cryptobitch.de>
X-Original-To: openpgp@ietfa.amsl.com
Delivered-To: openpgp@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CD4271311E1 for <openpgp@ietfa.amsl.com>; Sun, 17 Mar 2019 13:01:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KfOC2IyEu40J for <openpgp@ietfa.amsl.com>; Sun, 17 Mar 2019 13:01:34 -0700 (PDT)
Received: from bitbox.cryptobit.ch (cryptobit.ch [188.40.138.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 756A81311DF for <openpgp@ietf.org>; Sun, 17 Mar 2019 13:01:34 -0700 (PDT)
Received: from unibox.fritz.box (p5B0F5932.dip0.t-ipconnect.de [91.15.89.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mail.cryptobit.ch (Postfix) with ESMTPSA id 44Mqvc3d5qz13Bb4; Sun, 17 Mar 2019 21:01:32 +0100 (CET)
Message-ID: <734216a3ee1c0e252ecf0ad297be2cfdcb049c2e.camel@cryptobitch.de>
From: Tobias Mueller <muelli@cryptobitch.de>
To: "Neal H. Walfield" <neal@walfield.org>
Cc: Jon Callas <joncallas@icloud.com>, Bart Butler <bartbutler@protonmail.com>, "openpgp@ietf.org" <openpgp@ietf.org>, Vincent Breitmoser <look@my.amazin.horse>
Date: Sun, 17 Mar 2019 21:01:32 +0100
In-Reply-To: <878sxqy5kw.wl-neal@walfield.org>
References: <87mumh33nc.wl-neal@walfield.org> <F9VLV9HZWH.3RYL3UM3BN873@my.amazin.horse> <3WZ7-hy9V7TOy53p1gP5EXELzHJIqjouV9x0YTN3PWsBZedKkqvVCRm-2XzGZy-FYAYdTqP1-7YV4wbTWMWAYhSujQA6NmrnIuXfZLRHkdQ=@protonmail.com> <CAB941EE-6961-4CAB-9632-DFF738980467@icloud.com> <87h8co2t4v.wl-neal@walfield.org> <35C892F7-18A8-401E-828D-5CE180A3A731@icloud.com> <87r2brf0f1.wl-neal@walfield.org> <2a014c4a103ba7f52535546f7e77277ea2bdabdf.camel@cryptobitch.de> <878sxqy5kw.wl-neal@walfield.org>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.28.5-0ubuntu0.18.04.1
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/openpgp/LoqYW3MtTvVD0rhgKHPIYEt6xO0>
Subject: Re: [openpgp] AEAD Chunk Size
X-BeenThere: openpgp@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Ongoing discussion of OpenPGP issues." <openpgp.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/openpgp>, <mailto:openpgp-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/openpgp/>
List-Post: <mailto:openpgp@ietf.org>
List-Help: <mailto:openpgp-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/openpgp>, <mailto:openpgp-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 17 Mar 2019 20:01:37 -0000
Hi Neal, On Thu, 2019-03-07 at 18:09 +0100, Neal H. Walfield wrote: > On Sun, 03 Mar 2019 19:36:08 +0100, > Tobias Mueller wrote: > > By fixing a "chunk size" you take away the ability to benefit from > > AE > > for messages bigger than that size. > > Implementations could easily collect all chunks and only release the > > plaintext once all chunks check out successfully. But that could go > > wrong. And depending on implementations to get things right and > > clients > > to use those implementations correctly is exactly what enabled Efail > > to > > become an issue. I think it'd be much nicer if the protocol already > > ensures that my emails do indeed enjoy protection against > > modification > > rather than me having to rely too much on clients getting it right. > > You're afraid of bugs. So am I. Dynamically resizing buffers is > error prone. Even computing AEAD's chunk size is hard: point taken. Programming is hard. Programming C is even harder. Tooling could be improved. We ought to write a spec that achieves a security goal while making it less likely that people implement the spec wrongly. You've made a point but you haven't really taken a stance on the point that I raised. Which boils down to your proposal forcing everybody to use chunking which in turn can be implemented wrongly while providing no benefit in return. For the sake of the argument, let's assume a safe aead_decrypt oracle. I think that any CAESAR candidate provides such a function that is sufficiently well suited. I believe that the potential for releasing plaintext although the ciphertext has not been authenticated is higher if I need to call that oracle multiple times, invent a secure scheme for detecting truncation, and implement that scheme correctly. Your proposal removes the option for the message producer to decide whether it wants to have the recipient (which may very well be the same entity) rely on an implementation which gets all that right. Because with the status quo, one can opt for producing one chunk which is trivially protected against truncation and other attacks. Which is probably what I expect when I opted for protecting my message with AE. And as far as I understand the current spec, it would not be directly illegal for an implementation to release plaintext of a chunk early. So my only chance as a message producer to not rely on implementations to not release plaintext early is to use one chunk. (uff. so many negations. Sorry about that). Unless I understand your proposal wrong. But then my question is, how would I encode a message that is not susceptible to (at least) truncation attacks? Again, assuming the above mentioned oracle. I don't fully understand why you are mentioning dynamically resizing buffers. Strictly speaking, you need space for a single digit number of blocks (!) to decrypt an EAX or OCB message. And you are free to release unauthenticated plaintext as much as you want, much like your current proposal. If you want to process the message in 16kB pieces only, I don't see why you couldn't. Even with a 5 exabyte message (or chunk). If you don't want to reveal the plaintext, you could even mask the output and only offer to remove the mask after the last chunk has successfully checked out. I can understand that the concerns around relying on correct chunking can be dismissed, because several protocols exist that have successfully (for all we know so far) done such a thing, so implementations can copy and paste existing solutions. I think, though, that it has not been appreciated enough that chunking carries risks, because it relies on the implementation to not release plaintext early. And, as we all know, releasing unauthenticated plaintext has lead to EFail. If using proper AEAD is the response to EFail then your proposal to remove the option of not having to rely on clients not releasing early plaintext is not helping implementations to do the right thing, i.e. not release unauthenticated plaintext. > > #include <stdio.h> > > int chunk_size(int c) { > return 2 << (c + 6); > } > > int > main(int argc, char *argv[]) { > printf("%d\n", chunk_size(32)); > } > > $ gcc -Wall a.c > $ ./a.out > 128 > > (Note that gcc doesn't even produce a warning!) > > But, that's not all. If there is the possibility of failing due to a > lack of buffer space, implementors won't bother. And they won't > bother for a very good reason: users want to get work done. If a > security policy prevents them from getting their work done, they will > disable it. Or, an implementation will ignore it. And, because it is > only a problem if there is an attack, users will be thankful for the > more "powerful" solution. Right. One way of addressing your concern is to simply not allow chunking. Then there wouldn't be a more "powerful" solution. As such I like your proposal, but I would amend it and state that the chunk size is the size of the packet minus the overhead. We're not discussing that, though. Instead, your proposal is about *removing* the option for creating one chunk and *forcing* everybody to use multiple chunks and thus bear the risks of an implementation to get the chunking wrong. Additionally, one could argue that because you force the concept of partially authenticated plaintext onto every consumer, implementations might see no problem with releasing that plaintext early. That, in turn, can be considered worse than not releasing it, because it prevents living up to the semantics of true AE which is to not release any plaintext if the ciphertext has been modified. > > The secure implementation must be easy for both developers and users. > And the secure implementation is a small, fixed sized buffer, even if > that means the client may have to worry about truncation. Fair enough. Although I don't understand why you say "may" rather than "must". In my view, typical applications for OpenPGP do not have application level protection against truncation. I'm thinking Email and Backups. Neither of these can reliably detect that the message has been truncated. And the application can't use the partial output either. Are partially authenticated plaintexts important to you? What's your use-case for those? Could you realise your use-case with producing multiple OpenPGP messages instead? Anyway, that tradeoff which you mention, has not been part of your proposal. With one chunk, the ciphertext is trivially protected against truncation. With multiple chunks, you need to come up with a secure scheme and implement that correctly. This is far from trivial. The change you've proposed does not indicate that the potential for getting this wrongly has been considered. And it does not alert consumers that they should use the OpenPGP message format only for an application that can deal with truncated messages. Note that I don't believe the currently described chunking mechanism is broken. But the risk of implementing that wrongly is non-zero. I wonder whether it makes sense to encourage not releasing any plaintext if the ciphertext has been modified more strongly. > > > Having said that, I understand the desire for fixing a chunk size to > > reduce complexity for implementers. My desire as a user is to have > > a > > strong and resilient protocol. > > Perhaps you're not a typical user :D. Interesting point. That may be true. But I guess it's reasonable to expect a protection of the encrypted message to be as close to AE as possible when I'm using an AE cipher. As such, I expect to either get (the full) plaintext or an error. What do you think does the typical user expect when using an AE cipher? Do you think that the typical user has use for partially authenticated plaintexts? And if so, why would they not produce separate messages rather than one? > > > Is there another way to do real AE? > > Chunked AE is still real AE: it still has ciphertext integrity; it is > just susceptible to truncation attacks. First of all, you'd still need to collect all plaintexts before releasing them to get "real AE". Simply because you must either return plaintext or an error. I am not aware of a definition of AE that includes partial plaintexts. I bet that we haven't yet exhausted all possible ways which could lead to an attack. For example, I wonder whether certain applications will suffer from a timing attack because the attacker can determine which chunk they were able to tamper with. Of course, fantasizing about yet unknown attacks quickly yields esoteric scenarios and we better spend our time on currently known problems. And again, we know that it's hard to not release plaintext although the ciphertext has been modified. By using one chunk I can choose to not rely on my recipient's implementation to get the chunking right, simply because there is no chunking involved. Your proposal takes that option away and forces me to rely on the spec and the implementation having gotten the chunking right. As I've already mentioned, chunking is relatively easy and there are other things that implementations can screw up more easily instead. So it's fair to dismiss that concern. But it still feels weird to not have a way to express a truly AE protected message. That makes me concerned that users would turn to S/MIME instead. To move forward, let me ask you: Do you agree that calling something "authenticated encryption" raises the expectation of getting either the full plaintext or an error rather than partial plaintexts and an error? Do you appreciate the need for a one chunk ciphertext (this question is intentionally generic, i.e. regardless of difficulties implementing that)? Do you think we should encourage implementations to not release plaintext unless the whole ciphertext has been authenticated? Do you have use-cases for partially authenticated plaintexts? Would separating the AEAD modes into non-chunked and chunked help you? > That's not to say that > truncation attacks are not a serious issue, but I'd rather have > truncation attacks than truncation attacks *and* ciphertext modication > attacks. Are you saying that leaving the option to do one-chunk AE, as it currently exists, enables both truncation and ciphertext modification attacks? I don't understand why it would. > P.S. Consider realloc. It is only marginally easier to write this > wrong code: > > p = realloc(p, 2 * size); > if (! p) { > // Oops, p has been leaked! > return ENOMEM; > } > > than this code: > > void *temp = realloc(p, 2 * size); > if (! temp) { > free(p); > return ENOMEM; > } > p = temp; > > but I see lots of instances of the former when reading code. I hope you've filed the relevant bugs ;-) Just in case someone is finding this in the archives: GCC has __attribute__((cleanup)) and GLib has some convenience macros around it. Cheers, Tobi
- Re: [openpgp] AEAD Chunk Size Justus Winter
- [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Ronald Tse
- Re: [openpgp] AEAD Chunk Size Ronald Tse
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Hanno Böck
- Re: [openpgp] AEAD Chunk Size - Performance Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size - Performance Bart Butler
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size brian m. carlson
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size - Performance Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Sebastian Schinzel
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Sebastian Schinzel
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Peter Pentchev
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- [openpgp] WTF (Re: AEAD Chunk Size) Andre Heinecke
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Nickolay Olshevsky
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Wyllys Ingersoll
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size brian m. carlson
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] AEAD Chunk Size Conrado P. L. Gouvêa
- Re: [openpgp] AEAD Chunk Size Conrado P. L. Gouvêa
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Heiko Stamer
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] [EXT] Re: AEAD Chunk Size Neil Hunsperger