Re: [openpgp] AEAD Chunk Size
"Neal H. Walfield" <neal@walfield.org> Fri, 29 March 2019 09:08 UTC
Return-Path: <neal@walfield.org>
X-Original-To: openpgp@ietfa.amsl.com
Delivered-To: openpgp@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E279C120261 for <openpgp@ietfa.amsl.com>; Fri, 29 Mar 2019 02:08:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level:
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9D0KdLXS6tLT for <openpgp@ietfa.amsl.com>; Fri, 29 Mar 2019 02:07:58 -0700 (PDT)
Received: from mail.dasr.de (mail.dasr.de [217.69.77.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BCB35120257 for <openpgp@ietf.org>; Fri, 29 Mar 2019 02:07:58 -0700 (PDT)
Received: from p57b22663.dip0.t-ipconnect.de ([87.178.38.99] helo=grit.huenfield.org.walfield.org) by mail.dasr.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.86_2) (envelope-from <neal@walfield.org>) id 1h9nUR-0003fo-I0; Fri, 29 Mar 2019 09:07:55 +0000
Date: Fri, 29 Mar 2019 10:07:54 +0100
Message-ID: <87ftr6giad.wl-neal@walfield.org>
From: "Neal H. Walfield" <neal@walfield.org>
To: Jon Callas <joncallas=40icloud.com@dmarc.ietf.org>
Cc: openpgp@ietf.org, Justus Winter <justuswinter@gmail.com>, Jon Callas <joncallas@icloud.com>
In-Reply-To: <DD6BD098-A048-4513-BAAC-913BF52CDB1D@icloud.com>
References: <87mumh33nc.wl-neal@walfield.org> <878swzp4fb.fsf@europa.jade-hamburg.de> <E65F6E9D-8B0B-466D-936B-E8852F26E1FF@icloud.com> <8736n63bav.wl-neal@walfield.org> <DD6BD098-A048-4513-BAAC-913BF52CDB1D@icloud.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (Gojō) APEL/10.8 EasyPG/1.0.0 Emacs/24.5 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset="ISO-2022-JP"
Archived-At: <https://mailarchive.ietf.org/arch/msg/openpgp/332BqtMudkVPWKBlmIzi4OIzfv8>
Subject: Re: [openpgp] AEAD Chunk Size
X-BeenThere: openpgp@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Ongoing discussion of OpenPGP issues." <openpgp.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/openpgp>, <mailto:openpgp-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/openpgp/>
List-Post: <mailto:openpgp@ietf.org>
List-Help: <mailto:openpgp-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/openpgp>, <mailto:openpgp-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 29 Mar 2019 09:08:03 -0000
On Fri, 29 Mar 2019 03:19:39 +0100, Jon Callas wrote: > Like some interim replies, particularly Bart Butler, I thought we had a rough consensus and that it was approximately: > > * MUST support 16KB chunks. > * SHOULD support 256K chunks, as these are common (Protonmail). > * MAY support larger up to the present very large size. > * MAY reject or error out on chunks larger than 16KB, but repeating ourselves, SHOULD support 256K. I still think this is a bad idea. And my discussions offline with Justus and Marcus suggest that they also don't agree with this. As I understand Sebastian's position, he also doesn't agree: https://mailarchive.ietf.org/arch/msg/openpgp/SfVIqqwtYnBSOyhmdh8DnoFtlEw Bart and Sunny are also for reducing the range of allowable chunk sizes: https://mailarchive.ietf.org/arch/msg/openpgp/VN-lWmTK6JXxPKnluO7IbQJT52Y The proposal that I put forward removed the chunk size byte and standardized a single, fixed chunk size. I suggested 16kb, but given Bart's arguments, I was willing to compromise on 256kb: https://mailarchive.ietf.org/arch/msg/openpgp/JukTMAMY-RUoHsHxVxyZNht96R4 I also suggested using a different packet tag, and marking 20 as reserved to avoid breaking users of the current AEAD packet: https://mailarchive.ietf.org/arch/msg/openpgp/2SitJh7vA4cHvWGkbAMSzf1Zkio https://mailarchive.ietf.org/arch/msg/openpgp/WvaisWDd9WzoKXL1FyHDeZNtZ2U > I could also get behind a hard limit of 2^30 on the grounds that that’s what we had for partial body lengths, but I understand the comment that there are things like multi-terabyte S3 buckets and out and out forbidding those to be single-chunk is a bit churlish, but only a bit. This is the bit that I don't understand. Clearly you see some advantage to having large chunk sizes. I've asked several times on this list, but no one can tell me what concrete advantages large chunk sizes have. (Tobias has suggested completely removing chunking, which does protect against truncation attacks, but it sacrifices the ability to work with O(1)-sized buffers, i.e., streaming.) Can you please tell me why large chunk sizes are helpful? > If we really want to tie a bow around everything, then define some notation or other marker so that an implementation can mark in a key what the max chunk size is. > > How’s that? Please don't add even more complexity! Again, what are the advantages of choosing one chunk size over another? Ferguson, Schneier and Kohno write in "Cryptography Engineering": "There are already enough insecure fast systems; we don't need another one" (2.9) and "The more complex a system, the more likely it has security problems... complexity is the worst enemy of security" (2.10) I feel like making the chunk size configurable adds complexity whose only justification is some hand wavy "well, in the future it might help...". Please help me understand why large/variable chunk sizes are useful. > Now with some other points, I found myself going especially Socratic > because the missives that you and Justus wrote weren’t > *actionable*. When I was editor, a thing I would say from time to > time is at about 90% of the time someone would say, “Please change X > to Y for reason Z” it would be a no-brainer to do so. Most of the > remaining 10% would end up with some reasonable debate than then an > answer. But if someone said, “I think X is wrong” then 90% of the > time nothing would happen because there’s nothing actionable there, > meaning that there’s no clear point for the editor to be doing. All > of these standards are at some level compromises and there’s an old > aphorism that the best compromise is one where everyone is equally > unhappy. I don't understand why you think I haven't been concrete :/. In my original email, I provided a concrete patch to 4880bis: https://mailarchive.ietf.org/arch/msg/openpgp/XH098WlJe8lkOypIaB1IXTde2sY But, let me try to use your framing: I propose that we remove the chunk size parameter from the AEAD header and fix it to a small value (e.g., 64 KB or 256 KB), because 1.) no one has shown that a large chunk size is useful, 2.) large chunk sizes encourage implementations to release unauthenticated plaintext, which is a serious security concern [1], 3.) if an implementation releases unauthenticated plaintext for large chunks, then an attacker can always convince it to release unauthenticated plaintext for the first chunk in a message [2], 4.) making the chunk size variable increases complexity, which is a security concern. [1] https://mailarchive.ietf.org/arch/msg/openpgp/fmQgRm94jhvPLEOi0J-o7A8LpkY but RNP appears to do the same. [2] https://mailarchive.ietf.org/arch/msg/openpgp/WvaisWDd9WzoKXL1FyHDeZNtZ2U, starting at "Let's assume..." > In my particular case, I thought I mostly agreed with where you wanted to go, but I completely disagreed with some of your premises, and disagreed with some of the reasoning that got us to the conclusion I agreed with. That’s why I was being so Socratic. I thought before the missives today that we had a rough consensus and that I agreed with y’all at some basic level. Then I thought we really disagreed, and I gradually realized that we agree on the destination, but not on the path of how to get there. As I understand your position, you want to allow implementations a broad degree of flexibility in choosing the chunk size. Can you please explain why this is useful? I've reread your messages, but beyond what appears to me to be some hand wavy performance argument, and an apparent misunderstand (chunk size != message size) [3], neither of which are convincing, I can't find any other arguments. I'm sorry if you did and I didn't understand. [3] I believe that if we limit it to 16K, we *will* regret that, for performance or other reasons. And while some of the upper sizes are presently absurdly large, one never knows what happens a couple decades from now. Moreover, forbidding it says that we state now that no one could ever have a reasonable use; my experience is that categorical statements like that are just asking the fates to bite us in an uncomfortable place. Amazon S3 increased their max object size to 5TB a few years ago. I’m not saying it should be that large, but I think this is a pretty convincing argument that 16K is too small. https://mailarchive.ietf.org/arch/msg/openpgp/PLbg_sFze2H1INrwz_l1Uz1tPGE > If I may give some advice, it is better to make sure that your requests are actionable. If they’re not, then you’re less likely to get what you want, if for no other reason than the rest of us don’t know what you want, we’re having to guess. The basic argument that you were making ― that implementations such as yours need tight constraints ― is one we all can get behind. The other side here ― they want to do something kinda wacky (like terabyte chunks) ― can be pushed off into the land of MAYs. I'm a bit confused. In my original mail, I included a PR: https://mailarchive.ietf.org/arch/msg/openpgp/XH098WlJe8lkOypIaB1IXTde2sY Was that not actionable enough? Should I link to the PR more often? I feel that I need answers to two questions regarding the "allow large chunk size" position. I recently asked them, but I have gotten a response. Here they are again: I've spent some time thinking about use cases for different chunk sizes, and I can't come up with any modulo some, IMHO, insignificant performance tweaks. Can you please give some examples of use cases that would profit from different chunk sizes? What should / would you recommend an implementation do if it encounters a chunk that it can't buffer? I see two choices: report an error, or release unauthenticated plaintext. https://mailarchive.ietf.org/arch/msg/openpgp/m9hd4FwdvIUWKt1MWQfzBq3JahY Regarding our implementation: it doesn't actually have tight resource constraints. Our primary goal is to write a secure implementation. From that goal, we derived the constraint that we must always work in a bounded amount of space. The current version of AEAD doesn't allow this without potentially rejecting messages that other implementations can process by being insecure (releasing unauthenticated plaintext), which we don't want to do. But, my concerns are not only about my implementation. I'm concerned about the ecosystem. And, the current proposal encourages insecure implementations as demonstrated by GnuPG and RNP processing unauthenticated plaintext. I think the standard should not proactively make it harder to write a secure implementation. And that is what I see the AEAD chunk size doing. Thanks, Neal P.S. I hope it is clear that I'm trying to engage in a constructive manner. I sincerely haven't understand your position, and I really want to.
- Re: [openpgp] AEAD Chunk Size Justus Winter
- [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Ronald Tse
- Re: [openpgp] AEAD Chunk Size Ronald Tse
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Hanno Böck
- Re: [openpgp] AEAD Chunk Size - Performance Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size - Performance Bart Butler
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size brian m. carlson
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size - Performance Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Sebastian Schinzel
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Sebastian Schinzel
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Peter Pentchev
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- [openpgp] WTF (Re: AEAD Chunk Size) Andre Heinecke
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Nickolay Olshevsky
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Wyllys Ingersoll
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size brian m. carlson
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] AEAD Chunk Size Conrado P. L. Gouvêa
- Re: [openpgp] AEAD Chunk Size Conrado P. L. Gouvêa
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Heiko Stamer
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] [EXT] Re: AEAD Chunk Size Neil Hunsperger