Re: [openpgp] AEAD Chunk Size

[Jon Callas <[e-mail]>]

> On Mar 29, 2019, at 2:07 AM, Neal H. Walfield <[e-mail]> wrote:
> 
> On Fri, 29 Mar 2019 03:19:39 +0100,
> Jon Callas wrote:
>> Like some interim replies, particularly Bart Butler, I thought we had a rough consensus and that it was approximately:
>> 
>> * MUST support 16KB chunks.
>> * SHOULD support 256K chunks, as these are common (Protonmail).
>> * MAY support larger up to the present very large size.
>> * MAY reject or error out on chunks larger than 16KB, but repeating ourselves, SHOULD support 256K.
> 
> I still think this is a bad idea.  

Noted. It was a position that I threw out because I thought it encompassed a lot of things that people said and also allows you, the implementer to say, “screw it, we’re doing 256K only and to heck with all of you” and it would meet the standard. You could even implement 64K, thus generating a brinksmanship debate with you and Protonmail and I don’t have to listen to it.

I think that my proposal above is flawed because there’s this squishy space between 64K and 256K, and really hoped someone would either say, “Fine. I’ll give up on 256K” or “Fine. 256K is fine.” And then we’d modify the proposal and probably be done. It was, however a position I thought we might reason from.

>> I could also get behind a hard limit of 2^30 on the grounds that that’s what we had for partial body lengths, but I understand the comment that there are things like multi-terabyte S3 buckets and out and out forbidding those to be single-chunk is a bit churlish, but only a bit.
> 
> This is the bit that I don't understand.  Clearly you see some
> advantage to having large chunk sizes.

Well, actually what I was doing there was offering a number lower than 2^whatever for the max max max oh-dear-god-why-do-you-want-to-do-this size at a (to me) very, very generous gigabyte to the other side on the max size that you can then safely ignore.


> 
> But, let me try to use your framing:
> 
> I propose that we remove the chunk size parameter from the AEAD header
> and fix it to a small value (e.g., 64 KB or 256 KB), because 1.) no
> one has shown that a large chunk size is useful, 2.) large chunk sizes
> encourage implementations to release unauthenticated plaintext, which
> is a serious security concern [1], 3.) if an implementation releases
> unauthenticated plaintext for large chunks, then an attacker can
> always convince it to release unauthenticated plaintext for the first
> chunk in a message [2], 4.) making the chunk size variable increases
> complexity, which is a security concern.

This isn’t actionable.

There’s a lot of rationale here, but not an actual proposal.

My proposal (left at the top of this missive for easy referral) was a concrete proposal. The above paragraph has a lot of justification in it and is interesting polemic but it is not an actual proposal.

If you had said:

* Chunk size is 256K. Always. If you have less than 256K, pad it with zeroes.

Then that would be a proposal.

> As I understand your position, you want to allow implementations a
> broad degree of flexibility in choosing the chunk size.  Can you
> please explain why this is useful?  I've reread your messages, but
> beyond what appears to me to be some hand wavy performance argument,
> and an apparent misunderstand (chunk size != message size) [3],
> neither of which are convincing, I can't find any other arguments.
> I'm sorry if you did and I didn't understand.

You don’t understand my position.

My position is that there are a bunch of people (like you, at least I think you) who want to build a general-purpose implementation that you hope will be used all over the place. You have a legitimate need for small chunks.

There are other people who observe that there’s also a legitimate need for huge chunks because they want to do storage encryption on very large things and — well, they have considerations that I’m not paying a lot of attention to.

My proposal above lets them do their thing and lets you ignore them. That’s my position. Making as many people happy as possible.

> 
> I'm a bit confused.  In my original mail, I included a PR:
> 
>  https://mailarchive.ietf.org/arch/msg/openpgp/XH098WlJe8lkOypIaB1IXTde2sY
> 
> Was that not actionable enough?  Should I link to the PR more often?

It would help, but even better would be to send what the proposal is. I really did read it and it’s a patch file. Patch files are hard for humans to read. It’s really nice to have a redline, but it’s also nice to know what the full thing is without having to mentally emulate git.

> 
>  I've spent some time thinking about use cases for different chunk
>  sizes, and I can't come up with any modulo some, IMHO, insignificant
>  performance tweaks.  Can you please give some examples of use cases
>  that would profit from different chunk sizes?

I’m not the one proposing the large chunk sizes. And as I understand the people who are doing it, performance isn’t their issue, data integrity is.

There’s a dilemma in here.

One is that there’s a belief that 

> 
>  What should / would you recommend an implementation do if it
>  encounters a chunk that it can't buffer?  I see two choices: report
>  an error, or release unauthenticated plaintext.

Report an error. I’ve said that many times.

> Regarding our implementation: it doesn't actually have tight resource
> constraints.  Our primary goal is to write a secure implementation.

Okay, so you’re arguing resources when you don’t really mean it. Gotcha. I thought that was the case that the things you were saying didn’t jibe together.

> From that goal, we derived the constraint that we must always work in
> a bounded amount of space.  The current version of AEAD doesn't allow
> this without potentially rejecting messages that other implementations
> can process by being insecure (releasing unauthenticated plaintext),
> which we don't want to do.

That constraint is a long-standing meta-requirement in OpenPGP, so cool.

However, I detect that there is an impossible-to-solve problem here, and I’ll outline it in another note later.

> 
> But, my concerns are not only about my implementation.  I'm concerned
> about the ecosystem.  And, the current proposal encourages insecure
> implementations as demonstrated by GnuPG and RNP processing
> unauthenticated plaintext.  I think the standard should not
> proactively make it harder to write a secure implementation.  And that
> is what I see the AEAD chunk size doing.

I think I understand where you’re coming from, but this is a standards organization. Standards are concerned with interoperability between implementors. I learned that from Jeff Schiller ages ago. Think of a standard as being like a dictionary and grammar. When you send me a message, I use that grammar/dictionary to determine what it means. Similtharly, if I want to encode a message that I want you to understand, then I encode it according to that grammar as well.

My proposal above says to an encoder that the safest thing to do is to use a chunk size of 64K. Everyone can decode that, and you’re fine. Many, many, many of them are going to be okay with 256K, and odds are that anyone of any import will do fine with it. (That’s what the SHOULD means). It also says there are people out in Lala-land who might be playing with chunks that are out to 2^whatever, and that’s kinda cool, but ignore them. 

That’s why I made it. It gives a sandbox for people who want to go out on the edge that the mainstream can ignore.

> 
> 
> Thanks,
> 
> Neal
> 
> P.S. I hope it is clear that I'm trying to engage in a constructive
> manner.  I sincerely haven't understand your position, and I really
> want to.

I think that’s because I’m trying to support a community of people who have reasonable, differing needs, and you want to mandate your implementation ideas on the universe. I apologize if that’s harsh, but it’s what I perceive. If I’m mistaken, let me know.

	Jon