Re: [openpgp] AEAD Chunk Size

Jon Callas <> Thu, 28 February 2019 18:39 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 97007130EE0 for <>; Thu, 28 Feb 2019 10:39:22 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -4.299
X-Spam-Status: No, score=-4.299 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id xP88kOl292qW for <>; Thu, 28 Feb 2019 10:39:20 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 76CD012F1AB for <>; Thu, 28 Feb 2019 10:39:20 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=04042017; t=1551379159; bh=FPa/8K9Vs269PBEwkQ/uouIMY7fXYbYM8yrnWCxrmnw=; h=Content-Type:Mime-Version:Subject:From:Date:Message-Id:To; b=dkd9vVd7bJ+m2Azh3D3ANROMS8z/4oBe3VXIQJ5pbousPd6eqGRbabgF7h8yQZnrq Pr6/phixsBQnyR5+pkS07tYANs0D13/wWlWNBqnpxz828+R4eatWohX41hxyhV7w7Z KVH/gzqAgKSPrQiivtaH4Bd0JRiNrtMXHHZR6qOJp+70/lrhsx0WYScR0HsSGKUi8r C+yFNbBLzgj5Fcf30W4o01Fw24NJm9ktAG/RAl5EjSx90bYQgCxAuiyiKu/ReKt0uO a1QlNstlz2cpfB21Lsg/oq3xLaOtnfytQjFDliTMs3OHPm3bcSHHW2LOWLfoxrIsPw d1CK+XWD3A++g==
Received: from [] ( []) by (Postfix) with ESMTPSA id E8124400244; Thu, 28 Feb 2019 18:39:18 +0000 (UTC)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\))
From: Jon Callas <>
In-Reply-To: <>
Date: Thu, 28 Feb 2019 10:39:17 -0800
Cc: Jon Callas <>, Jon Callas <>, Bart Butler <>, "" <>, Vincent Breitmoser <>
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <> <> <>
To: "Neal H. Walfield" <>
X-Mailer: Apple Mail (2.3445.102.3)
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-02-28_10:, , signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1812120000 definitions=main-1902280124
Archived-At: <>
Subject: Re: [openpgp] AEAD Chunk Size
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Ongoing discussion of OpenPGP issues." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 28 Feb 2019 18:39:23 -0000

> On Feb 28, 2019, at 12:51 AM, Neal H. Walfield <> wrote:
> I think you misunderstand the point of the chunk size parameter.
> Modulo perhaps performance, the chunk size is not visible to the
> application.  That is, messages encrypted using AEAD can
> (theoretically) still be 4 exibyte large when using a 16 kiB chunk
> size; a message can consist of any number of chunks.  Chunk size is
> like HTTP's chunked transfer encoding or OpenPGP's partial body
> lengths.
> The reason that we don't want large chunk sizes is that when
> decrypting a chunk, the chunk MUST (RFC 5116) be buffered.  Since the
> encryptor generally doesn't know the context in which the decryption
> will occur, it must be conservative and choose a small chunk size.
> Otherwise, a decryptor will inevitably encounter a chunk size that it
> can't buffer and gratuitously fail.  Alternatively, it violates the
> must and outputs the decrypted data without first verifying it thereby
> resulting in the next EFAIL.

Perhaps I do misunderstand, but that’s kinda what I was expecting; an equivalent to the old streaming chunks.

>> So I think that a SHOULD is the right way to put it. I care less
>> about what the SHOULD limit should be, but a small hard limit sounds
>> like a bad idea.
> Do you still think a hard upper limit is a bad idea?

No, I don’t think a hard upper limit is a bad idea. In the text you quote above, I said a *small* hard upper limit. And no, I don’t know what small means. I think 16K is small. 256K might be small. AES-GCM has issues about 64G. One might argue that’s a reasonable large upper limit. Me, I think anything in the few megabytes to a gigabyte-ish is just fine.

Let me rewind a bit and get to my real point which I don’t think I’m making well.

When one creates a standard, one needs to be careful with parameters, particularly the MUSTs. Seemingly sensible things can have downstream effects that convince people to use some other protocol. Worse, someone’s angry blog post about something can quickly go into “not even wrong” territory and embed itself into folklore and you can’t get it out. There are plenty of bad ideas that someone else has a really reasonable use case for.

The Partial Body Lengths that OpePGP has had from the beginning have no restrictions on them. There’s non-normative guidance that pretty much says you shouldn’t even use them, but there’s no restriction on size. This has never caused a problem that I’m aware of. I’m sure it caused a problem that none of us are aware of, and the implementors just solved it on their own. It is this experience that has me wondering about what the restrictions out to be.

The best way to deal with it in a standard is to have non-normative guidance. It is non-normative guidance that I was suggesting. Let me write an example bit of non-normative guidance below:

- - - - -

Implementations should pick a chunk size with care. Depending on the AEAD algorithm, large chunk sizes may limit the ability to process chunks, particularly with an AEAD algorithm that is not single-pass, and thus the system decrypting it must hold the entire chunk as it decrypts it. In general, the chunk size is irrelevant to any code outside of the chunk handling code, so smaller is better.

An implementation SHOULD pick a chunk size of 256KiB or less. An implementation MAY use larger sizes, but this could limit interoperability with resource-limited systems and should be done with care.

- - - - -

I don’t think that’s perfect, but it’s okay as a first draft. I would completely support something like the text above. I’d support it if it said 16KiB too. I’ll also support hard limits, but my intuition is that that decision will come with frustration for someone else.

Does this make more sense?