Re: [openpgp] AEAD Chunk Size - Performance
Bart Butler <bartbutler@protonmail.com> Thu, 28 February 2019 19:11 UTC
Return-Path: <bartbutler@protonmail.com>
X-Original-To: openpgp@ietfa.amsl.com
Delivered-To: openpgp@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6A6B712D4F2 for <openpgp@ietfa.amsl.com>; Thu, 28 Feb 2019 11:11:18 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level:
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=protonmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uZ2MMSUDF1ms for <openpgp@ietfa.amsl.com>; Thu, 28 Feb 2019 11:11:15 -0800 (PST)
Received: from mail1.protonmail.ch (mail1.protonmail.ch [185.70.40.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E8F341289FA for <openpgp@ietf.org>; Thu, 28 Feb 2019 11:11:14 -0800 (PST)
Date: Thu, 28 Feb 2019 19:11:06 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=default; t=1551381072; bh=fplQ0kS9Nv7ai+rhKy2zCPGPt+zV18FEbHCQKJ2wh3o=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References: Feedback-ID:From; b=txpNRhfkZjHBhG1xQAwVp9ecIcZfcKIEE7TKotPtqqSBah1fl67DBeAX4Fpi6HfYV cwTmfrbWAZ49zV+2A/DNs+IpMTCyLfKIvjm3Vp1YiYKpoPnmsnyDbSrXmRs+/emL0W nXTqJb1uu1UXN22gh7z/xcc+dSYOz5xUVO7bgB44=
To: "Neal H. Walfield" <neal@walfield.org>
From: Bart Butler <bartbutler@protonmail.com>
Cc: Vincent Breitmoser <look@my.amazin.horse>, "openpgp@ietf.org" <openpgp@ietf.org>
Reply-To: Bart Butler <bartbutler@protonmail.com>
Message-ID: <C6Yk1rFFoqbz_qGTLC6hMtmhMtRifnQp_-iuICEa_uXIK4BSuKVZt9OId8oTvfV4SYAGr-Syo7lqdHgBgzEHvU7-BFN8KgGCEn766SargwQ=@protonmail.com>
In-Reply-To: <87k1hk2tpv.wl-neal@walfield.org>
References: <87mumh33nc.wl-neal@walfield.org> <F9VLV9HZWH.3RYL3UM3BN873@my.amazin.horse> <3WZ7-hy9V7TOy53p1gP5EXELzHJIqjouV9x0YTN3PWsBZedKkqvVCRm-2XzGZy-FYAYdTqP1-7YV4wbTWMWAYhSujQA6NmrnIuXfZLRHkdQ=@protonmail.com> <87k1hk2tpv.wl-neal@walfield.org>
Feedback-ID: XShtE-_o2KLy9dSshc6ANALRnvTQ9U24aqXW2ympbGschdpHbU6GYCTUCtfmGhY9HmOyP1Uweyandwh1AVDFrQ==:Ext:ProtonMail
MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="pgp-sha512"; boundary="---------------------7f714e1f4702dbb3e94c8d439e90f45f"; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/openpgp/fwkHyt2krTAJvKgjbqkBamJk844>
Subject: Re: [openpgp] AEAD Chunk Size - Performance
X-BeenThere: openpgp@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Ongoing discussion of OpenPGP issues." <openpgp.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/openpgp>, <mailto:openpgp-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/openpgp/>
List-Post: <mailto:openpgp@ietf.org>
List-Help: <mailto:openpgp-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/openpgp>, <mailto:openpgp-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 28 Feb 2019 19:11:19 -0000
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, February 28, 2019 12:38 AM, Neal H. Walfield <neal@walfield.org> wrote: > Hi Bart, > > Thanks for your feedback. > > On Wed, 27 Feb 2019 22:34:09 +0100, > Bart Butler wrote: > > > In our AEAD implementation for OpenPGP.js we settled on a default > > `c` of 12 rather than 8 after some performance benchmarking showed > > that 256 kiB chunks were more performant for the most common message > > sizes. Our result was specific to the web crypto API, and therefore > > specific to OpenPGP.js, but in general libraries may want to use > > different default sizes depending on implementation details like > > this. > > I don't need exact numbers, but I'd appreciate it if you could you > comment on the approximate performance of 256 kiB chunks relative to > 16 kiB chunks (e.g., factor 1.2 speedup). Of course, if you still > have the numbers lying around, that would be even better! > > There are two reasons that I find 16 kiB intuitively appealing from > a performance perspective: a typical L2 cache is about 256 kiB these > days. So a 16 kiB buffer has a good chance of staying completely in > L2. Second, on all desktop and mobile operating systems, it is almost > always reasonable to stack allocate a 16 kiB buffer whereas 256 kiB is > a bit big. > > From a robustness perspective, I'd like to avoid introducing a > threshold and having to do something brittle like: > > void *buffer; > if size > 16 kiB { > > buffer = malloc(size); > > > } else { > buffer = alloca(size); > } > > ... > > if size > 16 kiB { > > free(buffer); > > > } > > Unfortunately, I haven't benchmarked this type of thing in the context > of AEAD so this is just drawing from my limited prior experience with > sizing these sorts of things. > > :) Neal So, quoted from Daniel Huigens, who did the original benchmarking: "Re. Neal's request, I didn't have the numbers anymore, so I unscientifically created some new ones It looks like either Chrome or we have actually fixed the performance issues there with c=8 since the last time I tested it, as for most message sizes the performance is the same as c=12 or even a bit faster, except 128KB-256KB, which fits in one chunk with c=12 and there c=8 is about 1.1x - 1.5x slower. In Firefox, for >=64KB messages, c=8 is still about 1.5x - 2.5x slower than c=12, however, it looks like most of the overhead of smaller chunks comes from the streams polyfill, not the web crypto API. So that should probably be possible for us to fix." I see your logic on 16 kiB but in particular for file encryption I think there's some value to being able to go bigger in the future (L2 cache sizes have changed in the last 20 years as well and will probably change in the next two decades), and to keep the size byte to be able to configure this rather than have an entirely new packet. I think a SHOULD for 16 kiB though is totally appropriate now though for the reasons you mentioned.
- Re: [openpgp] AEAD Chunk Size Justus Winter
- [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Ronald Tse
- Re: [openpgp] AEAD Chunk Size Ronald Tse
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Hanno Böck
- Re: [openpgp] AEAD Chunk Size - Performance Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size - Performance Bart Butler
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size brian m. carlson
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size - Performance Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Sebastian Schinzel
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Sebastian Schinzel
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Peter Pentchev
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Vincent Breitmoser
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- [openpgp] WTF (Re: AEAD Chunk Size) Andre Heinecke
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Nickolay Olshevsky
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Tobias Mueller
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Bill Frantz
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Marcus Brinkmann
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Wyllys Ingersoll
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size brian m. carlson
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Peter Gutmann
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] AEAD Chunk Size Conrado P. L. Gouvêa
- Re: [openpgp] AEAD Chunk Size Conrado P. L. Gouvêa
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Neal H. Walfield
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Jon Callas
- Re: [openpgp] AEAD Chunk Size Heiko Stamer
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Bart Butler
- Re: [openpgp] AEAD Chunk Size Derek Atkins
- Re: [openpgp] AEAD Chunk Size Werner Koch
- Re: [openpgp] AEAD Chunk Size Benjamin Kaduk
- Re: [openpgp] [EXT] Re: AEAD Chunk Size Neil Hunsperger