[MLS] Performance measurement of full tree PR

Richard Barnes <rlb@ipv.sx> Fri, 27 May 2022 16:10 UTC

Return-Path: <rlb@ipv.sx>
X-Original-To: mls@ietfa.amsl.com
Delivered-To: mls@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 993ABC185144 for <mls@ietfa.amsl.com>; Fri, 27 May 2022 09:10:41 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.495
X-Spam-Level:
X-Spam-Status: No, score=-0.495 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=ipv-sx.20210112.gappssmtp.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qYqZ9PWHIkvt for <mls@ietfa.amsl.com>; Fri, 27 May 2022 09:10:40 -0700 (PDT)
Received: from mail-qt1-x832.google.com (mail-qt1-x832.google.com [IPv6:2607:f8b0:4864:20::832]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E8C4AC185134 for <mls@ietf.org>; Fri, 27 May 2022 09:10:40 -0700 (PDT)
Received: by mail-qt1-x832.google.com with SMTP id s31so5695546qtc.3 for <mls@ietf.org>; Fri, 27 May 2022 09:10:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipv-sx.20210112.gappssmtp.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=lHXmL89hxJkJ9SAQcvCOd9DK4Q5V9COi/Bz23FQrZHY=; b=sSxGk9TZJU04qbmlxpeGrquSNViUVwjUaiSJprAkrNijQsy2RpfHIM4nhvBt+08nTY 7TmV/G1pdUYvB+clSlxniEL5WDQg0hHBH7B/PFsOte1PhNIqWFy4odC1A+f8LzSX5Led xAaVxuIb/p5qZ9NMGpo3mOdHFVB+FkwBHhgZnWcDVJym3RzrvdR5Kl/5tG/i6/5EU1mZ k46EhcgwBdYEH+ZhCh3d8GLd+lupLCGX5s6ieFfuAvVW+f47iEfyMsLET9Fav0kowhNp uTCfY6iDjA24KjWCIMNQM8QbK6se06gQv7O0HJkztfUdZB7+kPp/u/CTf8CdugoJkuQX cvdw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=lHXmL89hxJkJ9SAQcvCOd9DK4Q5V9COi/Bz23FQrZHY=; b=adDkYCDgWv3nEQcRQVv9V7WqlKCMqFtFHyN68sgOxcHMaLClmtNSp3DN1R6IigzXW7 PG8ZSvx8wTpty0Izh3+k/lUVqu+dKLVjNeJaJTQIrkR7XymFHHye1ty/r1Tz3fkgUQ+H 4YgCWrvm4mHWEeeqtkAcH39uY4tK1Y8XUSDfNLiC/b5klJ63f76xa3tov7pjYt3R8Xlx NlWMYymk20Re8E24uM7Ex3GjK9lrY4IjFeKG5kD7Hy8yo1eYkr4uoR4vw5Fdq1xk9SMz sv4GVCcPnWydsw0R09hKyQc+751ym/GQXi52Mc4eqNsYnsCbKyZr60LuVcI37FlJApo0 8EOQ==
X-Gm-Message-State: AOAM533s/HPdWIGg1FYDaD8LW6nMT/so6OmNmq2gpP/eEFfDigWmeAW1 B8gWn1M/z900ZuJqaE3Y9NcjQn6PRoeT8B2Dvk5ZOgF8mJgWOVWS
X-Google-Smtp-Source: ABdhPJwOl6ZbvjktCFDZckGRSOTYulqTBn909DxeXk+cKCTJXgI1CmVzqm0QPpJV2RUMGTGG6YTXVdePvYvY1WOEQAk=
X-Received: by 2002:ac8:5750:0:b0:2f9:410b:7101 with SMTP id 16-20020ac85750000000b002f9410b7101mr16054181qtx.291.1653667839103; Fri, 27 May 2022 09:10:39 -0700 (PDT)
MIME-Version: 1.0
From: Richard Barnes <rlb@ipv.sx>
Date: Fri, 27 May 2022 12:10:28 -0400
Message-ID: <CAL02cgSq=VUz7Hh8jLbf6R+Nw9JC54CW30RgEKOk3gw_pxwjaA@mail.gmail.com>
To: Messaging Layer Security WG <mls@ietf.org>
Content-Type: multipart/alternative; boundary="00000000000020678c05e0008ca1"
Archived-At: <https://mailarchive.ietf.org/arch/msg/mls/42tmYeZIJRAAZTxnSb6pzXKZGa0>
Subject: [MLS] Performance measurement of full tree PR
X-BeenThere: mls@ietf.org
X-Mailman-Version: 2.1.34
Precedence: list
List-Id: Messaging Layer Security <mls.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/mls>, <mailto:mls-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/mls/>
List-Post: <mailto:mls@ietf.org>
List-Help: <mailto:mls-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/mls>, <mailto:mls-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 27 May 2022 16:10:41 -0000

Hi all,

As promised on yesterday's call, I ran some performance metrics on the
"Always use a full tree" PR (#645).  I took a prototype I had of the
tree-hash-based parent hash and extended it to use a full tree.  Note that
I did *not* implement the suppression of redundant nodes, so these trees
have the redundant nodes in them.  The only real bias that that would
introduce here is to do log(n) extra parent hash verifications in the
full-tree case; in other words, the full-tree case should be slightly
better in reality than it is here.

The testing methodology is basically the same as [1]: Make a tree with N
members and see how long it takes the Nth joiner to import the tree.  The
main difference is that since [1] I got a new laptop, so I can go to N=512
instead of just N=256 :)

PDF with plots of full results at [2].  Summary:

* Time to compute tree hash is up to 30% worse, never better
* Time to verify parent hashes is up to 40% better, never worse
* Overall time to ingest a tree (sum of the above) varies between 30% worse
and 20% better

The absolute total ingest time on the high end (512 participants) is on the
order of 60ms, so we’re talking about variations of around plus or minus
15ms.  This is on a current-year M1 MacBook Pro.

The primary driver for whether the full tree does better is how filled-in
the tree is, in the sense of parent nodes being populated.  If even 1/4 of
the tree is filled, then you’re basically always better off with a full
tree, even with 3/4 of leaves unmerged.

Overall, these numbers don’t seem intimidating to me, so I’m inclined to
merge the PR

—Richard

[1] https://mailarchive.ietf.org/arch/msg/mls/QrxmpAzKxoAGKTuzOM3M1UcKFAA/
[2]
https://drive.google.com/file/d/1yQWkmDTKnS1zpkjYb5gn7vmgcjLX0BKt/view?usp=sharing