Re: The TCP and UDP checksum algorithm may soon need updating

Phillip Hallam-Baker <phill@hallambaker.com> Fri, 05 June 2020 02:58 UTC

Return-Path: <hallam@gmail.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0BD3D3A117D for <ietf@ietfa.amsl.com>; Thu, 4 Jun 2020 19:58:05 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.398
X-Spam-Level:
X-Spam-Status: No, score=-1.398 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mANB13Zjg7io for <ietf@ietfa.amsl.com>; Thu, 4 Jun 2020 19:58:03 -0700 (PDT)
Received: from mail-ot1-f41.google.com (mail-ot1-f41.google.com [209.85.210.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C0DF13A117C for <ietf@ietf.org>; Thu, 4 Jun 2020 19:58:03 -0700 (PDT)
Received: by mail-ot1-f41.google.com with SMTP id m2so6454276otr.12 for <ietf@ietf.org>; Thu, 04 Jun 2020 19:58:03 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=r/AwP/wCv0cJ0uRzE98H7htKfYTPBsU0rpFw0a3rbDo=; b=HOUe9KDVhD5ytMWnqe/olYlH7mBdd6NVGk0Pv04WoTTkoTaz8TWt7Aj+sR1mDanrXU 4H74Vd1V2BU2VgehEq3Uhc+56yTTLidBKDSB1Kaz6qV6oSxjJ6EwjazBWRpOJNn+HB5m 3wHygEoH0xd34wWRdzFgtX1sU/ZdCshRqP5W88BEtzzPgmGN2FkbQzOCkQ/m7HQ6lyWC DXRujnJkDezNefTV80rMjd2kJE514OxnhvCMCYskEnqtT6hETirPdgYLDJCQkEdq2tG+ eGvRSFikqMhlMaqOZGNLP+9Z1HVnEtDDZIL/24otzS1oTyajYdXi7Mp8nwUHDQC+LlpQ hwpw==
X-Gm-Message-State: AOAM532+b6O5xM+nRg8V48+Y/LmVYBA7AIUeB2CP9vePHkMGjToGbdEW scEKVp3aOci+DfONG8bysQ6LNlVSyfkc/FRTlqA=
X-Google-Smtp-Source: ABdhPJwj3beiMCEjiRhmjZI9jH5L7xg8+roqjSGjUabAELHBPenwNGOGl5qwN45wgY1cnbV3itHyCG4WXNXdWFv74K8=
X-Received: by 2002:a9d:7e8d:: with SMTP id m13mr5934911otp.64.1591325882987; Thu, 04 Jun 2020 19:58:02 -0700 (PDT)
MIME-Version: 1.0
References: <CAHQj4Cem6YdTXKFPW6Mk6gK9Yt_2qD=M7PAE6nxFEdJrD==ZVA@mail.gmail.com> <8CDB0383-41B9-4D10-BCA8-FF6FC7AFF2DD@huitema.net> <db8943fc-5cd3-9ea7-2876-a5468216d86f@huitema.net>
In-Reply-To: <db8943fc-5cd3-9ea7-2876-a5468216d86f@huitema.net>
From: Phillip Hallam-Baker <phill@hallambaker.com>
Date: Thu, 4 Jun 2020 22:57:51 -0400
Message-ID: <CAMm+Lwj=5h_zgm0=fD6AjbLmsg91ctv7a6pW0fh8L9o38C1GmQ@mail.gmail.com>
Subject: Re: The TCP and UDP checksum algorithm may soon need updating
To: Christian Huitema <huitema@huitema.net>
Cc: Craig Partridge <craig@tereschau.net>, IETF discussion list <ietf@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000fa37b505a74d6d27"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/HGNEqk4h58BQnNAN6WEbPGZvars>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 05 Jun 2020 02:58:05 -0000

My first gut reaction to this was 'do the check at the application layer'.

And then I realized that was completely wrong. We have to know what error
rate we should expect at the TCP level so that we can write the application
in a sensible fashion.

Consider the case in which I am transfering a 60GB 4K movie over the net.
Say for the sake of argument there is a 1% chance of a one bit failure. If
we only have one checksum on that lot, we have a significant probability of
the effort being wasted.
There is another problem with the use of MACs which is that a MAC
verification failure is going to be reported (correctly) as an
authentication error and not as a communication error.


I don't think we can expect the transport layer to be 100% reliable. Using
MD5 makes no sense, SHA2 isn't that much slower if at all.

If we are using Merkle Damgard construction it is pretty easy to calculate
both a final value and interim values at (almost) no extra cost.

So lets imagine I am transferring that 60GB. I begin calculating the SHA2
value at the start. Every 10MB or so, I throw out the current output value.
If the receiver detects an error in the output value it calls back and
says, 'hey, that output value was wrong, resend that chunk'.

Can get fancier if people like and minimize the amount of data that needs
to be resent.

Alternatively, we could get creative and define a Merkle Tree construction
to replace Merkle-Damgard. Make it possible to parallelize digest
computation.