Re: [ftpext] draft-ietf-ftpext2-hash command name and default algorithm

Robert Oslin <rto@globalscape.com> Tue, 18 January 2011 15:49 UTC

Return-Path: <rto@globalscape.com>
X-Original-To: ftpext@core3.amsl.com
Delivered-To: ftpext@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CD25C3A7027 for <ftpext@core3.amsl.com>; Tue, 18 Jan 2011 07:49:32 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.979
X-Spam-Level:
X-Spam-Status: No, score=-1.979 tagged_above=-999 required=5 tests=[AWL=0.620, BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9aHTv40UGIMD for <ftpext@core3.amsl.com>; Tue, 18 Jan 2011 07:49:31 -0800 (PST)
Received: from webmail.globalscape.com (exchange.globalscape.com [208.89.186.61]) by core3.amsl.com (Postfix) with ESMTP id 46AAA3A7005 for <ftpext@ietf.org>; Tue, 18 Jan 2011 07:49:31 -0800 (PST)
Received: from exchange.forest.intranet.gs ([127.0.0.1]) by exchange ([127.0.0.1]) with mapi; Tue, 18 Jan 2011 09:52:08 -0600
From: Robert Oslin <rto@globalscape.com>
To: "ftpext@ietf.org" <ftpext@ietf.org>
Date: Tue, 18 Jan 2011 09:52:06 -0600
Thread-Topic: [ftpext] draft-ietf-ftpext2-hash command name and default algorithm
Thread-Index: Acu3HnjtSqH1wO6gQ8Sa0b3WVqCmugAABK+A
Message-ID: <F15941D3C8A2D54D92B341C20CACDF2311979DC5FE@exchange>
References: <F15941D3C8A2D54D92B341C20CACDF2311976FECC9@exchange> <20110117175600.2F89028C1A2@core3.amsl.com> <F15941D3C8A2D54D92B341C20CACDF2311976FF0A8@exchange> <alpine.DEB.2.00.1101181545230.988@tvnag.unkk.fr>
In-Reply-To: <alpine.DEB.2.00.1101181545230.988@tvnag.unkk.fr>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [ftpext] draft-ietf-ftpext2-hash command name and default algorithm
X-BeenThere: ftpext@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <ftpext.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/ftpext>, <mailto:ftpext-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ftpext>
List-Post: <mailto:ftpext@ietf.org>
List-Help: <mailto:ftpext-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ftpext>, <mailto:ftpext-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Jan 2011 15:49:32 -0000

>> I personally think CRC32 is too weak for multi-gigabyte files.

Do you mean that CRC32 is weak period? I don't see how file size would make a difference other than the time needed to calculate the hash. One may argue that the "weaker" the hash the faster it will compute, making it ideal for use with larger files (in order to mitigate the chance of client timeouts).

I know that I'm arguing from a losing position. If academics frown on MD5 then how much more will they frown on CRC32? For enterprise level mission critical transfers you want as close to zero chance of collisions, whether by malicious intent or otherwise, and CRC32 is relatively weak in this regard.

However the current draft states: 

"Implementations SHOULD NOT make any algorithm the default that is known to be weaker than SHA-1. Support for any additional algorithms is OPTIONAL"

SHA-1 at 153 MiB/Second is only marginally faster than SHA-256 at 111 MiB/Second while still being susceptible to collisions (http://en.wikipedia.org/wiki/SHA-1). If performance is the goal then changing from SHA-1 to CRC32 as the default algorithm will result in 2X performance gain while remaining just as susceptible to collisions. If security is the overriding goal then SHA-256 should be the minimum recommended, with less (and more) secure algorithms optional.

I.e. "Implementations SHOULD NOT make any algorithm the default that is known to be weaker than SHA-256. Support for any additional algorithms is OPTIONAL"

I don't know how else to balance the need for performance (avoiding timeouts) while minimizing the chance of collisions. CRC32 and MD5 are fast but collision prone. SHA-1 is slow and still collision prone. SHA-256 is slower but almost zero chance for collisions. Note that performance will become less of a problem in future as CPU speeds increase.

Sorry for the rant, but even the smallest decision can often result in huge repercussions for vendors and users later on. 

Robert Oslin

-----Original Message-----
From: Daniel Stenberg [mailto:daniel@haxx.se] 
Sent: Tuesday, January 18, 2011 8:46 AM
To: Robert Oslin
Cc: ftpext@ietf.org
Subject: Re: [ftpext] draft-ietf-ftpext2-hash command name and default algorithm

On Tue, 18 Jan 2011, Robert Oslin wrote:

> Again, I'm not insisting on XCRC over HASH and CRC32 over SHA-1 as the 
> default algorithm; I'm simply make the case that doing so will ease the path 
> for adopting the new standard for most FTP server client/server vendors.

Sure, if we want to standardize XCRC using CRC32

I personally think CRC32 is too weak for multi-gigabyte files.

-- 

  / daniel.haxx.se