Re: [ftpext] New Version Notification for draft-bryan-ftp-hash-05

Anthony Bryan <anthonybryan@gmail.com> Mon, 12 July 2010 22:39 UTC

Return-Path: <anthonybryan@gmail.com>
X-Original-To: ftpext@core3.amsl.com
Delivered-To: ftpext@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 780183A687D for <ftpext@core3.amsl.com>; Mon, 12 Jul 2010 15:39:14 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.166
X-Spam-Level:
X-Spam-Status: No, score=-2.166 tagged_above=-999 required=5 tests=[AWL=0.433, BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H0FU2BC4umh6 for <ftpext@core3.amsl.com>; Mon, 12 Jul 2010 15:39:12 -0700 (PDT)
Received: from mail-iw0-f172.google.com (mail-iw0-f172.google.com [209.85.214.172]) by core3.amsl.com (Postfix) with ESMTP id 9F9E53A681D for <ftpext@ietf.org>; Mon, 12 Jul 2010 15:39:12 -0700 (PDT)
Received: by mail-iw0-f172.google.com with SMTP id 38so5368504iwn.31 for <ftpext@ietf.org>; Mon, 12 Jul 2010 15:39:21 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=i8RtZUtQyHEcTWPuswjNnuf8tWH4NsXeAPCWm1cWHug=; b=QxQ85aahqokZ6Iq0+v+oKEykJFPGE8DhOxX8D3ruavbIHs4uLU4TvjWFeG2Qnk4Vza jV9kMM9BPdJNijlCOOrvl2Pv0HN0qeWZSj80SPD7WyxP50Mzp4HzeCTXMo7xGGbRdAF4 865ANhMUWRaMOGRTNWoTo5Hzxx7p8nY9GS6xA=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=YVk620KvM8FEM3NyYKsXE6yVpgFABDO0KKHriyW3aO81SxYpbyhM3jnlR4C5RnO3IM wVSvkB8SrSChgLcF3Pu9mPeFj24NTvpREyz9EB5W1YHa5LRdp5ByVe2NE1Pd1EGcmOie vxq+mFc0RpOYAon4jFR+T9hmLjjO3s7EEfR/U=
MIME-Version: 1.0
Received: by 10.231.37.75 with SMTP id w11mr12806577ibd.45.1278974360686; Mon, 12 Jul 2010 15:39:20 -0700 (PDT)
Received: by 10.231.161.143 with HTTP; Mon, 12 Jul 2010 15:39:20 -0700 (PDT)
In-Reply-To: <D2D3FF30-3B2D-4EFA-A308-18A8A56DE596@muada.com>
References: <20100629052221.092623A6A5C@core3.amsl.com> <alpine.DEB.2.00.1006300000470.1493@tvnag.unkk.fr> <AANLkTikZFhGd3PIwDNrNcKymcoRzc4RHxIJU8TGPztTp@mail.gmail.com> <alpine.DEB.2.00.1006291639050.30072@familiar.castaglia.org> <D2D3FF30-3B2D-4EFA-A308-18A8A56DE596@muada.com>
Date: Mon, 12 Jul 2010 18:39:20 -0400
Message-ID: <AANLkTikPOdAaVSO7wsfEFsXtZwKoBOAWoI9qwnF_wF1F@mail.gmail.com>
From: Anthony Bryan <anthonybryan@gmail.com>
To: Iljitsch van Beijnum <iljitsch@muada.com>
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: quoted-printable
Cc: ftpext@ietf.org
Subject: Re: [ftpext] New Version Notification for draft-bryan-ftp-hash-05
X-BeenThere: ftpext@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <ftpext.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/ftpext>, <mailto:ftpext-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ftpext>
List-Post: <mailto:ftpext@ietf.org>
List-Help: <mailto:ftpext-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ftpext>, <mailto:ftpext-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 12 Jul 2010 22:39:14 -0000

On Mon, Jul 12, 2010 at 5:59 AM, Iljitsch van Beijnum
<iljitsch@muada.com> wrote:
> A more efficient and less DoS-prone way to implement this would be for the client to tell the server the hash, and for the server to tell the client whether the file matches the hash.
>
> To optimize this, the hash wouldn't be a straigh hash over the entire file, but consist of three parts: the file length, a hash over a small part of the file (say, the first and last kilobyte) and finally a hash over the entire file. So if the length doesn't match, the file doesn't match the hash and there is no need to access the file. If the hash over the small part doesn't match, there is no match and hashing the entire file is avoided.

Iljitsch, interesting approach.

I believe one of the draft's earlier incarnations also had a CHKHSH
command which did just this.

unless I'm misunderstanding, this would be MORE indirect, unoptimized,
complex and DoSy than the current draft.

to be able to answer whether a hash is correct, the server still needs
to know the correct hash?
when you meet someone, do you ask "is your name Bob? no, how about
Jim? no, Ed?" or do you just come right out and ask "what's your
name?" which is simpler?

MirrorBrain is an example of a server, although HTTP not FTP, that is
a hash/mirror cache/provider: http://www.mirrorbrain.org/
it also might be worth examining what the other FTP servers that
implement the nonstandard hash commands do.

you mention partial file hashes, some of those nonstandard commands
like XCRC and XMD5 support it with a byte range.
do you think HASH should support partial file hashing?

> Nitpick: repeating the first part of the abstract in the introduction verbatim is a bit annoying.

sure, we'll work on that.

-- 
(( Anthony Bryan ... Metalink [ http://www.metalinker.org ]
  )) Easier, More Reliable, Self Healing Downloads