[ftpext] draft-ietf-ftpext2-hash - partial hashes

[Robert Oslin <[e-mail]>]

draft-ietf-ftpext2-hash open issues indicates: "Current version of the draft defines full file hashes, but not partial file hashes."

Partial hashes are valuable and applicable to re-world scenarios and should be considered for inclusion in the HASH specification.

Use Case: 

	User initiates download of multi-gigabyte file, such as ISO or similar.
	Transaction is interrupted
	[Later] User re-initiates download of same (supposedly) file

At this point the client software must make a decision as to whether to resume the transfer or not. Filename is the first criteria observed, followed by size. However, these do not take into consideration a corrupted partial file (even with identical byte count), or that the remote file could have changed, especially given the difficulty in assessing time differences between client and host systems.

By requesting the hash for the portion of the remote file matching the bytes for the partial local file, the client could determine whether the local file is indeed valid and partial and subsequently resume the transfer from the appropriate byte offset.

A similar need occurs when the client has no prior knowledge of the transfer (no queue or cache mechanism) and a same name file is identified and the client must determine whether the local file is just a segment/part of the larger file located on the remote. 

Below is an example of overwrite logic performed today by our FTP client using hashes and size comparisons: 

If a user requests to download a file and a file with the same name exists locally, the client will determine if the file sizes are the same or if the destination (local) size is smaller (indicating a possible partially transferred file). If file sizes are the same then the client will compute the hash for the entire file and ask the server for to provide the hash for the corresponding remote file. The client will then skip the transfer if the hashes are identical or overwrite the file if the hashes do not match. If  the remote (source) file is larger the client will ask the server for a partial hash, up to the bytes that match the local (destination) file size. If the partial hash matches then the client will resume the transfer from the byte offset. If the hashes are different then the client will overwrite the file. (e.g. local partial file was corrupted or is not same file).

Robert Oslin
Director of Product Management
Tel: [phone]
Fax: [phone]
Send me large files securely
www.globalscape.com