Re: [ftpext] "Distributed Transfer Support for FTP"

Damin Daster <damin@emuxperts.net> Fri, 12 November 2010 22:11 UTC

Return-Path: <damin@emuxperts.net>
X-Original-To: ftpext@core3.amsl.com
Delivered-To: ftpext@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A36613A6B5F for <ftpext@core3.amsl.com>; Fri, 12 Nov 2010 14:11:08 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.437
X-Spam-Level:
X-Spam-Status: No, score=-0.437 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FH_RELAY_NODNS=1.451, HELO_MISMATCH_NET=0.611, RDNS_NONE=0.1]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bK4rKD3nAU1v for <ftpext@core3.amsl.com>; Fri, 12 Nov 2010 14:11:00 -0800 (PST)
Received: from ca0.emuxperts.net (unknown [96.45.180.106]) by core3.amsl.com (Postfix) with ESMTP id C11B33A6B5D for <ftpext@ietf.org>; Fri, 12 Nov 2010 14:10:58 -0800 (PST)
Received: from localhost.localdomain (localhost [127.0.0.1]) by ca0.emuxperts.net (Postfix) with ESMTPA id 3E5F71FC070 for <ftpext@ietf.org>; Fri, 12 Nov 2010 22:11:32 +0000 (GMT)
Message-ID: <4CDD9161.70709@emuxperts.net>
Date: Fri, 12 Nov 2010 14:11:29 -0500
From: Damin Daster <damin@emuxperts.net>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.12) Gecko/20101027 Fedora/3.1.6-1.fc13 Thunderbird/3.1.6
MIME-Version: 1.0
To: ftpext@ietf.org
References: <alpine.DEB.2.00.1011101009580.17851@tvnag.unkk.fr> <039101cb8289$3d6041c0$b820c540$@com>
In-Reply-To: <039101cb8289$3d6041c0$b820c540$@com>
Content-Type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-Transfer-Encoding: 8bit
Subject: Re: [ftpext] "Distributed Transfer Support for FTP"
X-BeenThere: ftpext@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <ftpext.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/ftpext>, <mailto:ftpext-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ftpext>
List-Post: <mailto:ftpext@ietf.org>
List-Help: <mailto:ftpext-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ftpext>, <mailto:ftpext-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 12 Nov 2010 22:11:08 -0000

I might as well introduce this now;

This draft is meant to extend FTP in a way that allows for a distributed 
server. That is, a set of servers which act together as a single FTP site.

Distributed FTP Servers have many great benefits:
Disk Sharing (one "giant" filesystem)
Bandwidth Aggregation
Load-Balancing
Geo-Balancing (something similar to CDN?)
High-Availability for FTP Site (multi-mastered)
High-Availability for Files (one file on multiple slaves)

Here´s the basic idea...
A Distributed FTP site is a group of servers; masters and slaves
Masters hold a List of files. (and know where each file is)
Slaves transfer and store files.

If a client wants to download a file;
It connects to a master.
It tells the master it wants to download a file...
The master determines what slave has that file and instructs that slave 
and the client to connect to each other (depends on PASV or PORT)
The slave then sends the file DIRECTLY to the client.

Why this isn´t possible now;
FTP initiates a connection before we know what will be transferred 
through it.
If the master doesn´t know what the user is going to ask for, how would 
it know who it should instruct to connect?

The solution;
The client should tell the master what it wants before it asks to connect.
The master can then make an educated choice on who should connect.

Compatibility;
The proposal is very unobtrusive, backwards compatible, and forwards 
compatible for the most part.

Of course let me know if you see any problems or have any questions.

On 11/12/2010 11:47 AM, Alun Jones wrote:
> This seems a little related to the HOST extension.
>
> For those situations where the HOST extension would not apply, I think it's
> more appropriate (and better backward compatibility) for clients to CWD
> before transfer (so as to avoid paths in the transfer command), and for the
> server to ensure that different back-end services are hosted in different
> [virtual] subdirectories from one another.
>
> Perhaps I don't understand the nature of the problem, or the solutions that
> already exist.
>
> Alun.
> ~~~~
>
>> -----Original Message-----
>> From: ftpext-bounces@ietf.org [mailto:ftpext-bounces@ietf.org] On
>> Behalf Of Daniel Stenberg
>> Sent: Wednesday, November 10, 2010 1:11 AM
>> To: ftpext@ietf.org
>> Subject: [ftpext] "Distributed Transfer Support for FTP"
>>
>> Hi
>>
>> I just noticed this and thought people in this group might be
>> interested:
>>
>> Abstract
>>            This document specifies new File Transfer Protocol (FTP)
>>            commands to allow for a distributed FTP server to operate
>>            correctly. The purpose is to address an issue in the FTP
>>            Protocol which causes a truely distributed FTP server from
>>            being possible.
>>
>>   	http://tools.ietf.org/html/draft-dd-pret-00
>> --
>>
>>    / daniel.haxx.se
>> _______________________________________________
>> ftpext mailing list
>> ftpext@ietf.org
>> https://www.ietf.org/mailman/listinfo/ftpext
> _______________________________________________
> ftpext mailing list
> ftpext@ietf.org
> https://www.ietf.org/mailman/listinfo/ftpext