Re: [Storagesync] Some preliminary investigations on ownCloud

Linhui Sun <lh.sunlinh@gmail.com> Thu, 26 November 2015 03:29 UTC

Return-Path: <lh.sunlinh@gmail.com>
X-Original-To: storagesync@ietfa.amsl.com
Delivered-To: storagesync@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E1DA1A9174 for <storagesync@ietfa.amsl.com>; Wed, 25 Nov 2015 19:29:09 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level:
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tiKvGvf844oi for <storagesync@ietfa.amsl.com>; Wed, 25 Nov 2015 19:29:06 -0800 (PST)
Received: from mail-wm0-x22e.google.com (mail-wm0-x22e.google.com [IPv6:2a00:1450:400c:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B6E021A9173 for <storagesync@ietf.org>; Wed, 25 Nov 2015 19:29:05 -0800 (PST)
Received: by wmec201 with SMTP id c201so5708142wme.1 for <storagesync@ietf.org>; Wed, 25 Nov 2015 19:29:04 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=o3AIzRfL0B9Uq6aWHxEebd6WJo6nMP28ZrR+Slgb2Mg=; b=I/vtkb5LbkrQ/IngpbeO0SMfQIPPa7G2clqIknDqkRx4kSJmK2zeidSFqHM0vONZn1 bL8mRZ04Zzd6hduaa6uQawQYmOSHKhuzDD/MUWx2wb2ttfUfsGOr/GcJpJWL1ER2nUCe hd1sxYLsZEw4M8y+eRAggEOtLcVaMgOFGfrT/+FerkRK+uPbft7m+rWM+GqLliMcv4Qc +14DpHfWHsCxVRXfdkazxSz636QvwMhhSNydF1pIzCP45lSzBVc0Klnm80JpfDole/R/ MQmp6Bkl8L14DhhEpaxmiVAY3iFWPFVnd3j760gW8YtfgeCJUEWqUMWZnGEfpZxzG1kb CAAw==
X-Received: by 10.28.15.194 with SMTP id 185mr1075656wmp.9.1448508544342; Wed, 25 Nov 2015 19:29:04 -0800 (PST)
MIME-Version: 1.0
Received: by 10.28.27.147 with HTTP; Wed, 25 Nov 2015 19:28:44 -0800 (PST)
In-Reply-To: <2015112611202500034610@bjtu.edu.cn>
References: <CAO_YprZTz+O-e82hsTgMBOLr645jJqbhtVKngubnLhimyfB2cg@mail.gmail.com> <71E522FC-C622-4DDE-B444-5CE902980823@cern.ch> <2015112611202500034610@bjtu.edu.cn>
From: Linhui Sun <lh.sunlinh@gmail.com>
Date: Thu, 26 Nov 2015 11:28:44 +0800
Message-ID: <CAO_YprbwDzxrXBbcOMeOr1=Negr5cSzJEiWzNO7Xy7eP3bpchw@mail.gmail.com>
To: fsong <fsong@bjtu.edu.cn>
Content-Type: multipart/alternative; boundary="001a11469edc3d623b0525692d47"
Archived-At: <http://mailarchive.ietf.org/arch/msg/storagesync/wipxGUlWsKs1lvRp9fLHkWdnkTc>
Cc: Jakub Moscicki <Jakub.Moscicki@cern.ch>, storagesync <storagesync@ietf.org>
Subject: Re: [Storagesync] Some preliminary investigations on ownCloud
X-BeenThere: storagesync@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Mechanisms to synchronize client file systems with Internet-based data storage services <storagesync.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/storagesync>, <mailto:storagesync-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/storagesync/>
List-Post: <mailto:storagesync@ietf.org>
List-Help: <mailto:storagesync-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/storagesync>, <mailto:storagesync-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 26 Nov 2015 03:29:09 -0000

2015-11-26 11:20 GMT+08:00 Fei Song <fsong@bjtu.edu.cn>:

> Hi Jakub and all,
>
> I can't download the paper you mentioned. Could you please show me a link
> to it?
> For the URL http://cs3.ethz.ch/program.html ,there is no links...
>
+1

Regards,
Linhui

>
>
> --------------
> Fei Song
> >Hello,
> >
> >What kind of outcome are you looking for with this analysis? Some
> research in this area has already been done or is being done as we speak
> >
> >e.g. "A study of delta-sync and other optimisation in HTTP/Webdav
> synchonisation protocols"
> >
> >see "Technology and Research":
> >
> >http://cs3.ethz.ch/program.html
> >
> >It would be interesting to see if there is a potential for collaboration.
> Or maybe we already have some information you are looking for.
> >
> >Best regards,
> >
> >Jakub Moscicki
> >
> >—
> >
> >
> >On 25 Nov 2015, at 11:45, Linhui Sun <lh.sunlinh@gmail.com<mailto:
> lh.sunlinh@gmail.com>> wrote:
> >
> >Hi all,
> >
> >As I mentioned before, I think the developers could benefit from the IETF
> standards. The ownCloud (https://owncloud.org/) is just an example. It is
> developed for those who do not trust commercial storage services and want
> to build their own network-based storage services. The ownCloud is using
> WebDAV (RFC4918) to achieve the data sync. IMO, the WebDAV is designed for
> distributed work but not for the sync. Thus, I made some preliminary
> investigations on how the ownCloud uses WebDAV for sync purposes. A brief
> summary of what I've found is in the following, please correct me if I am
> wrong.
> >
> >I installed the ownCloud server (v8.2.1) on the CentOS7, and the client
> is a desktop client on Windows.
> >
> >1. To find whether there is a change to the synced directory, the client
> continuously sends PROPFIND to the server at regular intervals (around 34
> seconds under my observation). The server will respond a 207 Multi-Status
> Response to tell whether the main directory has been changed. To perform
> this regular check, the client will open a new TCP connection to send the
> PROPFIND, the server will close the existing TCP connection after
> responding the 207 Multi-Status Response. For the next check, the client
> will open another new TCP connection.
> >
> >2. Every time adding (or creating) a new file to the local folder, the
> client will open a new TCP connection (if there is no connection existing)
> to send the file asap. The client will first send several PROPFINDs to find
> out which sub-directory has been changed. And then it sends the file using
> PUT. The server will respond a 201 Created Response and then terminate the
> connection. Currently, I haven't found any application layer chunking, all
> the segmentation are performed by TCP.
> >
> >3. Every time I delete (or rename) a file locally, the client will also
> open a new TCP connection to send several PROPFINDs to find out which file
> has been removed (or renamed). Then it will send DELETE (or MOVE). The
> server will respond a 204 No Content Response (or 201 Created Response) and
> then terminate the connection.
> >
> >4. I open a file and frequently edit and save it (actually this is what I
> usually do with the Dropbox). The client will send the whole file to the
> server every time I save the file.
> >
> >To summarize, it seems that the ownCloud makes heavily use of PROPFIND to
> achieve the sync process. Each sync operation (e.g. upload, modify and
> etc.) will start with sending one or more PROPFINDs. And currently, if I
> add a file to the server (directly from the server side via web interface),
> the client cannot find the change. I need to interrupt the sync and recover
> it to make the client be aware of the change and download the newly added
> file. I'm not sure whether this is caused by the sync mechanism or an
> improper server configuration. I need to investigate this further and also
> how the ownCloud works for multiple clients (or devices).
> >
> >For ISS, I think ownCloud has demonstrated to some extent that similar
> IETF protocols could be deployed and employed. The intention of this
> message is to investigate the current state of using WebDAV for sync
> purposes to see what needs to be improved here and whether we need new
> protocols.
> >
> >Comments are welcome : )
> >
> >Regards,
> >Linhui
> >
> >
> >_______________________________________________
> >Storagesync mailing list
> >Storagesync@ietf.org<mailto:Storagesync@ietf.org>
> >https://www.ietf.org/mailman/listinfo/storagesync
> >