Re: [Storagesync] Some preliminary investigations on ownCloud

[Jakub Moscicki <[e-mail]>]

Hello,

What kind of outcome are you looking for with this analysis? Some research in this area has already been done or is being done as we speak

e.g. "A study of delta-sync and other optimisation in HTTP/Webdav synchonisation protocols"

see "Technology and Research":

http://cs3.ethz.ch/program.html

It would be interesting to see if there is a potential for collaboration. Or maybe we already have some information you are looking for.

Best regards,

Jakub Moscicki

—


On 25 Nov 2015, at 11:45, Linhui Sun <[e-mail]<mailto:[e-mail]>> wrote:

Hi all,

As I mentioned before, I think the developers could benefit from the IETF standards. The ownCloud (https://owncloud.org/) is just an example. It is developed for those who do not trust commercial storage services and want to build their own network-based storage services. The ownCloud is using WebDAV (RFC4918) to achieve the data sync. IMO, the WebDAV is designed for distributed work but not for the sync. Thus, I made some preliminary investigations on how the ownCloud uses WebDAV for sync purposes. A brief summary of what I've found is in the following, please correct me if I am wrong.

I installed the ownCloud server (v8.2.1) on the CentOS7, and the client is a desktop client on Windows.

1. To find whether there is a change to the synced directory, the client continuously sends PROPFIND to the server at regular intervals (around 34 seconds under my observation). The server will respond a 207 Multi-Status Response to tell whether the main directory has been changed. To perform this regular check, the client will open a new TCP connection to send the PROPFIND, the server will close the existing TCP connection after responding the 207 Multi-Status Response. For the next check, the client will open another new TCP connection.

2. Every time adding (or creating) a new file to the local folder, the client will open a new TCP connection (if there is no connection existing) to send the file asap. The client will first send several PROPFINDs to find out which sub-directory has been changed. And then it sends the file using PUT. The server will respond a 201 Created Response and then terminate the connection. Currently, I haven't found any application layer chunking, all the segmentation are performed by TCP.

3. Every time I delete (or rename) a file locally, the client will also open a new TCP connection to send several PROPFINDs to find out which file has been removed (or renamed). Then it will send DELETE (or MOVE). The server will respond a 204 No Content Response (or 201 Created Response) and then terminate the connection.

4. I open a file and frequently edit and save it (actually this is what I usually do with the Dropbox). The client will send the whole file to the server every time I save the file.

To summarize, it seems that the ownCloud makes heavily use of PROPFIND to achieve the sync process. Each sync operation (e.g. upload, modify and etc.) will start with sending one or more PROPFINDs. And currently, if I add a file to the server (directly from the server side via web interface), the client cannot find the change. I need to interrupt the sync and recover it to make the client be aware of the change and download the newly added file. I'm not sure whether this is caused by the sync mechanism or an improper server configuration. I need to investigate this further and also how the ownCloud works for multiple clients (or devices).

For ISS, I think ownCloud has demonstrated to some extent that similar IETF protocols could be deployed and employed. The intention of this message is to investigate the current state of using WebDAV for sync purposes to see what needs to be improved here and whether we need new protocols.

Comments are welcome : )

Regards,
Linhui


_______________________________________________
Storagesync mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/storagesync