[Storagesync] recent issues discussed

["Fei Song" <[e-mail]>]

I add this into "recent issues discussed". Here is the latest version:
 
1. The design targets of WebDAV, rsync and other existing approaches?
2. The potential use cases of ISS, such as client/server, git-like pattern, svn, etc.
3. The efficiency improvements might be the second goal for standardizing ISS protocol
4. CORS headers on storage sync APIs
5. What is needed for the ISS: a sync protocol or a generalized API
6. Some related organizations, events, projects, etc.: GEANT community, OpenCloudMesh, ownCloud, CS3, remoteStorage, ClawIO



--------------
Fei Song
>2015-11-26 11:28 GMT+08:00 Fei Song <[e-mail]>:
>
>> BTW, Based on the last sentence of last email:"The intention of this
>> message is to investigate the current state of using WebDAV for sync
>> purposes to see what needs to be improved here and whether we need new
>> protocols"
>>
>> The outcome he/she wanted might be just the links like
>> http://cs3.ethz.ch/program.html :)
>>
>In another word, I think what I want finally might be a discussion about
>"What is needed for the ISS: a sync protocol or a generalized API". Sorry
>for the poor expression : )
>
>Regards,
>Linhui
>
>>
>>
>> --------------
>> Fei Song
>> >Hello,
>> >
>> >What kind of outcome are you looking for with this analysis? Some
>> research in this area has already been done or is being done as we speak
>> >
>> >e.g. "A study of delta-sync and other optimisation in HTTP/Webdav
>> synchonisation protocols"
>> >
>> >see "Technology and Research":
>> >
>> >http://cs3.ethz.ch/program.html
>> >
>> >It would be interesting to see if there is a potential for collaboration.
>> Or maybe we already have some information you are looking for.
>> >
>> >Best regards,
>> >
>> >Jakub Moscicki
>> >
>> >―
>> >
>> >
>> >On 25 Nov 2015, at 11:45, Linhui Sun <[e-mail]<mailto:
>> [e-mail]>> wrote:
>> >
>> >Hi all,
>> >
>> >As I mentioned before, I think the developers could benefit from the IETF
>> standards. The ownCloud (https://owncloud.org/) is just an example. It is
>> developed for those who do not trust commercial storage services and want
>> to build their own network-based storage services. The ownCloud is using
>> WebDAV (RFC4918) to achieve the data sync. IMO, the WebDAV is designed for
>> distributed work but not for the sync. Thus, I made some preliminary
>> investigations on how the ownCloud uses WebDAV for sync purposes. A brief
>> summary of what I've found is in the following, please correct me if I am
>> wrong.
>> >
>> >I installed the ownCloud server (v8.2.1) on the CentOS7, and the client
>> is a desktop client on Windows.
>> >
>> >1. To find whether there is a change to the synced directory, the client
>> continuously sends PROPFIND to the server at regular intervals (around 34
>> seconds under my observation). The server will respond a 207 Multi-Status
>> Response to tell whether the main directory has been changed. To perform
>> this regular check, the client will open a new TCP connection to send the
>> PROPFIND, the server will close the existing TCP connection after
>> responding the 207 Multi-Status Response. For the next check, the client
>> will open another new TCP connection.
>> >
>> >2. Every time adding (or creating) a new file to the local folder, the
>> client will open a new TCP connection (if there is no connection existing)
>> to send the file asap. The client will first send several PROPFINDs to find
>> out which sub-directory has been changed. And then it sends the file using
>> PUT. The server will respond a 201 Created Response and then terminate the
>> connection. Currently, I haven't found any application layer chunking, all
>> the segmentation are performed by TCP.
>> >
>> >3. Every time I delete (or rename) a file locally, the client will also
>> open a new TCP connection to send several PROPFINDs to find out which file
>> has been removed (or renamed). Then it will send DELETE (or MOVE). The
>> server will respond a 204 No Content Response (or 201 Created Response) and
>> then terminate the connection.
>> >
>> >4. I open a file and frequently edit and save it (actually this is what I
>> usually do with the Dropbox). The client will send the whole file to the
>> server every time I save the file.
>> >
>> >To summarize, it seems that the ownCloud makes heavily use of PROPFIND to
>> achieve the sync process. Each sync operation (e.g. upload, modify and
>> etc.) will start with sending one or more PROPFINDs. And currently, if I
>> add a file to the server (directly from the server side via web interface),
>> the client cannot find the change. I need to interrupt the sync and recover
>> it to make the client be aware of the change and download the newly added
>> file. I'm not sure whether this is caused by the sync mechanism or an
>> improper server configuration. I need to investigate this further and also
>> how the ownCloud works for multiple clients (or devices).
>> >
>> >For ISS, I think ownCloud has demonstrated to some extent that similar
>> IETF protocols could be deployed and employed. The intention of this
>> message is to investigate the current state of using WebDAV for sync
>> purposes to see what needs to be improved here and whether we need new
>> protocols.
>> >
>> >Comments are welcome : )
>> >
>> >Regards,
>> >Linhui
>> >
>> >
>> >_______________________________________________
>> >Storagesync mailing list
>> >[e-mail]<mailto:[e-mail]>
>> >https://www.ietf.org/mailman/listinfo/storagesync
>> >
>> _______________________________________________
>> Storagesync mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/storagesync
>>
>