Re: [Storagesync] Some preliminary investigations on ownCloud

"Fei Song" <fsong@bjtu.edu.cn> Thu, 26 November 2015 03:36 UTC

Return-Path: <fsong@bjtu.edu.cn>
X-Original-To: storagesync@ietfa.amsl.com
Delivered-To: storagesync@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B76C31A9250 for <storagesync@ietfa.amsl.com>; Wed, 25 Nov 2015 19:36:59 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.486
X-Spam-Level:
X-Spam-Status: No, score=-2.486 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_BASE64_BLANKS=0.001, RP_MATCHES_RCVD=-0.585, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id imrc1liXVr_w for <storagesync@ietfa.amsl.com>; Wed, 25 Nov 2015 19:36:56 -0800 (PST)
Received: from bjtu.edu.cn (mail.bjtu.edu.cn [218.249.29.198]) by ietfa.amsl.com (Postfix) with ESMTP id 0D6B71A924B for <storagesync@ietf.org>; Wed, 25 Nov 2015 19:36:55 -0800 (PST)
Received: from PC-201001061KKK (unknown [211.71.74.217]) by Jdweb1 (Coremail) with SMTP id Mp5wygBnSQbbflZWdHHjAA--.16194S2; Thu, 26 Nov 2015 11:39:07 +0800 (CST)
Date: Thu, 26 Nov 2015 11:36:57 +0800
From: Fei Song <fsong@bjtu.edu.cn>
To: Linhui Sun <lh.sunlinh@gmail.com>
References: <CAO_YprZTz+O-e82hsTgMBOLr645jJqbhtVKngubnLhimyfB2cg@mail.gmail.com> <201511261111483122529@bjtu.edu.cn>, <CAO_YpraNavS0S-K6wSM-ijjtBDCSELxSnFhkYqS01WET4gjBSA@mail.gmail.com>
X-Priority: 3
X-Has-Attach: no
X-Mailer: Foxmail 7.0.1.91[cn]
Mime-Version: 1.0
Message-ID: <2015112611365765643513@bjtu.edu.cn>
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
X-CM-TRANSID: Mp5wygBnSQbbflZWdHHjAA--.16194S2
X-Coremail-Antispam: 1UD129KBjvJXoWxAw4fAw4xArWfKFWkWrWxZwb_yoWrtF47pr ZIkas7KF1kJrWxCwn2qFyjkr1F9rZ3JrW7Xrn8Jw4xArZ09ryftF4Iyr1ruasrCrZ3Gr1j qr4YgrW3Xwn8AaDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUv0b7Iv0xC_tr1lb4IE77IF4wAFF20E14v26r1j6r4UM7CY07I2 0VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28lY4IEw2IIxxk0rw A2z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j 6F4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oV Cq3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0 I7IYx2IY67AKxVWUGVWUXwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r 4UM4x0Y48IcxkI7VAKI48JM4xvF2IEb7IF0Fy264kE64k0F24lFcxC0VAYjxAxZF0Ex2Iq xwCY02Avz4vE14v_Gr4l42xK82IYc2Ij64vIr41lx2IqxVAqx4xG67AKxVWUJVWUGwC20s 026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1Y6r17MIIYrxkI7VAKI48JMIIF 0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6xkF7I0E14v26r1j6r4UMIIF0x vE42xK8VAvwI8IcIk0rVWrJr0_WFyUJwCI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E 87Iv6xkF7I0E14v26r1j6r4UMVCEFcxC0VAYjxAxZFUvcSsGvfC2KfnxnUUI43ZEXa7IU8 EJm5UUUUU==
X-CM-SenderInfo: aytwlqpemw3hxhgxhubq/
Archived-At: <http://mailarchive.ietf.org/arch/msg/storagesync/YsVYrxxNYRubM8ZQN79b-27K2z4>
Cc: storagesync <storagesync@ietf.org>
Subject: Re: [Storagesync] Some preliminary investigations on ownCloud
X-BeenThere: storagesync@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: fsong <fsong@bjtu.edu.cn>
List-Id: Mechanisms to synchronize client file systems with Internet-based data storage services <storagesync.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/storagesync>, <mailto:storagesync-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/storagesync/>
List-Post: <mailto:storagesync@ietf.org>
List-Help: <mailto:storagesync-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/storagesync>, <mailto:storagesync-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 26 Nov 2015 03:36:59 -0000

Hi Linhui,


--------------
Fei Song
>Hi Fei,
>
>2015-11-26 11:11 GMT+08:00 Fei Song <fsong@bjtu.edu.cn>:
>
>> Hi all
>>
>>
>> --------------
>> Fei Song
>> >Hi all,
>> >
>> >As I mentioned before, I think the developers could benefit from the IETF
>> >standards. The ownCloud (https://owncloud.org/) is just an example. It is
>> >developed for those who do not trust commercial storage services and want
>> >to build their own network-based storage services. The ownCloud is using
>> >WebDAV (RFC4918) to achieve the data sync. IMO, the WebDAV is designed for
>> >distributed work but not for the sync. Thus, I made some preliminary
>> >investigations on how the ownCloud uses WebDAV for sync purposes. A brief
>> >summary of what I've found is in the following, please correct me if I am
>> >wrong.
>> >
>> >I installed the ownCloud server (v8.2.1) on the CentOS7, and the client is
>> >a desktop client on Windows.
>> >
>> >1. To find whether there is a change to the synced directory, the client
>> >continuously sends PROPFIND to the server at regular intervals (around 34
>> >seconds under my observation). The server will respond a 207 Multi-Status
>> >Response to tell whether the main directory has been changed. To perform
>> >this regular check, the client will open a new TCP connection to send the
>> >PROPFIND, the server will close the existing TCP connection after
>> >responding the 207 Multi-Status Response. For the next check, the client
>> >will open another new TCP connection.
>>
>> I wonder whether these message you got had been encrypt or not.
>>
>The ownCloud suggests people use HTTPS, but you could also use the HTTP.
>And that is what I did.

Thanks!

>
>>
>> >
>> >2. Every time adding (or creating) a new file to the local folder, the
>> >client will open a new TCP connection (if there is no connection existing)
>> >to send the file asap. The client will first send several PROPFINDs to
>> find
>> >out which sub-directory has been changed. And then it sends the file using
>> >PUT. The server will respond a 201 Created Response and then terminate the
>> >connection. Currently, I haven't found any application layer chunking, all
>> >the segmentation are performed by TCP.
>> >
>> >3. Every time I delete (or rename) a file locally, the client will also
>> >open a new TCP connection to send several PROPFINDs to find out which file
>> >has been removed (or renamed). Then it will send DELETE (or MOVE). The
>> >server will respond a 204 No Content Response (or 201 Created Response)
>> and
>> >then terminate the connection.
>> >
>> >4. I open a file and frequently edit and save it (actually this is what I
>> >usually do with the Dropbox). The client will send the whole file to the
>> >server every time I save the file.
>> >
>> >To summarize, it seems that the ownCloud makes heavily use of PROPFIND to
>> >achieve the sync process. Each sync operation (e.g. upload, modify and
>> >etc.) will start with sending one or more PROPFINDs. And currently, if I
>> >add a file to the server (directly from the server side via web
>> interface),
>> >the client cannot find the change. I need to interrupt the sync and
>> recover
>> >it to make the client be aware of the change and download the newly added
>> >file. I'm not sure whether this is caused by the sync mechanism or an
>> >improper server configuration. I need to investigate this further and also
>> >how the ownCloud works for multiple clients (or devices).
>>
>> What do you mean by "directly from the server side via web interface".
>> If all the configuration were correct, do you want to identify that the
>> sync in
>> ownCloud is only unidirectional?
>>
>People could access to the ownCloud service via desktop, mobile client or
>web interface (i.e. browser), the point I want to say here is an operation
>on the server side cannot be detected automatically. I'm not sure whether
>I've configured the server correctly. But the sync is not unidirectional
>since after I recovering the sync, the client could download the asap.

I see, then there might be an option at the server side 
can sync the changes to the client automatically.


>
>>
>> >
>> >For ISS, I think ownCloud has demonstrated to some extent that similar
>> IETF
>> >protocols could be deployed and employed. The intention of this message is
>> >to investigate the current state of using WebDAV for sync purposes to see
>> >what needs to be improved here and whether we need new protocols.
>>
>>
>> >
>> >Comments are welcome : )
>> >
>> >Regards,
>> >Linhui
>> >
>> _______________________________________________
>> Storagesync mailing list
>> Storagesync@ietf.org
>> https://www.ietf.org/mailman/listinfo/storagesync
>>
>