[Storagesync] recent issues discussed (html)

"Fei Song" <fsong@bjtu.edu.cn> Tue, 08 December 2015 04:46 UTC

Return-Path: <fsong@bjtu.edu.cn>
X-Original-To: storagesync@ietfa.amsl.com
Delivered-To: storagesync@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7B3981A8754 for <storagesync@ietfa.amsl.com>; Mon, 7 Dec 2015 20:46:38 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 4.089
X-Spam-Level: ****
X-Spam-Status: No, score=4.089 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HTML_MESSAGE=0.001, J_CHICKENPOX_62=0.6, RCVD_IN_PSBL=2.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 284Qf-OzKaXi for <storagesync@ietfa.amsl.com>; Mon, 7 Dec 2015 20:46:36 -0800 (PST)
Received: from bjtu.edu.cn (mail.bjtu.edu.cn [218.249.29.198]) by ietfa.amsl.com (Postfix) with ESMTP id 4E69A1A8741 for <storagesync@ietf.org>; Mon, 7 Dec 2015 20:46:33 -0800 (PST)
Received: from PC-201001061KKK (unknown [211.71.74.217]) by Jdweb3 (Coremail) with SMTP id d55wygDHz_s0YWZWacIHAA--.1204S2; Tue, 08 Dec 2015 12:48:53 +0800 (CST)
Date: Tue, 08 Dec 2015 12:46:02 +0800
From: Fei Song <fsong@bjtu.edu.cn>
To: storagesync <storagesync@ietf.org>
References: <CAO_YprZTz+O-e82hsTgMBOLr645jJqbhtVKngubnLhimyfB2cg@mail.gmail.com> <71E522FC-C622-4DDE-B444-5CE902980823@cern.ch> <2015112611280303192311@bjtu.edu.cn>, <CAO_YprbsHsDYewHh38uDJ6gOoSZKwO8MoCR+UHz21iarEqEQOA@mail.gmail.com>
X-Priority: 3
X-Has-Attach: no
X-Mailer: Foxmail 7.0.1.91[cn]
Mime-Version: 1.0
Message-ID: <2015120812455826565698@bjtu.edu.cn>
Content-Type: multipart/alternative; boundary="----=_001_NextPart824166633356_=----"
X-CM-TRANSID: d55wygDHz_s0YWZWacIHAA--.1204S2
X-Coremail-Antispam: 1UD129KBjvJXoW7tFy7WryDJr4xuFWDCF47twb_yoW5Jry3pF yxJws8Kayqq3yaqrWkWF4I9r40qFn3Gw43tFsxGw47Arn0gas29F1IvF4F9FZ7JryUZw1q qr1Yvas8CF1DZaDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmEb7Iv0xC_Kw4lb4IE77IF4wAFF20E14v26r1j6r4UM7CY07I2 0VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28lY4IEw2IIxxk0rw A2z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j 6F4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oV Cq3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40Eb7x2x7xS6r1j6r4UMc02F40E57IF 67AEF4xIwI1l5I8CrVAKz4kIr2xC04v26r1j6r4UMc02F40E42I26xC2a48xMc02F40Ex7 xS67I2xxkvbII20VAFz48EcVAYj21lYx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE 14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwACY4xI67k042 43AVAKzVAKj4xxM4xvF2IEb7IF0Fy26I8I3I1lc2xSY4AK67AK6r45MxAIw28IcxkI7VAK I48JMI8I3I0E5I8CrVAFwI0_JrI_JrWlx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7 AF67AKxVWUJVWUXwCIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE 2Ix0cI8IcVCY1x0267AKxVWUJVW8JwCI42IY6xAIw20EY4v20xvaj40_WFyUJVCq3wCI42 IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r1j6r4UMVCEFcxC0VAY jxAxZFUvcSsGvfC2KfnxnUUI43ZEXa7IU5Wmh7UUUUU==
X-CM-SenderInfo: aytwlqpemw3hxhgxhubq/
Archived-At: <http://mailarchive.ietf.org/arch/msg/storagesync/eVIjsrXTsgyL0IACjO_wmkTHZi4>
Subject: [Storagesync] recent issues discussed (html)
X-BeenThere: storagesync@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: fsong <fsong@bjtu.edu.cn>
List-Id: Mechanisms to synchronize client file systems with Internet-based data storage services <storagesync.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/storagesync>, <mailto:storagesync-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/storagesync/>
List-Post: <mailto:storagesync@ietf.org>
List-Help: <mailto:storagesync-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/storagesync>, <mailto:storagesync-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 08 Dec 2015 04:46:38 -0000

Here is the latest version. Please email me if anything is missing:
 
1.         The design targets of WebDAV, rsync and other existing approaches?
2.         The potential use cases of ISS, such as client/server, git-like pattern, svn, etc.
3.         The efficiency improvements might be the second goal for standardizing ISS protocol
4.         CORS headers on storage sync APIs
5.         What is needed for the ISS: a sync protocol or a generalized API
6.         remoteStorage draft discussion
a)         relationship vs WebDAV
b)         MOVE action (synchronization) should be added or not
c)         Beside web browser, desktop apps (by hacking way)
d)         comics of new standard
e)         etag issues
                         i.              is mainly for identifying whether a document is changed or not
                       ii.              is easy to implement than that of WebDAV sync protocol or not
                      iii.              the metadata file contains all etags for all files at both client and server side or not
f)          the distributed peer model (no server) and C/S mode
g)         a fancy example (with pics) of OfflineIMAP’s sync process in following URL
http://blog.ezyang.com/2012/08/how-offlineimap-works/
7.         GitHub (instead of email messages) has been created:
https://github.com/labkode/Internet-Storage-Sync
a)         What is the topic? 
                         i.              Whether it is suitable to build on WebDAV
                       ii.              WebDAV vs remoteStorage
                      iii.              Advantages vs disadvantages of WebDAV
8.         Metadata and data separation scheme and platform for synchronization
a)         ownCloud when configured to use an Object Storage as the Primary User Storage. (Metadata is handled by a MySQL database)
b)         CERN EOS Storage System. (Metadata is handled in high performance in-memory structures).
c)         DropBox. (As far as I know it uses S3 behind. For metadata it is unknown, but probably not on S3). 
Paper for analyzing DropBox:
http://annasperotto.org/papers/2012/imc140-drago.pdf
d)         ClawIO will have an implementation of this approach in the next phase using Swift.
9.         Whether we should keep metadata history or modification history or action history at server side (or other places)
10.     How to handle the “conflict resolution” issues
a)         A good demonstration of this issue was given in details (File F, Client A, Client B, etc.)
b)         Should it be a layer above syncing
c)         Should it depend on the use case
d)         Should it be a one-size-fits-all approach
 
 
Some related organizations, events, projects, etc.: 
GEANT community, OpenCloudMesh, ownCloud, CS3, remoteStorage, ClawIO, crosscloud, Dropbox, CERN EOS Storage System,to be added…