Re: [Storagesync] Storagesync Digest, Vol 5, Issue 1

[Linhui Sun <[e-mail]>]

2015-12-02 20:43 GMT+08:00 Hugo González Labrador <[e-mail]>:

>
>
>
> On Wed, Dec 2, 2015, at 01:30 PM, Michiel de Jong wrote:
>
> Hi all!
> Thanks for all your reactions to the remoteStorage Internet-Draft.
>
> We get the question about WebDAV a lot, in the next version we should add
> a remark about it in the intro. The folder descriptions returned when you
> GET a URL that ends with a / are indeed a deviation from the XML returned
> by WebDAV server, and this is just because nowadays JSON is easier to use
> than XML for developers, both client-side and server-side.
>
>
> I totally agree here, this was going to happen soon or later and it is
> really appreciated.
>
>
> The fact that we don't require servers to support WebDAV's custom verbs
> like PROPPATCH etc. is for three reasons:
> 1) it's a lot of work to implement this without using an existing WebDAV
> library
> 2) in practice, a lot of WebDAV servers get it wrong, or don't implement
> all of WebDAV. It's very annoying for client implementers to have to deal
> with servers that e.g. chose not to implement LOCK and UNLOCK.
> 3) we don't really need all these advanced features on top of standard
> HTTP, just supporting GET/PUT/DELETE for resources, and adding a simple
> folder description format, is enough for most applications.
>
> Other than that, the remoteStorage Internet-Draft specifies a *lot* more
> than just how each HTTP verb should behave:
> * requiring support for OAuth implicit-grant flow
> * requiring ETag support and nested versioning (i.e. the folder's ETag
> changes if anything within that folder changes)
> * requiring CORS headers
> * requiring a WebFinger announcement for service discovery
> It would be easy to add these three things on top of WebDAV, instead of
> putting them on top of our minimal GET/PUT/DELETE API definition. In fact,
> we could probably separate it into two Internet-Drafts: one for the
> 'RESTful folders' API which is our simplification of WebDAV, and a second
> one for OAuth/ETags/CORS/WebFinger on top of either WebDAV or 'RESTful
> folders' or whatever other HTTP-based API you want.
>
>
> There is one requirement that all synchronisers need to handle: moving
> resources. In this spec the alternative of a WebDAV MOVE is not specified.
>
> It is true that a MOVE could be replaced with a DELETE + UPLOAD but that
> is not acceptable in most cases due to the inefficiency of such operation
> (cpu, bandwidth consumption ...)
>
> Is there a plan to support such basic feature?
>
> From the current remoteStorage spec, the ownCloud sync protocol can be
> implemented. The missing bit is tracking those remote moves.
>
I agree with Hugo that MOVE is useful, sometimes if you just rename a file
it will be perfect. But the question I have is that whether we need to make
two documents? Multiple choices is not good for standardization. In my
view, webdav is something that we need to have a very clear decision on
whether to consider it or not.

Regards,
Linhui

>
> Cheers
>
>
> On Wed, Dec 2, 2015 at 11:28 AM, Hugo González Labrador <
> [e-mail]> wrote:
>
>
>
>
>
>
> On Wed, Dec 2, 2015, at 11:18 AM, Linhui Sun wrote:
>
> Hi
>
> On 周三, 12月 2, 2015 at 17:56, Hugo González Labrador <[e-mail]>
> wrote:
>
>
>
>
> On Wed, Dec 2, 2015, at 08:20 AM, Linhui Sun wrote:
>
> Hi
>
> 2015-12-02 5:14 GMT+08:00 Hugo González Labrador <[e-mail]>:
>
> Hi,
>
> from my point of view the remoteStorage project addresses a subset of
> the use cases of the  WebDAV specification.
>
> The main difference I can observe is that the specification is built on
> REST instead of XML-based communication.
>
>
> I personally like much more REST than WebDAV because it is more
> developer friendly and it is faster to develop.
>  Maybe the remoteStorage API becomes the next WebDAV :)
>
> However, the remoteStorage API does not provide a way of synchronising
> data, this task is delegated to the developers.
> Is there a plan to provide such feature based on the outcome of this
> draft?
>
> I'm a little bit confused why you say the remoteStorage does not provide
> that. Is that because remoteStorage does not perform like a typical sync
> services (e.g. dropbox...) or you are saying something else?
>
> Yes, because it does not offer synchronisation capabilities.
>
> Got it. And What I am wondering is that do we need to include those
> capabilities in a base specifications. Since it is hard to standardize the
> capabilities like dedup or delta. Maybe a better way is to define those in
> a separate specification,
>
>
>
>
> Thanks for giving these examples - so by 'synchronisation capabilities'
> you mean 1) deduplication and 2) delta updates? Anything else or is that an
> exhaustive list?
>
>
>
> something like extensions. For a base document, we just need to define how
> to perform sync operations.
>
>
>
> Yes, I agree that may be an extension of this draft could handle the
> synchronisation part.
>
>
>
>
>
> Our Internet-Draft is heavily focused on the world wide web, whose URLs
> are not content-addressable, we can't change that. So that architecture is
> not very friendly to deduplication, compared to for instance BitTorrent. As
> you already said, developers can still dedupe on the server-side when
> storing blobs to disk, and can also dedupe on the client side before
> creating the resources the client uploads.
> As far as I know, delta updates are not supported by the ETag system - you
> cannot do a range request to find out if certain bytes within a document
> have changed. However, the folder system we define does encourage you, for
> instance when you develop a To Do List app, put each task on its own
> document, and then query the folder to see which of them changed, instead
> of putting them all in one big document and retrieving the whole document
> each time.
>
> Cheers,
> Michiel.
>
>
>
>
>
>
>
> BTW, I want to introduce ClawIO ( <http://clawio.github.io>
> http://clawio.github.io), a research
> project to benchmark different synchronisation protocols against
> different data backends with special attention to provide a common sync
> API.
>
> A common API for different sync protocols is being created based on the
> architecture specified in this draft (control and data servers) and the
>
> I cannot find a distributed architecture in this draft. It seems that they
> handle metadata and content data together, just like a normal web service.
>
>
> ClawIO is fully distributed. Every logical unit is a different server than
> be scaled out. Data and Metadata channels are independent from each other
> and reside on different servers.
>
> That is widely employed in popular sync services. And that is also
> beneficial to privacy to some extent. But in the context of sync in browser
> (which is mainly considered in the remoteStorage), I don't know whether
> this is reasonable. But I think we should deploy distributed architecture
> although it will make things complicated.
>
>
>
> Of course, the remoteStorage is targeted to browsers, so syncing does not
> make too much sense in this case.
> With the rise of Linux container micro-service based architectures, the
> deployment of  such highly complex systems should become easier and faster.
>
> Best,
>
>
> Hugo
>
>
>
> Regards,
> Linhui
>
>
>
>
> Regards,
> Linhui
>
> one from the CERN EOS project (management, disk and queue servers).
>
> The Phase I has implemented the ownCloud Sync Protocol and Phase II will
> implement the SeaFile Sync Protocol.
> The choice of these protocols among others is because they are really
> opposed to each other in terms of syncing (delta vs non-delta,
> state-based vs log/event/git-based sync …), so finding a common approach
> is more challenging.
>
> Providing a base specification/architecture to measure the feasibility
> of this draft is one of the objectives of the project.
>
> I believe that the work being done here and in ClawIO are supplementary
> to each other and I think mutual collaboration could be beneficial for
> both sides.
>
> Also, if there is interest, the remoteStorage API can be added to
> ClawIO.
>
> Best regards,
>
> Hugo Gonzalez Labrador
>
> On Tue, Dec 1, 2015, at 09:00 PM, [e-mail] wrote:
> > Send Storagesync mailing list submissions to
> >       [e-mail]
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >        <https://www.ietf.org/mailman/listinfo/storagesync>
> https://www.ietf.org/mailman/listinfo/storagesync
> > or, via email, send a message with subject or body 'help' to
> >       [e-mail]
> >
> > You can reach the person managing the list at
> >       [e-mail]
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of Storagesync digest..."
> > Today's Topics:
> >
> >    1. New version of draft-dejong-remotestorage    Internet-Draft
> >       available (Michiel de Jong)
> >    2. Re: New version of draft-dejong-remotestorage Internet-Draft
> >       available (Gihan Dias)
> >    3. Re: New version of draft-dejong-remotestorage Internet-Draft
> >       available (Fei Song)
> > _______________________________________________
> > Storagesync mailing list
> > [e-mail]
> > <https://www.ietf.org/mailman/listinfo/storagesync>
> https://www.ietf.org/mailman/listinfo/storagesync
> > Email had 3 attachments:
> > + [Storagesync] New version of draft-dejong-remotestorage
> > Internet-Draft available
> >   2k (message/rfc822)
> > + Re: [Storagesync] New version of draft-dejong-remotestorage
> > Internet-Draft available
> >   1k (message/rfc822)
> > + Re: [Storagesync] New version of draft-dejong-remotestorage
> > Internet-Draft available
> >   2k (message/rfc822)
>
> _______________________________________________
> Storagesync mailing list
> [e-mail]
> <https://www.ietf.org/mailman/listinfo/storagesync>
> https://www.ietf.org/mailman/listinfo/storagesync
>
>
>
>
>
>
>