Re: Draft for Resumable Uploads

[Austin Wright <[e-mail]>]

Hello Marius & WG,

I’m glad to see some activity on this issue; I wrote the draft that you linked to in [1]. See [2] for the most relevant of the documents.

One of the reasons I wrote a new document rather than adopting TUS, is that it appears to be using HTTP as a substrate, rather than extending HTTP as an application unto itself. The draft seems to have made some improvements in this area, but it still relies on some assumptions that I tried to avoid.

Notably, it introduces many new headers that modify the semantics of the method. The  "Upload-Offset” and "Upload-Incomplete” headers must be understood by all the HTTP intermedaries the message passes through. This could confuse caches or gateways.

I avoided this by using the PATCH method whenever possible (to modify an existing resource without replacing it, exactly the semantics we’re looking for). The HTTP server, in all cases, gets to select the URI uses to identify resources.

I sort of gave up on my draft after it received no feedback. And I never liked how resumable requests work (e.g. resuming a POST upload), it seems difficult to implement.

However I came back upon it recently. I think 1xx interim responses are ideal for communicating information about an ongoing upload. And I plan on submitting a "message/byterange” media type for consideration as a PATCH body. While this by itself does not provide discoverability, I believe APIs could at least adopt this as a standard media type for segmented/partial uploads. [3]

Do you have any thoughts on this approach?

Thanks,

Austin.

[1] https://lists.w3.org/Archives/Public/ietf-http-wg/2019JulSep/0066.html <https://lists.w3.org/Archives/Public/ietf-http-wg/2019JulSep/0066.html>
[2] https://tools.ietf.org/id/draft-wright-http-partial-upload-01.html <https://tools.ietf.org/id/draft-wright-http-partial-upload-01.html>
[3] https://github.com/awwright/http-progress/issues/1#issuecomment-1061314730 <https://github.com/awwright/http-progress/issues/1#issuecomment-1061314730> for one possible way to use message/byterange with existing, standard HTTP features.


> On Apr 1, 2022, at 02:48, Marius Kleidl <[e-mail]> wrote:
> 
> Hello HTTP working group,
> 
> we are all familiar with connectivity disruptions affecting our internet activities. One example is when a large file download is interrupted; say a 100 MB file download encounters a network loss after the client receives 70 MB. Fortunately, resumable HTTP downloads using range requests are a widely deployed standard feature that allows clients to fetch the remaining 30 MB only, saving time and resources for both endpoints. However, in the opposite direction, there is not a standard convention for resuming HTTP uploads.
> 
> Across the HTTP ecosystem there are several different approaches to providing resumable uploads. We are aware of at least one attempt to try and standardize an approach [1], but to our knowledge none have succeeded in being adopted and driven to conclusion.
> 
> We believe resumable uploads are a common problem and that there is value in a standard resumable upload approach. We've been working on a document [2] [3] that uses HTTP to solve what we believe to be the core problem set, while also allowing for extended use cases. We are bringing this to the list to understand if there is interest in the working group to solve the problem, and whether our document is a good basis for a solution.
> 
> In case you are interested in the background of this draft: The origin is within the tus project [4], which has been developing a HTTP-based protocol for resumable uploads [5] since 2013 (tus was also posted on this mailing list at the time [6]). Furthermore, we also provide various open-source implementations [7] to allow easy usage on the web, in mobile applications, desktop application, or server environments. tus has seen great adaption, proving that there is a demand for an open-source solution providing resumable uploads.
> 
> We hope to bring resumable uploads to more people. For this, adopting resumable uploads into HTTP would be a great step. There is also interest in including support for resumable uploads natively into platforms, like browsers and mobile SDKs, so that developers do not have to bring their own library for resumable uploads.
> 
> We have taken the main uploading process from our tus protocol and reworked it into a self-containing draft, which we want to present to you! As such, this draft can be seen as an evolution of our work on tus and as a step to increase availability of resumable uploads.
> 
> Thank you for any feedback in advance!
> 
> Best regards,
> Marius Kleidl
> 
> [1] https://lists.w3.org/Archives/Public/ietf-http-wg/2019JulSep/0066.html <https://lists.w3.org/Archives/Public/ietf-http-wg/2019JulSep/0066.html>
> [2] https://datatracker.ietf.org/doc/draft-tus-httpbis-resumable-uploads-protocol/ <https://datatracker.ietf.org/doc/draft-tus-httpbis-resumable-uploads-protocol/>
> [3] https://github.com/tus/tus-v2 <https://github.com/tus/tus-v2>
> [4] https://tus.io/ <https://tus.io/>
> [5] https://tus.io/protocols/resumable-upload.html <https://tus.io/protocols/resumable-upload.html>
> [6] https://mailarchive.ietf.org/arch/msg/httpbisa/I__B5Kc7h-1TvRRB9rmjY8tR-T0/ <https://mailarchive.ietf.org/arch/msg/httpbisa/I__B5Kc7h-1TvRRB9rmjY8tR-T0/>
> [7] https://tus.io/implementations.html <https://tus.io/implementations.html>
>