Re: Draft v1 Update for Resumable Uploads

[Glenn Strauss <[e-mail]>]

On Mon, Jun 20, 2022 at 03:41:02AM +0100, Lucas Pardue wrote:
> 
> On Mon, Jun 20, 2022 at 3:03 AM Glenn Strauss <
> [e-mail]> wrote:
> 
> > On Sun, Jun 19, 2022 at 04:52:58PM +0100, Lucas Pardue wrote:
> I see Guoye has also responded about automatic retries.

Guoye's narrow use case is append-only and his response places that
append-only restriction on all resumable uploads.

> But are a chain of
> GET range requests really retries, or are they simply a sequence of related
> requests with different byte ranges?

For idempotent request method GET, the distinction does not matter,
does it?

> > > A different reading is that,
> > > having seen multiple HTTP-based custom resumable upload approaches from
> > > different vendors, there is an unaddressed use case.
> >
> > I do agree with the sentiment, just not the proposed solution.
> >
> > > Hence it is not seldom
> > > that for applicaitons that feature uploads, resumption is desirable.
> >
> > Again:
> > upload recovery is **application-specific** with a non-idempotent method
> > (automatic retrying by a client is not generically or universally safe)
> >
> > "tus - Resumable Uploads Protocol" is application-specific as I read it.
> > I think the draft should proceed to describe an application-specific
> > protocol and should severely scale back its attempt to generically
> > extend HTTP for resumable uploads.
> >
> 
> Thanks for these comments. Speaking to both points, if there were a
> different form of solution would you feel it could be less
> application-specific? I think that would be a good thing for the WG to try
> and come to some answer on; is this a problem that can ever be solved in
> HTTP or not.

The only item that comes to my mind that might be useful to standardize
is partial-PUT for append-only, but only if you do not think that it is
already sufficiently specified, and only if partial-PUT is deemed
sufficient for use instead of PATCH with media-type application/tus-v2.


More generally, I think it useful to separate uploading a large resource
(which might be interrupted) from processing that resource.

If a resource is uploaded via partial-PUT to a temporary file,
e.g. to a path assigned by initial upload request, then the resource
can be extended via partial-PUT until complete.

Once resource is completely uploaded, an HTTP request can be made to
an endpoint requesting that the target be processed.

This sequence of requests can be achieved today with existing standards.

This loosely couples multiple HTTP requests to achieve a stateful upload
before processing, where both client and server know that the upload is
complete or not, and know that processing did not occur until the upload
was complete.

Pre-requisites to allowing uploads, quotas, managing total disk usage,
temp file lifetimes, etc. can be site-specific or application-specific,
or both.

Cheers, Glenn