Re: Draft v1 Update for Resumable Uploads

[Lucas Pardue <[e-mail]>]

Hi Glenn,

Responding in line:

On Mon, Jun 20, 2022 at 3:03 AM Glenn Strauss <
[e-mail]> wrote:

> On Sun, Jun 19, 2022 at 04:52:58PM +0100, Lucas Pardue wrote:
> > Hi Glenn,
> >
> > I'd like to respond to one aspectdirectly:
> >
> > On Sun, Jun 19, 2022 at 7:04 AM <[e-mail]>
> wrote:
> >
> > > On Thu, Jun 16, 2022 at 02:30:59PM -0700, Guoye Zhang wrote:
> > > > Our previous resumable upload draft generated a lot of discussions.
> > >
> > > At least in my case, I attempted to be polite after you submitted a
> > > draft without first doing a survey of existing RFCs.  You admitted no
> > > knowledge of WebDAV RFCs, which I deemed a large oversight considering
> > > the nature of the tus-v2 protocol.
> > >
> > > > I’m glad to announce that we have a new draft ready to address many
> > > feedbacks that suggested adopting the PATCH method.
> > >
> > > The draft abstract begins with unsubstantiated claims to justify
> itself,
> > > and I believe that almost all of those claims are also misleading.
> > >
> > > "HTTP clients often encounter interrupted data transfers as a result of
> > > canceled requests or dropped connections. [...] it is often desirable
> to
> > > issue subsequent requests that transfer only the remainder of the
> > > representation."
> > >
> > > The multiple uses of "often" are misrepresentations, IMHO.
> > >
> > > A large percentage of HTTP requests are GET/HEAD and have no body.
> > > A sizable percentage (if not more) of HTTP POST requests are small,
> > > e.g. using POST as an alternative to GET along with XSRF tokens.
> > >
> > > What data do you have to support the claims in the draft Abstract?
> > > What percentage of requests have request bodies, and further have
> > > request bodies that are sufficiently large that it is excessively
> > > wasteful to resend the entire representation? (and when safe to do so!)
> > >
> >
> > The text you quote from the tus v2 draft is based on an almost verbatim
> > section of text from RFC 9110 Section 14 [1], which describes Range
> > Requests. Quote:
> >
> > > Clients often encounter interrupted data transfers as a result of
> > canceled requests or dropped connections. When a client has stored a
> > partial representation, it is desirable to request the remainder of that
> > representation in a subsequent request rather than transfer the entire
> > representation.
>
> In RFC 9110, that paragraph *immediately* follows the heading "Range
> Requests".  In that context, "Clients often encounter interrupted data
> transfers" implicitly refers to downloads, as described in the section.
>

RFC 9110 Section 14.5 also includes information about uploads wrt partial
PUT, which was motivated by discussion of use cases in
https://github.com/httpwg/http-core/issues/618 and survey of approached in
https://lists.w3.org/Archives/Public/ietf-http-wg/2020OctDec/0285.html

So while I see the logic, I don't think it's fair to assume that RFC 9110
is only about downloads.


> On the other hand, when quoted in the Abstract of the tus draft titled
> "tus - Resumable Uploads Protocol", I mistook that to refer to uploads.
> Hence, I found the statements misleading, since "interrupted data
> transfers" is ambiguous and could refer to downloads or uploads.
>

When failures happen on a connection, they'll affect an upload flow just as
much as a download one. But I take the point that the document title in
combination with the opening sentence could be misinterpreted. A better way
would be to present what is supported today, present the capability gap,
and talk about the goal of the draft. Subsequent versions of a document
could even drop the exposition and focus on the specification protocol
details, should the WG manage to get so far.


> > I wrote the tus v2 text to explicity juxtapose resumable HTTP downloads
> > against uploads. To quote the abstract in full:
> >
> > > HTTP clients often encounter interrupted data transfers as a result
> >    of canceled requests or dropped connections.  Prior to interruption,
> >    part of a representation may have been exchanged.  To complete the
> >    data transfer of the entire representation, it is often desirable to
> >    issue subsequent requests that transfer only the remainder of the
> >    representation.  HTTP range requests support this concept of
> >    resumable downloads from server to client.  This document describes a
> >    mechanism that supports resumable uploads from client to server using
> >    HTTP.
>
> When a client uses an idempotent HTTP request method, such as GET, then
> the client may repeat a disconnected request, and may attempt to utilize
> a Range request.
>
> When a client is providing a request body upload with a non-idempotent
> method, such as POST or PUT, then the request is not necessarily safe to
> repeat.  Appropriate behavior to attempt to recover may be
> application-specific.
>
> I want to emphasize: **application-specific**
>
> Hence, I still think that Abstract is misleading and an inaccurate
> attempt to convey equivalence between client download using an
> idempotent method, and client upload using a non-idempotent method.
> They are not equivalent; there are very important distinctions that
> should be explicitly stated before trying to describe similarities.
>

I see Guoye has also responded about automatic retries. But are a chain of
GET range requests really retries, or are they simply a sequence of related
requests with different byte ranges?


> > A different reading is that,
> > having seen multiple HTTP-based custom resumable upload approaches from
> > different vendors, there is an unaddressed use case.
>
> I do agree with the sentiment, just not the proposed solution.
>
> > Hence it is not seldom
> > that for applicaitons that feature uploads, resumption is desirable.
>
> Again:
> upload recovery is **application-specific** with a non-idempotent method
> (automatic retrying by a client is not generically or universally safe)
>
> "tus - Resumable Uploads Protocol" is application-specific as I read it.
> I think the draft should proceed to describe an application-specific
> protocol and should severely scale back its attempt to generically
> extend HTTP for resumable uploads.
>

Thanks for these comments. Speaking to both points, if there were a
different form of solution would you feel it could be less
application-specific? I think that would be a good thing for the WG to try
and come to some answer on; is this a problem that can ever be solved in
HTTP or not.

Cheers
Lucas