Re: Draft v1 Update for Resumable Uploads

[Guoye Zhang <[e-mail]>]

> On Jun 20, 2022, at 22:42, Greg Wilkins <[e-mail]> wrote:
> 
> Gouye,
> 
> It was not immediately clear to me on initially reading the draft exactly what the intent of this proposal is with regards to the existing usage of multipart by clients to upload files (https://datatracker.ietf.org/doc/html/rfc7578).  Some more text in the intro (and 3.2 series of parts) to help to make the intent clearer.
> 
> To be clear myself, I understand that:
> This is not a replacement for multipart uploads, rather it is a mechanism that could be applied on top that could allow a multipart upload (among other content types) to be interrupted and resumed.
> The usage of this mechanism to deliberately split an upload into parts is distinct from RFC7578 parts (although no reason why each RFC7578 part could not be sent in a separate resumable request if server support is known).
Yes, you are entirely correct. tus-v2 supports multipart upload, and the parts in tus-v2 has no relationship with parts in multipart. The multipart body is treated as an opaque binary blob like any other request bodies.
> The use of this mechanism to initiate parallel requests to "speed up" the upload is explicitly not allowed.  Perhaps you should note that this is asymmetric to down loads with range request, which can be done in parallel.  
Good point! We will add a note about it.

We think parallel upload places unnecessary burden on the server code to support it, and it is generally an abuse of the network. It is not supported by the current production tus protocol, either.
> When the server is queried for an offset, it is free to pick an offset as it likes. For example, it could wind back the transfer to the last boundary of a multi-part content, even if some of those bytes had already been received.      Could the server use this mechanism to skip ahead?  for example during a large upload the server determines that it is only interested in the last part of the content. The if the server closes the current transfer and when a offset query request is received, can it send an offset in advance of any bytes sent by the client, so the client would then need to skip through to the offset (rather than rewind)? 
This is a really interesting use case. We do not prevent this in the current draft, and it will probably just work with our current experimental implementations. Is there a concrete example where this capability is useful?

We’ll definitely discuss it internally. If it isn’t too much trouble for client implementations, I don’t see any reason not to support it explicitly in the draft.

Guoye
> 
> regards
> 
> 
> 
> 
> On Fri, 17 Jun 2022 at 07:36, Guoye Zhang <[e-mail] <mailto:[e-mail]>> wrote:
>> Hi all,
>> 
>> Our previous resumable upload draft generated a lot of discussions. I’m glad to announce that we have a new draft ready to address many feedbacks that suggested adopting the PATCH method. In this draft, we split the Upload Transfer Procedure into 2 separate procedures: Upload Creation Procedure and Upload Appending Procedure.
>> 
>> https://datatracker.ietf.org/doc/draft-tus-httpbis-resumable-uploads-protocol/
>> 
>> 1. Content-Range
>> 
>> We attempted adopting Content-Range header, however, we realized that it doesn’t support unknown lengths which is an important use case that our clients require. Therefore we kept Upload-Offset and Upload-Incomplete headers.
>> 
>> We are open to discuss other options, such as modifying the semantics of the Content-Range header if that’s preferred, although it might cause more breakages than defining new headers.
>> 
>> 2. Media types
>> 
>> PATCH currently doesn’t define a media type. We went through the list of media types but couldn’t find the appropriate category for the Upload Appending Procedure. It is a generic byte-appending operation that can modify any types of media, so we don’t think it fits into an application media type.
>> 
>> We are open to suggestions if a media type is desired.
>> 
>> 3. 1xx intermediate response
>> 
>> We surveyed the most popular HTTP libraries in many languages, and nearly all of them consider 1xx responses an internal signaling mechanism so they don’t expose the ability for applications to handle them. (We are also guilty of this as maintainers of URLSession API on Apple platforms.) If we use 1xx response for any critical information, it would prevent nearly all tus-v1 adopters to switch to this new protocol until it’s natively supported in HTTP libraries.
>> 
>> We think having just the feature detection part using 1xx response is a good balance, both eliminating any extra round trips for HTTP libraries implementing this protocol and allowing application adopters to ignore it.
>> 
>> 4. Can we PATCH a PATCH?
>> 
>> Yes, Upload Creation Procedure supports any method, including PATCH. We included a section “Request Identification” about the nuances in this area. Unfortunately, this added complexity is the result of splitting the procedures, but we don’t think it will complicate the implementations in most cases. Servers can still decide what methods make sense for their use case and whether to support PATCH.
>> 
>> 
>> Looking forward to continuing the discussions and refinements of the draft.
>> 
>> Best regards,
>> Guoye Zhang
> 
> 
> -- 
> Greg Wilkins <[e-mail] <mailto:[e-mail]>> CTO http://webtide.com <http://webtide.com/>