Re: Draft for Resumable Uploads

[Austin Wright <[e-mail]>]

> On Apr 5, 2022, at 22:16, Eric J Bowman <[e-mail]> wrote:
> 
> Hi Austin, you ask good questions!
> 
> >
> > Finally, I’m not sure I fully understand what the response 
> > would indicate exactly. Shall the server actually support 
> > “sparse documents” (where some bytes are undefined)? 
> >
> 
> It's all up to how the client supports rendering the media type at hand. Regardless of serving my deliberately-broken image files as 200 or 206 for several years, some browsers "filled it in" with blocks of grey, others used their broken-image icon; some browsers went from broken-icon to filled-in image, while others regressed from filled-in image to broken-image icon. Some browsers behaved differently depending on platform, i.e. comes down to image-rendering libraries. All the server can really do, is somehow indicate to the client that the requested representation is incomplete, if the server knows that for a fact.
> 
> >
> > This would allow users to upload segments out-of-order,
> > and in parallel from different uplinks. If the client does 
> > not support sparse documents, how should the server 
> > respond? (Fill in the undefined regions with zeros?)
> >
> 
> The server only cares about media-type, really. Oh, you need PNG? I have that. Just wanna let you know it's broken! Do with it what you will. I'll allow PUT/PATCH if you no likey and are authorized for those methods.
> 
> Distributed out-of-order uploads... Hmmm. I had to give that some thought, but it doesn't change my position. Any number of participating clients contributing to a file upload, just have to agree how to "fill in the blanks" on the returned representation until it's 200 OK.
> 
> I think you and mnot are correct that we need better-defined PATCH media types, I believe that's where to solve this problem, but how any media type is rendered has traditionally and properly been a client-side concern in HTTP.

So then I think a simple modification of my “Partial Uploads” document (draft-wright-http-partial-upload-01 <https://tools.ietf.org/id/draft-wright-http-partial-upload-01.html>) would work well.

First, it defines the message/byterange media type, for making changes to a specific byte range. This is the bulk of the desired functionality, I think.

Second, 2__ Sparse Resource would indicate that the resource has some regions filled in by the server, and might not be valid according to the media type definition. But I’m not confident that all user-agents would safely handle a 2__ Sparse Resource. If the resource represents executable code, the result could be very bad. Maybe I remove this? It seems wrong to me that a server could send back a document it knows is invalid according to the media type. Maybe there should be an error for clients that request a sparse resource without indicating they can support the response.

Finally, to that document, I would add a Prefer header that indicates the client supports sparse resources, and can accept a 206 Partial Content response that excludes any undefined byte ranges (unlike 2__, which would zero out these regions). If the request is a Range request (with a Range header), the server would exclude undefined ranges in its response. The client could specify in the Prefer header if it prefers multipart responses, or only single-part responses with a Content-Range header.

Synchronizing an interrupted upload might look something like this:

> HEAD /upload-target HTTP/1.1
> Prefer: sparse=single

< HTTP/1.1 206 Partial Content
< Content-Range: 400-499/1000
< Content-Length: 100

Note how the sparse=single forces the response to include a Content-Range header (instead of being embedded in a multipart response); and how the presence of this header indicates the server supports sparse resources.

By the way, I’m calling these “sparse resources” in the same way there’s sparse files <https://en.wikipedia.org/wiki/Sparse_file>. Servers could potentially implement sparse resources by directly mapping onto sparse files.

Thanks,

Austin.

> 
> -Eric
> 
>