Re: [httpapi] Byte range PATCH

[Austin William Wright <[e-mail]>]

> On Feb 5, 2023, at 08:47, Darrel Miller <[e-mail]> wrote:
> 
> Austin,
>  
> Thank you for sharing your proposal.  It is definitely the case that API providers do need this ability to perform partial updates to a resource.
>  
> However, it was my assumption that the updates to RFC 9110 that explicitly call out the potential for doing partial PUT https://www.rfc-editor.org/rfc/rfc9110.html#name-partial-put using the Content-Range header was the acknowledgement that this is what folks are doing and its ok to do it.
>  
> There are several assertions in your draft that suggest you don’t agree with this.
>  
> “this technique cannot generally be used in PUT, because the server may ignore the Content-Range header”
>  
> “However, if an upload is interrupted, no mechanism exists to upload only the remaining data; the entire request must be retried.”
>  
> “Although the Content-Range field cannot be used in the request headers without risking data corruption...”

Yes, I didn’t directly acknowledge RFC 9110. Despite mentioning it, Content-Range still cannot be used on servers not known to support it (it would cause data corruption). The spec says this outright:

> though such support is inconsistent and depends on private agreements with user agents

In contrast, a patch media type wouldn't depend on any prior knowledge, it could safely be used opportunistically (i.e. even if server support is unknown).

> In section 14.5 there is guidance that a server SHOULD respond with a 400 if Content-Range is received but not supported.  If this were a MUST, would that mitigate the concerns you raised?

Unfortunately clients wouldn’t be able to rely on this behavior, as it would be incompatible with existing HTTP servers. And it seems unlikely that new HTTP servers (Node.js applications, etc) would honor this requirement.

Also, it doesn’t make sense to me that specific features like this should be anticipated in the HTTP core semantics. This would introduce an exception to the “unknown headers are ignored” guideline of HTTP extensibility. Byte range patching should be an optional feature that servers can implement or ignore. And of the major extension points offered in HTTP (a header, method, or content-type), a new method or content-type is the correct option.

> Regarding your proposal, I do think there is value to being able to provide multiple parts to the content of the PATCH request.  I could imagine certain sync scenarios where that would be an efficient way of  communicating changes.

Yeah, an important use to me is working with many kinds of media files, where you can perform an append, but you also need to update an index or heading, in one operation.

> I’m not sure I understand the value of having a Content-Type as part of the header. I would have thought that patching would happen assuming the target is effectively application/octet-stream.  It would seem problematic to suggest doing a byte range update to a content-type that is text/plain when the charset is either not known or unspecified.  I don’t see how any kind of media type semantics would impact how the patch is performed.  I am only considering the case where range units are bytes.

For existing resources, Content-Type is not meaningful and would probably be omitted. However when creating new resources, there has to be a substitute for the Content-Type header in a PUT request with Content-Range.

Given this request:

PUT /file.mp3 HTTP/1.1
If-None-Match: *
Content-Range: 0-40/*
Content-Type: image/png

How would you perform an equivalent PATCH request that creates the resource with the correct content type? The content-type of the resource has to be included inside the "envelope" of the patch.

> My only other thought here is, are we reinventing things like VCDIFF https://www.rfc-editor.org/rfc/rfc3284 at this point?

Byte range patch is simpler and maps directly to standard filesystem operations, and so is more likely to be adopted. Even a small VCDIFF file could require a very large file to be entirely rewritten, and not all servers are interested in supporting this. The only thing they really have in common is they’re generic ways to patch an existing document (unlike JSON Patch, these patch formats are not interested in the meaning of the data being written to).

That said, maybe there ought to be a VCDIFF media type, for patching an executable file or tarball, etc. I’m sure there’s a lengthy list of prior work that would have benefitted from a standard.

Thanks,

Austin.

>  
> Darrel
>  
>  
> From: Austin William Wright <mailto:[e-mail]>
> Sent: January 23, 2023 4:14 PM
> To: HTTP APIs Working Group <mailto:[e-mail]>
> Subject: [httpapi] Byte range PATCH
>  
> Hello HTTP APIs,
>  
> I’m seeking to standardize a media type for writing to particular byte offsets in a target document. This seems to be a common problem in HTTP applications, including in my own work with streaming parsers.
>  
> This sort of operation is common in filesystems. For example, you append audio data to a .wav file, and then you update the header at the start of the file. I don’t know of any way to do this operation over HTTP, except a PUT that replaces the entire resource.
>  
> This has been discussed in the HTTP WG list, as part of “resumable uploads” <https://httpwg.org/http-extensions/draft-ietf-httpbis-resumable-upload.html>, and I emailed there some months ago about this I-D. But aside from the resumable uploads being in HTTP WG, this seems more like an API-related feature.
>  
> Here’s the I-D I’ve written: draft-wright-http-patch-byterange-01 <https://www.ietf.org/archive/id/draft-wright-http-patch-byterange-01.html>
>  
> Editor’s copy: https://awwright.github.io/http-progress/draft-wright-http-patch-byterange.html
>  
> This is the successor to version 00; in response to some feedback, I added a binary format. And I discuss what is necessary to support “splicing” (an insertion that shifts other content around) and indeterminate-length uploads, which is a requested feature for the “resumable uploads” spec.
>  
> Is HTTP APIs the best place to pursue this? I’m hoping I can discuss this at IETF 116.
>  
> Thanks,
>  
> Austin Wright.