Re: Draft v1 Update for Resumable Uploads

Guoye Zhang <> Fri, 17 June 2022 03:28 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 30994C15D874 for <>; Thu, 16 Jun 2022 20:28:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -3.502
X-Spam-Status: No, score=-3.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.745, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.25, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id J0An7zynMBPF for <>; Thu, 16 Jun 2022 20:28:34 -0700 (PDT)
Received: from ( []) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by (Postfix) with ESMTPS id 42A64C15790C for <>; Thu, 16 Jun 2022 20:28:33 -0700 (PDT)
Received: from lists by with local (Exim 4.92) (envelope-from <>) id 1o22cE-0008OM-Ul for; Fri, 17 Jun 2022 03:25:46 +0000
Resent-Date: Fri, 17 Jun 2022 03:25:46 +0000
Resent-Message-Id: <>
Received: from ([]) by with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <>) id 1o22cC-0008ND-EW for; Fri, 17 Jun 2022 03:25:44 +0000
Received: from ([]) by with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <>) id 1o22cA-0007f8-N4 for; Fri, 17 Jun 2022 03:25:44 +0000
Received: from pps.filterd ( []) by ( with SMTP id 25H3DrqL007460; Thu, 16 Jun 2022 20:25:30 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; h=from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=i4AfmHRwgwsnefbRyQFt6n39pSgxOiJglz9M1bI9v+Q=; b=nH+Q1cERQwzNaotCdipQq2kYzrYc4tB1Iio3kUEPRmWPG6JyZ1Np3okT2Bk8mlAshaVG 3gf8Ys4c3ax9xVeVNhCrlIKg47lRzEMrRvXv5fPsmeoWPXInSD3L5TR1i1qv06tu3hjg EBMbUMhV9Ry7li0M+T0MAKY9YQLVN24DfRB+UpzWL5M8bYvMQlUF62L+NmvXyPsRyklR qn0kvRBaHYZbeQSSy40Sreo9Cf6+pgZQegvmIkOD9DfKjX1mcvD+H7lv2Cvm1mz72+Jk mOj0h95dTGGV37GXVSq95AN9t3t6BJvaKZbnDDHpDbYrUqZcq1JcPND+jofNlNDDFO7a TQ==
Received: from ( []) by with ESMTP id 3gmt1xgjne-21 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 16 Jun 2022 20:25:30 -0700
Received: from ( []) by (Oracle Communications Messaging Server 64bit (built Apr 7 2022)) with ESMTPS id <>; Thu, 16 Jun 2022 20:25:26 -0700 (PDT)
Received: from by (Oracle Communications Messaging Server 64bit (built Apr 7 2022)) id <>; Thu, 16 Jun 2022 20:25:26 -0700 (PDT)
X-Va-T-CD: e1c62ae64077d0caba95a15956bd4788
X-Va-E-CD: 7fa589823f194c8498e6df6440bddbf3
X-Va-R-CD: 87a202228b76ae5a02807a21fbbc1b7c
X-Va-CD: 0
X-Va-ID: 4d12d5c3-24bd-482f-8efa-6ed11aaa226f
X-V-T-CD: e1c62ae64077d0caba95a15956bd4788
X-V-E-CD: 7fa589823f194c8498e6df6440bddbf3
X-V-R-CD: 87a202228b76ae5a02807a21fbbc1b7c
X-V-CD: 0
X-V-ID: 26965b83-9991-469e-8441-5b0a295c2b5a
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.883 definitions=2022-06-17_02:2022-06-16,2022-06-17 signatures=0
Received: from (unknown []) by (Oracle Communications Messaging Server 64bit (built Apr 7 2022)) with ESMTPSA id <>; Thu, 16 Jun 2022 20:25:25 -0700 (PDT)
From: Guoye Zhang <>
Message-id: <>
Content-type: multipart/alternative; boundary="Apple-Mail=_FBC7916A-4B93-4E89-82B1-5CFF9C46BF43"
MIME-version: 1.0 (Mac OS X Mail 16.0 \(3724.\))
Date: Thu, 16 Jun 2022 20:25:14 -0700
In-reply-to: <>
Cc: HTTP Working Group <>
To: Austin William Wright <>
References: <> <>
X-Mailer: Apple Mail (2.3724.
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.883 definitions=2022-06-17_02:2022-06-16,2022-06-17 signatures=0
Received-SPF: pass client-ip=;;
X-W3C-Hub-DKIM-Status: validation passed: (, signature is good
X-W3C-Hub-Spam-Status: No, score=-7.0
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.571, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: 1o22cA-0007f8-N4 6fc10c5bf7a90990480b163ed858c8b4
Subject: Re: Draft v1 Update for Resumable Uploads
Archived-At: <>
X-Mailing-List: <> archive/latest/40140
Precedence: list
List-Id: <>
List-Help: <>
List-Post: <>
List-Unsubscribe: <>

Hi Austin,

Thanks for the review, replying inline:

> On Jun 16, 2022, at 17:59, Austin William Wright <> wrote:
>> On Jun 16, 2022, at 14:30, Guoye Zhang < <>> wrote:
>> 1. Content-Range
>> We attempted adopting Content-Range header, however, we realized that it doesn’t support unknown lengths which is an important use case that our clients require. Therefore we kept Upload-Offset and Upload-Incomplete headers.
>> We are open to discuss other options, such as modifying the semantics of the Content-Range header if that’s preferred, although it might cause more breakages than defining new headers.
> The Content-Range header does support unknown lengths <>:
> > An asterisk character ("*") in place of the complete-length indicates that the representation length was unknown when the header field was generated.
> For example:
> Content-Range: bytes 42-1233/*
> And again, while you can’t use this in requests… you could use it in a new media type (see below).

For dynamically generated content, we don’t know the full length and we also don’t want to chunk them into small upload requests. Essentially we need “bytes 42-/*” which isn’t currently allowed by Content-Range.
>> 2. Media types
>> PATCH currently doesn’t define a media type. We went through the list of media types but couldn’t find the appropriate category for the Upload Appending Procedure. It is a generic byte-appending operation that can modify any types of media, so we don’t think it fits into an application media type.
>> We are open to suggestions if a media type is desired.
> I believe a media type is required, you’re supposed to define one to use with PATCH.
> By doing this, you also guarantee that the server will support resumption, since servers that do not will error "415 Unsupported Media Type”.
> I’m working on a draft for “message/byte-range” that would suit this purpose perfectly. Its entire function would be to overwrite the specified byte range with some content. I’ve previously shared an I-D that includes this media type among other things, but I am going to split it out because it’s reusable for so many other purposes. Watch the httpapi list.

Happy to adopt this media type if it becomes available. One thing I would ask to support is the ability for the server to reject all operations other than appending. Many existing resumable upload implementations do not support overriding previously uploaded content, as the bytes might have already been processed and discarded.
>> 3. 1xx intermediate response
>> We surveyed the most popular HTTP libraries in many languages, and nearly all of them consider 1xx responses an internal signaling mechanism so they don’t expose the ability for applications to handle them. (We are also guilty of this as maintainers of URLSession API on Apple platforms.) If we use 1xx response for any critical information, it would prevent nearly all tus-v1 adopters to switch to this new protocol until it’s natively supported in HTTP libraries.
>> We think having just the feature detection part using 1xx response is a good balance, both eliminating any extra round trips for HTTP libraries implementing this protocol and allowing application adopters to ignore it.
>> 4. Can we PATCH a PATCH?
>> Yes, Upload Creation Procedure supports any method, including PATCH. We included a section “Request Identification” about the nuances in this area. Unfortunately, this added complexity is the result of splitting the procedures, but we don’t think it will complicate the implementations in most cases. Servers can still decide what methods make sense for their use case and whether to support PATCH.
> Some other remarks:
> 1. Upload-Token should be unnecessary in requests, the server should use a URI to identify operations in progress.

Let’s suppose there is a service that creates a thumbnail from a full-sized image: Anybody can send an image in the request body and receive a thumbnail in the response.

Our goal is that any interrupted uploads should be resumable. However, there isn’t a straightforward to mechanically change the URI to distinguish between attempts. Many other resumable upload protocols require you to first request a token then use it to upload which adds a roundtrip, so having a client-generated token is the best approach we could come up.
> 2. Upload-Offset as a new header should be unnecessary, this data can be stored in the PATCH body.
> 3. Feature detection: Feature detection is not a reliable way to ensure that your request will be understood correctly. In general, the request must have the expected effect even if sent to a server that doesn’t understand it. This easy to ensure this by using a new method, or a new media type, in the request.

The Upload Creation Procedure is designed so that a server that ignores unknown headers would treat it as a regular non-resumable upload. This allows generic HTTP client implementations to convert all uploads to resumable upload under the hood. This is the reason we need a feature detection mechanism.

> Cheers,
> Austin.
>> Looking forward to continuing the discussions and refinements of the draft.
>> Best regards,
>> Guoye Zhang