Re: Draft v1 Update for Resumable Uploads

Austin William Wright <aaa@bzfx.net> Fri, 17 June 2022 01:02 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EDFECC15AACA for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 16 Jun 2022 18:02:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.757
X-Spam-Level:
X-Spam-Status: No, score=-2.757 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.25, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=bzfx.net
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FmGcTZmAg2OA for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 16 Jun 2022 18:02:26 -0700 (PDT)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 50551C14F693 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 16 Jun 2022 18:02:25 -0700 (PDT)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1o20L3-0004nH-H8 for ietf-http-wg-dist@listhub.w3.org; Fri, 17 Jun 2022 00:59:53 +0000
Resent-Date: Fri, 17 Jun 2022 00:59:53 +0000
Resent-Message-Id: <E1o20L3-0004nH-H8@lyra.w3.org>
Received: from titan.w3.org ([128.30.52.76]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <aaa@bzfx.net>) id 1o20L1-0004mO-Tq for ietf-http-wg@listhub.w3.org; Fri, 17 Jun 2022 00:59:51 +0000
Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by titan.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from <aaa@bzfx.net>) id 1o20L0-0001fH-1G for ietf-http-wg@w3.org; Fri, 17 Jun 2022 00:59:51 +0000
Received: by mail-pf1-x432.google.com with SMTP id z17so2886735pff.7 for <ietf-http-wg@w3.org>; Thu, 16 Jun 2022 17:59:49 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bzfx.net; s=google; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=0r8DCdQHoTtCalNsxmv/BCNigd2MGLzIaKP2hvJTuKY=; b=aOVyfUupSyNmHxUdHr0ouV7kqJOX/xhcA1xuXBWzmSupO5OwQJaVdiBISXww7N6fa/ 5YEIJxA83S0WFOVZ0fQ+r08qmlA1p1CDE1jnjXL7f+6VdiNJUZVMAxJc2CVEqxWd1LQa KNIEf+qfQ+41s5zsvyDhLLrayHgfGuAcRkwNw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=0r8DCdQHoTtCalNsxmv/BCNigd2MGLzIaKP2hvJTuKY=; b=ucs2vypUFddJs5WWiNy0MmednhS13DtCipoB1jKBovkfbGp5v9SBC/lXbK68ejibHG GXWa/2fG3JTdrTukxYmOpyWWGHqzBlyjZRaTDJp1n4IqFhoe9E7tc1DN5ndCZZ2HlNcj 3eefmL78pzHPkWKnDmPcs53OEmL3AtAimny5oQ+BWt8s4LoUJhlIrzmFW+5jOo4CVdfw VggwMciTi8AsXxvxZuWoM9MiVrhlfOB8Hnzk+iq4akX221YMo9qkmd3R8SzetfY2cvI6 NiZuWz4pYXRcDxa4lQfoB5by/dwJjKm7Z88m0FOmNHlzaYNlF0Cn5gbeqx0gsgMEbQMX HgDQ==
X-Gm-Message-State: AJIora/aUgp+euDz4tlCp/ZITDqVvW5qS6xyt9sKmc6NHIYca4Qn0Am4 eOtIH0kaF14/MOxEmAWhNdl/IcjSrWnRkg==
X-Google-Smtp-Source: AGRyM1sdX9Evn7ga+XqTxH8Q+IBf4KnBt3vQ/uVAoEQO+isl23yNvkN2t0If4aLpkhcj0EcqOjYc+w==
X-Received: by 2002:a05:6a00:2389:b0:51c:3ca7:b177 with SMTP id f9-20020a056a00238900b0051c3ca7b177mr7495405pfc.17.1655427578365; Thu, 16 Jun 2022 17:59:38 -0700 (PDT)
Received: from smtpclient.apple (71-223-174-56.phnx.qwest.net. [71.223.174.56]) by smtp.gmail.com with ESMTPSA id l3-20020a17090aaa8300b001e08461ceaesm4366186pjq.37.2022.06.16.17.59.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 Jun 2022 17:59:37 -0700 (PDT)
From: Austin William Wright <aaa@bzfx.net>
Message-Id: <65DCF777-6AA0-4A4C-BA4F-2F260AF0BDD3@bzfx.net>
Content-Type: multipart/alternative; boundary="Apple-Mail=_7A730DA3-42DE-4966-AF44-5D90F84AC481"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.100.31\))
Date: Thu, 16 Jun 2022 17:59:36 -0700
In-Reply-To: <BED5A5BC-3F7F-47E2-815E-DC0483328DFD@apple.com>
Cc: HTTP Working Group <ietf-http-wg@w3.org>
To: Guoye Zhang <guoye_zhang@apple.com>
References: <BED5A5BC-3F7F-47E2-815E-DC0483328DFD@apple.com>
X-Mailer: Apple Mail (2.3696.100.31)
Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=aaa@bzfx.net; helo=mail-pf1-x432.google.com
X-W3C-Hub-DKIM-Status: validation passed: (address=aaa@bzfx.net domain=bzfx.net), signature is good
X-W3C-Hub-Spam-Status: No, score=-6.1
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, W3C_AA=-1, W3C_DB=-1, W3C_IRA=-1, W3C_WL=-1
X-W3C-Scan-Sig: titan.w3.org 1o20L0-0001fH-1G 86a03cd8add76740b71443833ca8fa3e
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Draft v1 Update for Resumable Uploads
Archived-At: <https://www.w3.org/mid/65DCF777-6AA0-4A4C-BA4F-2F260AF0BDD3@bzfx.net>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/40139
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

> On Jun 16, 2022, at 14:30, Guoye Zhang <guoye_zhang@apple.com> wrote:
> 
> 1. Content-Range
> 
> We attempted adopting Content-Range header, however, we realized that it doesn’t support unknown lengths which is an important use case that our clients require. Therefore we kept Upload-Offset and Upload-Incomplete headers.
> 
> We are open to discuss other options, such as modifying the semantics of the Content-Range header if that’s preferred, although it might cause more breakages than defining new headers.

The Content-Range header does support unknown lengths <https://httpwg.org/specs/rfc9110.html#field.content-range>:

> An asterisk character ("*") in place of the complete-length indicates that the representation length was unknown when the header field was generated.

For example:
Content-Range: bytes 42-1233/*
And again, while you can’t use this in requests… you could use it in a new media type (see below).

> 2. Media types
> 
> PATCH currently doesn’t define a media type. We went through the list of media types but couldn’t find the appropriate category for the Upload Appending Procedure. It is a generic byte-appending operation that can modify any types of media, so we don’t think it fits into an application media type.
> 
> We are open to suggestions if a media type is desired.

I believe a media type is required, you’re supposed to define one to use with PATCH.

By doing this, you also guarantee that the server will support resumption, since servers that do not will error "415 Unsupported Media Type”.

I’m working on a draft for “message/byte-range” that would suit this purpose perfectly. Its entire function would be to overwrite the specified byte range with some content. I’ve previously shared an I-D that includes this media type among other things, but I am going to split it out because it’s reusable for so many other purposes. Watch the httpapi list.

> 3. 1xx intermediate response
> 
> We surveyed the most popular HTTP libraries in many languages, and nearly all of them consider 1xx responses an internal signaling mechanism so they don’t expose the ability for applications to handle them. (We are also guilty of this as maintainers of URLSession API on Apple platforms.) If we use 1xx response for any critical information, it would prevent nearly all tus-v1 adopters to switch to this new protocol until it’s natively supported in HTTP libraries.
> 
> We think having just the feature detection part using 1xx response is a good balance, both eliminating any extra round trips for HTTP libraries implementing this protocol and allowing application adopters to ignore it.
> 
> 4. Can we PATCH a PATCH?
> 
> Yes, Upload Creation Procedure supports any method, including PATCH. We included a section “Request Identification” about the nuances in this area. Unfortunately, this added complexity is the result of splitting the procedures, but we don’t think it will complicate the implementations in most cases. Servers can still decide what methods make sense for their use case and whether to support PATCH.

Some other remarks:

1. Upload-Token should be unnecessary in requests, the server should use a URI to identify operations in progress.

2. Upload-Offset as a new header should be unnecessary, this data can be stored in the PATCH body.

3. Feature detection: Feature detection is not a reliable way to ensure that your request will be understood correctly. In general, the request must have the expected effect even if sent to a server that doesn’t understand it. This easy to ensure this by using a new method, or a new media type, in the request.


Cheers,

Austin.

> 
> 
> Looking forward to continuing the discussions and refinements of the draft.
> 
> Best regards,
> Guoye Zhang