Re: Draft v1 Update for Resumable Uploads

Glenn Strauss <gs-lists-ietf-http-wg@gluelogic.com> Mon, 20 June 2022 04:52 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 18CD4C14F733 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Sun, 19 Jun 2022 21:52:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.662
X-Spam-Level:
X-Spam-Status: No, score=-2.662 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, MAILING_LIST_MULTI=-1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4z0ZHCQbMQ4a for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Sun, 19 Jun 2022 21:52:29 -0700 (PDT)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 81577C15D89C for <httpbisa-archive-bis2Juki@lists.ietf.org>; Sun, 19 Jun 2022 21:52:29 -0700 (PDT)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1o39M4-0005DI-0m for ietf-http-wg-dist@listhub.w3.org; Mon, 20 Jun 2022 04:49:40 +0000
Resent-Date: Mon, 20 Jun 2022 04:49:40 +0000
Resent-Message-Id: <E1o39M4-0005DI-0m@lyra.w3.org>
Received: from mimas.w3.org ([128.30.52.79]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <gs-lists-ietf-http-wg@gluelogic.com>) id 1o39M3-0005CX-4L for ietf-http-wg@listhub.w3.org; Mon, 20 Jun 2022 04:49:39 +0000
Received: from smtp1.atof.net ([52.86.233.228]) by mimas.w3.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (Exim 4.94.2) (envelope-from <gs-lists-ietf-http-wg@gluelogic.com>) id 1o39M2-0018Or-LR for ietf-http-wg@w3.org; Mon, 20 Jun 2022 04:49:38 +0000
X-Spam-Language: en
X-Spam-Relay-Country:
X-Spam-DCC: B=; R=smtp1.atof.net 1102; Body=1 Fuz1=1 Fuz2=1
X-Spam-RBL:
X-Spam-PYZOR: Reported 0 times.
Date: Mon, 20 Jun 2022 00:49:18 -0400
From: Glenn Strauss <gs-lists-ietf-http-wg@gluelogic.com>
To: Lucas Pardue <lucaspardue.24.7@gmail.com>
Cc: Guoye Zhang <guoye_zhang@apple.com>, HTTP Working Group <ietf-http-wg@w3.org>
Message-ID: <Yq/8TtFbArWFsQUo@xps13>
References: <BED5A5BC-3F7F-47E2-815E-DC0483328DFD@apple.com> <Yq67WGkb0LtJIAP9@xps13> <CALGR9oa-zUTL4z_jrzvknBOoeYoksT-mPgz3m3ddUFzBZZ6khg@mail.gmail.com> <Yq/VYMe+VUDdnY1c@xps13> <CALGR9oamykn+QbTG89ELMLCuHHdP=PEod_jgJ7-6JgJivnNhKA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
In-Reply-To: <CALGR9oamykn+QbTG89ELMLCuHHdP=PEod_jgJ7-6JgJivnNhKA@mail.gmail.com>
Received-SPF: pass client-ip=52.86.233.228; envelope-from=gs-lists-ietf-http-wg@gluelogic.com; helo=smtp1.atof.net
X-W3C-Hub-Spam-Status: No, score=-6.2
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: mimas.w3.org 1o39M2-0018Or-LR e175309da88a9792f3bb8a4db73bccc8
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Draft v1 Update for Resumable Uploads
Archived-At: <https://www.w3.org/mid/Yq/8TtFbArWFsQUo@xps13>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/40181
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

On Mon, Jun 20, 2022 at 03:41:02AM +0100, Lucas Pardue wrote:
> 
> On Mon, Jun 20, 2022 at 3:03 AM Glenn Strauss <
> gs-lists-ietf-http-wg@gluelogic.com> wrote:
> 
> > On Sun, Jun 19, 2022 at 04:52:58PM +0100, Lucas Pardue wrote:
> I see Guoye has also responded about automatic retries.

Guoye's narrow use case is append-only and his response places that
append-only restriction on all resumable uploads.

> But are a chain of
> GET range requests really retries, or are they simply a sequence of related
> requests with different byte ranges?

For idempotent request method GET, the distinction does not matter,
does it?

> > > A different reading is that,
> > > having seen multiple HTTP-based custom resumable upload approaches from
> > > different vendors, there is an unaddressed use case.
> >
> > I do agree with the sentiment, just not the proposed solution.
> >
> > > Hence it is not seldom
> > > that for applicaitons that feature uploads, resumption is desirable.
> >
> > Again:
> > upload recovery is **application-specific** with a non-idempotent method
> > (automatic retrying by a client is not generically or universally safe)
> >
> > "tus - Resumable Uploads Protocol" is application-specific as I read it.
> > I think the draft should proceed to describe an application-specific
> > protocol and should severely scale back its attempt to generically
> > extend HTTP for resumable uploads.
> >
> 
> Thanks for these comments. Speaking to both points, if there were a
> different form of solution would you feel it could be less
> application-specific? I think that would be a good thing for the WG to try
> and come to some answer on; is this a problem that can ever be solved in
> HTTP or not.

The only item that comes to my mind that might be useful to standardize
is partial-PUT for append-only, but only if you do not think that it is
already sufficiently specified, and only if partial-PUT is deemed
sufficient for use instead of PATCH with media-type application/tus-v2.


More generally, I think it useful to separate uploading a large resource
(which might be interrupted) from processing that resource.

If a resource is uploaded via partial-PUT to a temporary file,
e.g. to a path assigned by initial upload request, then the resource
can be extended via partial-PUT until complete.

Once resource is completely uploaded, an HTTP request can be made to
an endpoint requesting that the target be processed.

This sequence of requests can be achieved today with existing standards.

This loosely couples multiple HTTP requests to achieve a stateful upload
before processing, where both client and server know that the upload is
complete or not, and know that processing did not occur until the upload
was complete.

Pre-requisites to allowing uploads, quotas, managing total disk usage,
temp file lifetimes, etc. can be site-specific or application-specific,
or both.

Cheers, Glenn