Re: Draft v1 Update for Resumable Uploads

Greg Wilkins <gregw@webtide.com> Tue, 21 June 2022 05:45 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D7105C15D882 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Mon, 20 Jun 2022 22:45:48 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.657
X-Spam-Level:
X-Spam-Status: No, score=-7.657 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.25, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=webtide-com.20210112.gappssmtp.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aUIbDSQCvQcm for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Mon, 20 Jun 2022 22:45:45 -0700 (PDT)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DADCDC15D87F for <httpbisa-archive-bis2Juki@lists.ietf.org>; Mon, 20 Jun 2022 22:45:44 -0700 (PDT)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1o3Wf2-0001jf-Pr for ietf-http-wg-dist@listhub.w3.org; Tue, 21 Jun 2022 05:42:48 +0000
Resent-Date: Tue, 21 Jun 2022 05:42:48 +0000
Resent-Message-Id: <E1o3Wf2-0001jf-Pr@lyra.w3.org>
Received: from mimas.w3.org ([128.30.52.79]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <gregw@webtide.com>) id 1o3Wf0-0001iM-K1 for ietf-http-wg@listhub.w3.org; Tue, 21 Jun 2022 05:42:46 +0000
Received: from mail-lf1-x135.google.com ([2a00:1450:4864:20::135]) by mimas.w3.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from <gregw@webtide.com>) id 1o3Wez-001noQ-Di for ietf-http-wg@w3.org; Tue, 21 Jun 2022 05:42:46 +0000
Received: by mail-lf1-x135.google.com with SMTP id f39so4906935lfv.3 for <ietf-http-wg@w3.org>; Mon, 20 Jun 2022 22:42:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=webtide-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=CMKFWogYIrR1ls9ldmZ7xkeBXHkv05hfod0juFMk5ls=; b=Cn4s1wCtTE0N1eCx/oH/X3j+7eL9uX9gB4gb3MPIfHkuwsIuyJfIL4gvH2y2vCGKOA wPeYHpz45LNXyyJfpeiZeVQ+9/CJvpWDUC5gx2jxwP9MpfzH5NIdU2LaOolasnQkNBhV B0pn9pa3a2ZUOdqaE3U5ssVrawFNkq1ULgTxkcRwTzvivQpI4o4kromJUf1jzsuud8UB RNqbfUzmuBmrCj7vI9O5oKoJCaJBOcepR2qk8ngT/b2YMeKb4QLzwKLCmWP2uwXGu73H eVOyaTwMBtmpp6IjiAbPiomNYP4q0TuqwkUvOpqQlA1Lk3YWBzYTIfX9AI7O1wSAlqUH m9aQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=CMKFWogYIrR1ls9ldmZ7xkeBXHkv05hfod0juFMk5ls=; b=IFrQfkOF+lpjv9sU7A4MXnsXKSWZRKMsK9up4kP+WHCJQKiP/clPjCzatWOZGdiCEY T10bkU7IO3knWewYP4tgAtC0Xj79UDLW03l6ESNlx+7TcsapRYiUOPu8AYJn2ANYX/Qy mPXDHKGOF5Adm5q8uD5LhmXHqBNunRfGzSzpGtGNJXdhSzzgd2/0ftzaOhGUxP60tR+Q g77ubleM2vVKGRDa9ufVdzSKVQ4t3bW6RVVDKvhJJf48BMx9MkiKjSJqtZJPvA9l8oep SFVkRQhmv08ZpCUTJVM7SA0/UfSUXWvge9oW+vNbVscIwDDMOvVLp2zy7Q3pj0zOLueC dOQA==
X-Gm-Message-State: AJIora89nBAWxot2GkLLFEwwNTfMPBpukVg0asOnFfN2qDRv0yvPiE7o fk5Oda4zooXEdp0vgwkDAqjcdiJ0ThQ6DMYZrYi19UCiFVfibw==
X-Google-Smtp-Source: AGRyM1u35eIqUgO8FPuCF+sQGjEHxvL5Zwe13KZWGgjfPX+HN7gTQvDCi5agXIMcal3F0IT36Ter6IZlzr8OC0Y/gH4=
X-Received: by 2002:a05:6512:10cc:b0:47f:42ff:71db with SMTP id k12-20020a05651210cc00b0047f42ff71dbmr15288728lfg.34.1655790151717; Mon, 20 Jun 2022 22:42:31 -0700 (PDT)
MIME-Version: 1.0
References: <BED5A5BC-3F7F-47E2-815E-DC0483328DFD@apple.com>
In-Reply-To: <BED5A5BC-3F7F-47E2-815E-DC0483328DFD@apple.com>
From: Greg Wilkins <gregw@webtide.com>
Date: Tue, 21 Jun 2022 15:42:20 +1000
Message-ID: <CAAPGdfEgMF6PQjejUP-r2+vT19fS8my4Mu=V_XvwCUy=8r9Abw@mail.gmail.com>
To: Guoye Zhang <guoye_zhang@apple.com>
Cc: ietf-http-wg@w3.org
Content-Type: multipart/alternative; boundary="000000000000d0eda905e1eeafe5"
Received-SPF: softfail client-ip=2a00:1450:4864:20::135; envelope-from=gregw@webtide.com; helo=mail-lf1-x135.google.com
X-W3C-Hub-DKIM-Status: validation passed: (address=gregw@webtide.com domain=webtide-com.20210112.gappssmtp.com), signature is good
X-W3C-Hub-Spam-Status: No, score=-3.2
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665, T_SCC_BODY_TEXT_LINE=-0.01, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: mimas.w3.org 1o3Wez-001noQ-Di 3a095f1b07fc6b65a2ed0776f6e4fd65
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Draft v1 Update for Resumable Uploads
Archived-At: <https://www.w3.org/mid/CAAPGdfEgMF6PQjejUP-r2+vT19fS8my4Mu=V_XvwCUy=8r9Abw@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/40194
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

Gouye,

It was not immediately clear to me on initially reading the draft exactly
what the intent of this proposal is with regards to the existing usage of
multipart by clients to upload files (
https://datatracker.ietf.org/doc/html/rfc7578).  Some more text in the
intro (and 3.2 series of parts) to help to make the intent clearer.

To be clear myself, I understand that:

   1. This is not a replacement for multipart uploads, rather it is a
   mechanism that could be applied on top that could allow a multipart upload
   (among other content types) to be interrupted and resumed.
   2. The usage of this mechanism to deliberately split an upload into
   parts is distinct from RFC7578 parts (although no reason why each RFC7578
   part could not be sent in a separate resumable request if server support is
   known).
   3. The use of this mechanism to initiate parallel requests to "speed up"
   the upload is explicitly not allowed.  Perhaps you should note that this is
   asymmetric to down loads with range request, which can be done in
   parallel.
   4. When the server is queried for an offset, it is free to pick an
   offset as it likes. For example, it could wind back the transfer to the
   last boundary of a multi-part content, even if some of those bytes had
   already been received.      Could the server use this mechanism to skip
   ahead?  for example during a large upload the server determines that it is
   only interested in the last part of the content. The if the server closes
   the current transfer and when a offset query request is received, can it
   send an offset in advance of any bytes sent by the client, so the client
   would then need to skip through to the offset (rather than rewind)?


regards




On Fri, 17 Jun 2022 at 07:36, Guoye Zhang <guoye_zhang@apple.com> wrote:

> Hi all,
>
> Our previous resumable upload draft generated a lot of discussions. I’m
> glad to announce that we have a new draft ready to address many feedbacks
> that suggested adopting the PATCH method. In this draft, we split the
> Upload Transfer Procedure into 2 separate procedures: Upload Creation
> Procedure and Upload Appending Procedure.
>
>
> https://datatracker.ietf.org/doc/draft-tus-httpbis-resumable-uploads-protocol/
>
> 1. Content-Range
>
> We attempted adopting Content-Range header, however, we realized that it
> doesn’t support unknown lengths which is an important use case that our
> clients require. Therefore we kept Upload-Offset and Upload-Incomplete
> headers.
>
> We are open to discuss other options, such as modifying the semantics of
> the Content-Range header if that’s preferred, although it might cause more
> breakages than defining new headers.
>
> 2. Media types
>
> PATCH currently doesn’t define a media type. We went through the list of
> media types but couldn’t find the appropriate category for the Upload
> Appending Procedure. It is a generic byte-appending operation that can
> modify any types of media, so we don’t think it fits into an application
> media type.
>
> We are open to suggestions if a media type is desired.
>
> 3. 1xx intermediate response
>
> We surveyed the most popular HTTP libraries in many languages, and nearly
> all of them consider 1xx responses an internal signaling mechanism so they
> don’t expose the ability for applications to handle them. (We are also
> guilty of this as maintainers of URLSession API on Apple platforms.) If we
> use 1xx response for any critical information, it would prevent nearly all
> tus-v1 adopters to switch to this new protocol until it’s natively
> supported in HTTP libraries.
>
> We think having just the feature detection part using 1xx response is a
> good balance, both eliminating any extra round trips for HTTP libraries
> implementing this protocol and allowing application adopters to ignore it.
>
> 4. Can we PATCH a PATCH?
>
> Yes, Upload Creation Procedure supports any method, including PATCH. We
> included a section “Request Identification” about the nuances in this area.
> Unfortunately, this added complexity is the result of splitting the
> procedures, but we don’t think it will complicate the implementations in
> most cases. Servers can still decide what methods make sense for their use
> case and whether to support PATCH.
>
>
> Looking forward to continuing the discussions and refinements of the draft.
>
> Best regards,
> Guoye Zhang
>


-- 
Greg Wilkins <gregw@webtide.com> CTO http://webtide.com