Re: Draft v1 Update for Resumable Uploads

Guoye Zhang <guoye_zhang@apple.com> Tue, 21 June 2022 08:19 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B8202C13A24D for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Tue, 21 Jun 2022 01:19:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.505
X-Spam-Level:
X-Spam-Status: No, score=-3.505 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.745, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.25, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=apple.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L9DsftlHac0G for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Tue, 21 Jun 2022 01:19:45 -0700 (PDT)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4BA21C13A23E for <httpbisa-archive-bis2Juki@lists.ietf.org>; Tue, 21 Jun 2022 01:19:44 -0700 (PDT)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1o3Z3X-0003FI-Rf for ietf-http-wg-dist@listhub.w3.org; Tue, 21 Jun 2022 08:16:15 +0000
Resent-Date: Tue, 21 Jun 2022 08:16:15 +0000
Resent-Message-Id: <E1o3Z3X-0003FI-Rf@lyra.w3.org>
Received: from mimas.w3.org ([128.30.52.79]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <guoye_zhang@apple.com>) id 1o3Z3V-0003Dy-SU for ietf-http-wg@listhub.w3.org; Tue, 21 Jun 2022 08:16:13 +0000
Received: from rn-mailsvcp-ppex-lapp34.rno.apple.com ([17.179.253.43] helo=rn-mailsvcp-ppex-lapp34.apple.com) by mimas.w3.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from <guoye_zhang@apple.com>) id 1o3Z3U-001sJu-J2 for ietf-http-wg@w3.org; Tue, 21 Jun 2022 08:16:13 +0000
Received: from pps.filterd (rn-mailsvcp-ppex-lapp34.rno.apple.com [127.0.0.1]) by rn-mailsvcp-ppex-lapp34.rno.apple.com (8.16.1.2/8.16.1.2) with SMTP id 25L8ETkq031612; Tue, 21 Jun 2022 01:15:59 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=s2d+d3wJsFr3et671boIlS98uHeA7it9sFjMtoG/CdU=; b=uDU+/lN0Gv55/vdC4JLe9jrzavG5XbPzbPVWJIFk8t8KBwvJdGHUvqtYqtDVkPIa2jKx sKAKuA8Zsldg++NypnhaKMdGDY9n8BYGXCeWjeNRjebgQCSTvFM/6j5lV31PHUAZi61u aQeUb0PlcMlb5159RwJn/Ns6NSJ0HpnUy5tgrT7O5Xpa3vRJv1GshRa33QJ3MgS1ayaK Bbhdsf1g3UwWRVVI8iAnFz6eevbF66OaNMNyFo8SlylxoJEgDH6K9ZkBosADUraGPUdo L5K/+hVyMjLIinLujuRsDAHuxzjzVG35cEtkgeDKWpVubsM2oN4JXsNdotpg8PuShCrk +g==
Received: from rn-mailsvcp-mta-lapp01.rno.apple.com (rn-mailsvcp-mta-lapp01.rno.apple.com [10.225.203.149]) by rn-mailsvcp-ppex-lapp34.rno.apple.com with ESMTP id 3gsaj13btb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Tue, 21 Jun 2022 01:15:59 -0700
Received: from rn-mailsvcp-mmp-lapp01.rno.apple.com (rn-mailsvcp-mmp-lapp01.rno.apple.com [17.179.253.14]) by rn-mailsvcp-mta-lapp01.rno.apple.com (Oracle Communications Messaging Server 8.1.0.18.20220407 64bit (built Apr 7 2022)) with ESMTPS id <0RDT002AUIYNG040@rn-mailsvcp-mta-lapp01.rno.apple.com>; Tue, 21 Jun 2022 01:15:59 -0700 (PDT)
Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp01.rno.apple.com by rn-mailsvcp-mmp-lapp01.rno.apple.com (Oracle Communications Messaging Server 8.1.0.18.20220407 64bit (built Apr 7 2022)) id <0RDT00D00IKFYP00@rn-mailsvcp-mmp-lapp01.rno.apple.com>; Tue, 21 Jun 2022 01:15:59 -0700 (PDT)
X-Va-A:
X-Va-T-CD: ad1c7db98facf33010cfd2d0b48e1bee
X-Va-E-CD: 7fa589823f194c8498e6df6440bddbf3
X-Va-R-CD: 87a202228b76ae5a02807a21fbbc1b7c
X-Va-CD: 0
X-Va-ID: 32b451ed-980e-4fc9-b63a-9054f2de3df1
X-V-A:
X-V-T-CD: ad1c7db98facf33010cfd2d0b48e1bee
X-V-E-CD: 7fa589823f194c8498e6df6440bddbf3
X-V-R-CD: 87a202228b76ae5a02807a21fbbc1b7c
X-V-CD: 0
X-V-ID: 1c76a70d-b8db-464c-b3c0-299838f1fd59
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.883 definitions=2022-06-21_03:2022-06-17,2022-06-21 signatures=0
Received: from smtpclient.apple (unknown [17.11.144.143]) by rn-mailsvcp-mmp-lapp01.rno.apple.com (Oracle Communications Messaging Server 8.1.0.18.20220407 64bit (built Apr 7 2022)) with ESMTPSA id <0RDT00Y0HIYMI900@rn-mailsvcp-mmp-lapp01.rno.apple.com>; Tue, 21 Jun 2022 01:15:59 -0700 (PDT)
From: Guoye Zhang <guoye_zhang@apple.com>
Message-id: <6374342D-C283-4F2F-AEBD-0A66CEA795CD@apple.com>
Content-type: multipart/alternative; boundary="Apple-Mail=_F3872703-88A7-4677-9BDD-A2B8672ECB82"
MIME-version: 1.0 (Mac OS X Mail 16.0 \(3724.0.1.1.31\))
Date: Tue, 21 Jun 2022 01:15:48 -0700
In-reply-to: <CAAPGdfGv76e8DW3DdkXMx2y0or6dZ4c=BG1Vesx_Yu5JO2Rm+A@mail.gmail.com>
Cc: ietf-http-wg@w3.org
To: Greg Wilkins <gregw@webtide.com>
References: <BED5A5BC-3F7F-47E2-815E-DC0483328DFD@apple.com> <CAAPGdfEgMF6PQjejUP-r2+vT19fS8my4Mu=V_XvwCUy=8r9Abw@mail.gmail.com> <3B21D19E-A95C-4A68-B40F-23A59C9BA5DB@apple.com> <CAAPGdfGv76e8DW3DdkXMx2y0or6dZ4c=BG1Vesx_Yu5JO2Rm+A@mail.gmail.com>
X-Mailer: Apple Mail (2.3724.0.1.1.31)
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.883 definitions=2022-06-21_03:2022-06-17,2022-06-21 signatures=0
Received-SPF: pass client-ip=17.179.253.43; envelope-from=guoye_zhang@apple.com; helo=rn-mailsvcp-ppex-lapp34.apple.com
X-W3C-Hub-DKIM-Status: validation passed: (address=guoye_zhang@apple.com domain=apple.com), signature is good
X-W3C-Hub-Spam-Status: No, score=-4.7
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.574, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: mimas.w3.org 1o3Z3U-001sJu-J2 2b01ca9ce63465524b66851db48980e0
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Draft v1 Update for Resumable Uploads
Archived-At: <https://www.w3.org/mid/6374342D-C283-4F2F-AEBD-0A66CEA795CD@apple.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/40200
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>


> On Jun 21, 2022, at 00:24, Greg Wilkins <gregw@webtide.com> wrote:
> 
> 
> On Tue, 21 Jun 2022 at 16:26, Guoye Zhang <guoye_zhang@apple.com <mailto:guoye_zhang@apple.com>> wrote:
>> We think parallel upload places unnecessary burden on the server code to support it, and it is generally an abuse of the network.
> 
> +1 
>>> ... Could the server use this mechanism to skip ahead? ... 
>> This is a really interesting use case. We do not prevent this in the current draft, and it will probably just work with our current experimental implementations. Is there a concrete example where this capability is useful?
> 
> I have no specific use-case.  I was just thinking through any constraints that might be on offset, such as rewinding to zero or a previous boundary.   So I thought about the skip forward case when thinking about constraints in general.
> I certainly know that range requests were/are used to skip ahead to specific sections of a PDF document to rapidly display them, so perhaps there are similar examples of a server that wants to check a signature at the back of a content before accepting the whole thing.   Thus I could image a sequence like:
> client posts a large document
> server send 104 
> server reads initial section containing indexes into the content
> server fails the transfer with a 5xx
> client requests offset
> an offset is given at a specific index deep inside the content
> the client sends the content from the deep offset
> server reads the deep offset content and is satisfied that the document is applicable
> server terminates the transfer with a 5xx
> client requests offset and server now responds with an offset just after the initial index section
> client sends the full content.
> This would be no stranger than what PDF viewers have done with range requests.   
> 
> In fact I can even think of a use-case now.   If a document system had given out a huge PDF to be signed by a human on some specific pages, then when that signed document is uploaded, the server might want to avoid uploading all the unsigned pages and just get the signature pages.
> 
>> We’ll definitely discuss it internally. If it isn’t too much trouble for client implementations, I don’t see any reason not to support it explicitly in the draft.
> 
> 
> For our client in the jetty project, we would have no problem with a skip forward semantic.  In fact we actually have more problems with a rewind semantic as a client can be configured with a content that generates bytes on the fly and cannot (or should not) be replayed.

Yes, rewinding could be a problem. It is always possible to rewind due to connection drop and packets in various buffers getting lost.

If the client is uploading a file and it’s arbitrarily seek-able, rewinding isn’t a problem at all. For dynamically generated content, my thinking is to keep a rolling buffer and hope it’s big enough. We’ll need real-world telemetry to figure out the necessary buffer size.

> Which now makes me think that the draft might need a bit more clarification on why there are no security implications for sending the body of a POST (or other non idempotent request).   Whilst a lot of security concerns are mitigated by asking the server for the offset and then using a PATCH method,  I can still see some implications for some types of client.
> 
> Consider a client that dynamically generates a one-time password/signature. Then a man-in-the-middle attack could read the entire on-time password, use it, but fail the transfer to the client and reply with a 0 offset.  The client would then resume from 0, generating a new one-time-password on the fly and the man in the middle could use that again and repeat.   I've no idea if this is really exploitable in any way, but I'm just thinking about when we might consider it safe to rewind our dynamic contents in our client.
> 
> It would be good if the security implications section could expand on the impact for non-idempotent requests and give recommendations of when content can be replayed in a resumed request. 

I’m leaning towards banning any content change during rewinding. If the client doesn’t have the data available any more and cannot generate the exact same bytes again, the upload should just fail. Then it would be no worse than retransmitting lost packets.

I filed issues for these topics: https://github.com/tus/tus-v2/issues/53 https://github.com/tus/tus-v2/issues/52 https://github.com/tus/tus-v2/issues/51

Guoye
> 
> cheers
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> Greg Wilkins <gregw@webtide.com <mailto:gregw@webtide.com>> CTO http://webtide.com <http://webtide.com/>