Resumable Uploads

Felix Geisendörfer <felix@transloadit.com> Thu, 18 April 2013 17:38 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8192721F8948 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 18 Apr 2013 10:38:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.676
X-Spam-Level:
X-Spam-Status: No, score=-9.676 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qV5DRMN4tSHz for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 18 Apr 2013 10:38:26 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id D0DE121F88E3 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 18 Apr 2013 10:38:25 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1USsmG-0005bH-Gu for ietf-http-wg-dist@listhub.w3.org; Thu, 18 Apr 2013 17:37:44 +0000
Resent-Message-Id: <E1USsmG-0005bH-Gu@frink.w3.org>
Received: from lisa.w3.org ([128.30.52.41]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <ylafon@w3.org>) id 1USsmE-0005ZZ-Do for ietf-http-wg@listhub.w3.org; Thu, 18 Apr 2013 17:37:42 +0000
Received: from jay.w3.org ([128.30.52.169]) by lisa.w3.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from <ylafon@w3.org>) id 1USsmE-00069c-CZ for ietf-http-wg@w3.org; Thu, 18 Apr 2013 17:37:42 +0000
Received: from ylafon by jay.w3.org with local (Exim 4.72) (envelope-from <ylafon@w3.org>) id 1USsmE-0002EO-7B for ietf-http-wg@w3.org; Thu, 18 Apr 2013 13:37:42 -0400
X-Return-path: <>
X-Received: from maggie.w3.org ([128.30.52.39]) by jay.w3.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.72) id 1USp27-0002pZ-Fn for ylafon@jay.w3.org; Thu, 18 Apr 2013 09:37:51 -0400
X-Received: from frink.w3.org ([128.30.52.56]) by maggie.w3.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.72) id 1USp27-0000It-By for ylafon@w3.org; Thu, 18 Apr 2013 13:37:51 +0000
X-Received: from lists by frink.w3.org with local (Exim 4.72) id 1USp27-0007dX-4P for ylafon@w3.org; Thu, 18 Apr 2013 13:37:51 +0000
Date: Thu, 18 Apr 2013 13:37:51 +0000
X-From_: felix.geisendoerfer@transloadit.com Thu Apr 18 13:37:48 2013
X-Received: from maggie.w3.org ([128.30.52.39]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <felix.geisendoerfer@transloadit.com>) id 1USp24-0007bg-PP for ietf-http-wg@listhub.w3.org; Thu, 18 Apr 2013 13:37:48 +0000
X-Received: from mail-vb0-f54.google.com ([209.85.212.54]) by maggie.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <felix.geisendoerfer@transloadit.com>) id 1USp23-0000HV-18 for ietf-http-wg@w3.org; Thu, 18 Apr 2013 13:37:48 +0000
X-Received: by mail-vb0-f54.google.com with SMTP id w16so2389498vbf.27 for <ietf-http-wg@w3.org>; Thu, 18 Apr 2013 06:37:21 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:sender:x-originating-ip:from:date :x-google-sender-auth:message-id:subject:to:content-type :x-gm-message-state; bh=CT38Q/5zqsgezOHxZXreJKteC53BAPMRg6ibNXnjpyw=; b=AaaQi4Bp1NFC8kqHVKgzVOPv7WsN+T3zNzKSIh82ueogHhOj93mPK755sGV/vinIyu xbc70zAz9DD3WDx4hCGthwXmwt7ARX4uplvnvRqtoDMzHNgX82MeF88QTuzkH4f8iocQ CwohXSPI1iCk8mQ9fVfskvtrFzBNw0Ldhhoc79cxlHCUpf9fDURXz3CI8HObM3XAPHJH y1eiLbys6z1sWEBkE+l2R0NAx+NDAuxA8Jum7OWB4N6q20oLcqXLHr41QZYi/qI7AUGv DcEUPSLSJpwjJPlaiIa//l+Z3G3Y2/zq6679c/AfjzeIwqvqFZhYcsbY7HPLKY9rN9eT DZ4Q==
X-Received: by 10.58.224.101 with SMTP id rb5mr8307460vec.17.1366292240991; Thu, 18 Apr 2013 06:37:20 -0700 (PDT)
MIME-Version: 1.0
Sender: felix.geisendoerfer@transloadit.com
X-Received: by 10.58.15.165 with HTTP; Thu, 18 Apr 2013 06:36:59 -0700 (PDT)
X-Originating-IP: [91.64.81.5]
From: =?ISO-8859-1?Q?Felix_Geisend=F6rfer?= <felix@transloadit.com>
Old-Date: Thu, 18 Apr 2013 15:36:59 +0200
X-Google-Sender-Auth: OxlaVyH4lgEMqrQFQDPTUjMB5R0
Message-ID: <CADZbJ9dYFGyrceh03M3B0KdKto7160Dis_geh9um0BhVe1re0g@mail.gmail.com>
To: ietf-http-wg@w3.org
Content-Type: multipart/alternative; boundary=089e0117635faf3dc704daa2b3f8
X-Gm-Message-State: ALoCoQmmrkkRiC5N1KM49JgabNZgHezp0UXzP423qCUc/8BAgms0qbTZn8O/gB+3NxeSxvoCCSc4
Received-SPF: none client-ip=209.85.212.54; envelope-from=felix.geisendoerfer@transloadit.com; helo=mail-vb0-f54.google.com
X-W3C-Hub-Spam-Status: No, score=-0.7
X-W3C-Hub-Spam-Report: HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7
X-W3C-Scan-Sig: maggie.w3.org 1USp23-0000HV-18 ec7dc3b4311e5b931b74fdb507386510
Old-X-Envelope-To: ietf-http-wg
X-DSPAM-Result: Innocent
X-DSPAM-Processed: Thu Apr 18 09:45:31 2013
X-DSPAM-Confidence: 0.9981
X-DSPAM-Improbability: 1 in 52869 chance of being spam
X-DSPAM-Probability: 0.0000
X-DSPAM-Signature: 516ff8fb108911684769271
ReSent-Date: Thu, 18 Apr 2013 13:37:38 -0400 (EDT)
ReSent-From: Yves Lafon <ylafon@w3.org>
ReSent-To: ietf-http-wg@w3.org
ReSent-Subject: [Moderator Action] Resumable Uploads
ReSent-User-Agent: Alpine 2.00 (DEB 1167 2008-08-23)
X-Original-To: ietf-http-wg@w3.org
Subject: Resumable Uploads
Archived-At: <http://www.w3.org/mid/CADZbJ9dYFGyrceh03M3B0KdKto7160Dis_geh9um0BhVe1re0g@mail.gmail.com>
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/17341
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

Hi,

I'm interested in finding out how to perform resumable uploads over http
while being compliant with existing specifications. The result of this work
will be shared with the community to create interopable server/client
software to simplify file uploading on the web.

The first draft for this can be seen here [1]. I'm new to protocol
authorship, so please forgive any ignorance you may see in my work. This is
no attempt at competing with existing standards, I merely want to identify
the relevant subsets of http and make them easy for people to implement.

Anyway, here are the things I'm wondering about specifically:

- How should a http client transfer a partial byte range of a given
resource? Specifically:
- Should the PUT or PATCH method be used? (neither seem fully appropiate
according to my research)
- What header should be used to indicate the transfered range for the
upload? Julian Reschke pointed out that Content-Range may not be appropiate
[2].
- What header should be used to indicate the received range of an
interrupted upload?

Ideally clients should be able to transfer multiple parts of a file using
parallel requests (to fully utilize long fat pipes). This will require
servers to indicate which parts of a file have previously been received.
Another thing I'd like for servers, is to process partially received
requests, otherwise small chunk sizes would have to be used for mobile
which would lead to significant overhead.

Thanks in advance for any insights you might be able to provide!

[1]: http://www.tus.io/protocols/resumable-upload.html
[2]: https://twitter.com/jreschke/status/324872511697326080

--
Felix Geisendörfer (felixge.de)
Co-Founder, Transloadit (transloadit.com)