Re: Resumable Uploads

Felix Geisendörfer <felix@transloadit.com> Mon, 22 April 2013 07:59 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8980921F8CDD for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Mon, 22 Apr 2013 00:59:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.576
X-Spam-Level:
X-Spam-Status: No, score=-9.576 tagged_above=-999 required=5 tests=[AWL=0.100, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6Hc+rxfcQcxP for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Mon, 22 Apr 2013 00:59:39 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id C78C021F8CA5 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Mon, 22 Apr 2013 00:59:39 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1UUBdw-00039r-1P for ietf-http-wg-dist@listhub.w3.org; Mon, 22 Apr 2013 07:58:32 +0000
Resent-Message-Id: <E1UUBdw-00039r-1P@frink.w3.org>
Received: from maggie.w3.org ([128.30.52.39]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <ylafon@w3.org>) id 1UUBdt-00038i-4G for ietf-http-wg@listhub.w3.org; Mon, 22 Apr 2013 07:58:29 +0000
Received: from jay.w3.org ([128.30.52.169]) by maggie.w3.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from <ylafon@w3.org>) id 1UUBdt-0004NR-2f for ietf-http-wg@w3.org; Mon, 22 Apr 2013 07:58:29 +0000
Received: from ylafon by jay.w3.org with local (Exim 4.72) (envelope-from <ylafon@w3.org>) id 1UUBds-00078k-TE for ietf-http-wg@w3.org; Mon, 22 Apr 2013 03:58:28 -0400
X-Return-path: <>
X-Received: from maggie.w3.org ([128.30.52.39]) by jay.w3.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.72) id 1UT91d-00060U-7a for ylafon@jay.w3.org; Fri, 19 Apr 2013 06:58:41 -0400
X-Received: from frink.w3.org ([128.30.52.56]) by maggie.w3.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.72) id 1UT91d-0005ED-46 for ylafon@w3.org; Fri, 19 Apr 2013 10:58:41 +0000
X-Received: from lists by frink.w3.org with local (Exim 4.72) id 1UT91c-0007WE-UI for ylafon@w3.org; Fri, 19 Apr 2013 10:58:40 +0000
Date: Fri, 19 Apr 2013 10:58:40 +0000
X-From_: felix.geisendoerfer@transloadit.com Fri Apr 19 10:58:38 2013
X-Received: from lisa.w3.org ([128.30.52.41]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <felix.geisendoerfer@transloadit.com>) id 1UT91a-0007VZ-OF for ietf-http-wg@listhub.w3.org; Fri, 19 Apr 2013 10:58:38 +0000
X-Received: from mail-vb0-f48.google.com ([209.85.212.48]) by lisa.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <felix.geisendoerfer@transloadit.com>) id 1UT91Z-0000ON-Tc for ietf-http-wg@w3.org; Fri, 19 Apr 2013 10:58:38 +0000
X-Received: by mail-vb0-f48.google.com with SMTP id p13so3464328vbe.35 for <ietf-http-wg@w3.org>; Fri, 19 Apr 2013 03:58:12 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:sender:x-originating-ip:in-reply-to :references:from:date:x-google-sender-auth:message-id:subject:to:cc :content-type:x-gm-message-state; bh=mbj7ox/jxs5jgmAo1e04/hyyS+b/iEo6cmLAp7kL1JA=; b=F51BJczo4mwK0xukDL6wq3DvuwmmUvNtxudJQzOjEXTRS1ju+sZ06EGlMl8YksJA3m bXSrVAr8iFQor7DHhIQrPf145hsOePfPW1VZP7Kinh6ZFuFn8R6oya4zVuRSPWT5a8aQ wLKjTrqYhnRqwUS8Wz/dTibve6e79OvE+BnWgMv566ppexDJkYCfVsNZtRKhi3Cz8kk/ gltPlAXr9xTvrLsuVoOudZ8S02PTjZBjiSQF8JGJjbTz7LshNK2GAyQgmaMtdlbcrpkA WIBnaCPxkDgwLHmEMLEA1kZtNT5bJ3+FmsCXRiXl0vHun80xK/URPKFG+400gizgvjP8 iUWg==
X-Received: by 10.220.168.2 with SMTP id s2mr11127376vcy.6.1366369092236; Fri, 19 Apr 2013 03:58:12 -0700 (PDT)
MIME-Version: 1.0
Sender: felix.geisendoerfer@transloadit.com
X-Received: by 10.58.15.165 with HTTP; Fri, 19 Apr 2013 03:57:52 -0700 (PDT)
X-Originating-IP: [91.64.81.5]
In-Reply-To: <5170E2A3.6010706@gmx.de>
References: <CADZbJ9dYFGyrceh03M3B0KdKto7160Dis_geh9um0BhVe1re0g@mail.gmail.com> <alpine.DEB.2.00.1304182006001.21288@tvnag.unkk.fr> <29DE6A70-E3B9-4DCE-8C7E-506F6A0ADC92@gmail.com> <51706F32.5030108@panix.com> <5170E2A3.6010706@gmx.de>
From: =?ISO-8859-1?Q?Felix_Geisend=F6rfer?= <felix@transloadit.com>
Old-Date: Fri, 19 Apr 2013 12:57:52 +0200
X-Google-Sender-Auth: H7EAXFusBj0KxcaKt14Xj5PQaTQ
Message-ID: <CADZbJ9dGEVq-fQmhjRsddYdcg459r_zLfrOddkzHLOprZM0dNg@mail.gmail.com>
To: Julian Reschke <julian.reschke@gmx.de>
Cc: Albert Lunde <atlunde@panix.com>, Kevin Swiber <kswiber@gmail.com>, Daniel Stenberg <daniel@haxx.se>, HTTP Working Group <ietf-http-wg@w3.org>, Yves Lafon <ylafon@w3.org>
Content-Type: multipart/alternative; boundary=001a11c18aa460502404dab49899
X-Gm-Message-State: ALoCoQk0zqWcGeUFaqm3SSedWikqeV1g99bDf7/aZDlVlV8ymSQRpReajqjdmoGSd/hHU1sSOgU0
Received-SPF: none client-ip=209.85.212.48; envelope-from=felix.geisendoerfer@transloadit.com; helo=mail-vb0-f48.google.com
X-W3C-Hub-Spam-Status: No, score=-2.8
X-W3C-Hub-Spam-Report: AWL=-2.087, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7
X-W3C-Scan-Sig: lisa.w3.org 1UT91Z-0000ON-Tc 6e9c7410ffaa6a2c2bacb8b52c90280b
Old-X-Envelope-To: ietf-http-wg
X-DSPAM-Result: Innocent
X-DSPAM-Processed: Fri Apr 19 07:00:20 2013
X-DSPAM-Confidence: 0.9991
X-DSPAM-Improbability: 1 in 109771 chance of being spam
X-DSPAM-Probability: 0.0000
X-DSPAM-Signature: 517123c3231001804284693
ReSent-Date: Mon, 22 Apr 2013 03:58:26 -0400 (EDT)
ReSent-From: Yves Lafon <ylafon@w3.org>
ReSent-To: ietf-http-wg@w3.org
ReSent-Subject: [Moderator Action] Re: Resumable Uploads
ReSent-User-Agent: Alpine 2.00 (DEB 1167 2008-08-23)
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Resumable Uploads
Archived-At: <http://www.w3.org/mid/CADZbJ9dGEVq-fQmhjRsddYdcg459r_zLfrOddkzHLOprZM0dNg@mail.gmail.com>
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/17462
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

Thanks for the replies!

Using PATCH with a custom media type is unfortunately not a viable solution
for browser based clients.  The XHR2 send interface [1] and File API [2]
are not flexible enough to perform the body wrapping without loading the
entire request body into memory. There is no way to stream body data other
than providing a file object (or a slice of it).

This makes specifying the enclosed byte-range in the request header the
most desirable option. So far we are abusing the "Content-Range" header for
this (and so does the YouTube API [3]).

So I'm wondering what you'd consider to be the lesser evil: Using
Content-Range for this, or creating a custom header?

Don't get me wrong, I really want a proper solution for this in the long
run and would love to help with specifying it, if it's an interesting
problem to solve with http2. But if such a solution can't be had today, I'd
still like to create a stopgap for now.

[1] http://www.w3.org/TR/XMLHttpRequest2/#the-send-method
[2] http://www.w3.org/TR/FileAPI/
[3]
https://developers.google.com/youtube/v3/guides/using_resumable_upload_protocol

--
Felix Geisendörfer (felixge.de)
Co-Founder, Transloadit (transloadit.com)