Re: Draft v1 Update for Resumable Uploads
Greg Wilkins <gregw@webtide.com> Thu, 23 June 2022 07:03 UTC
Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5F3EFC157B3E for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 23 Jun 2022 00:03:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.657
X-Spam-Level:
X-Spam-Status: No, score=-7.657 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.25, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=webtide-com.20210112.gappssmtp.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kRDhltlSneWx for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 23 Jun 2022 00:03:49 -0700 (PDT)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3188BC157B33 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 23 Jun 2022 00:03:48 -0700 (PDT)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1o4GpD-0002tm-GZ for ietf-http-wg-dist@listhub.w3.org; Thu, 23 Jun 2022 07:00:23 +0000
Resent-Date: Thu, 23 Jun 2022 07:00:23 +0000
Resent-Message-Id: <E1o4GpD-0002tm-GZ@lyra.w3.org>
Received: from mimas.w3.org ([128.30.52.79]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <gregw@webtide.com>) id 1o4GpB-0002ss-SB for ietf-http-wg@listhub.w3.org; Thu, 23 Jun 2022 07:00:21 +0000
Received: from mail-lf1-x130.google.com ([2a00:1450:4864:20::130]) by mimas.w3.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from <gregw@webtide.com>) id 1o4GpA-003A9S-Mb for ietf-http-wg@w3.org; Thu, 23 Jun 2022 07:00:21 +0000
Received: by mail-lf1-x130.google.com with SMTP id z13so1236802lfj.13 for <ietf-http-wg@w3.org>; Thu, 23 Jun 2022 00:00:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=webtide-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=zCcEHmz3M+QXM9yfrG2MmjRBgclPRivdHa5W96Yk5r8=; b=1vWMXM1/oYr5ml1/J7XasXvh/zajJ7C833/RyHGkMwEUo2CYfoiPYPk+iaTXu6q51o 99myBZbMg43Arizcvy2qN0PHFLn1qCgiByZ3MksdWoxd7dpTawIabMcvL57BpMeSI1qs 3XPaIer2Q/AoqjI5tZIy5njfKho93KppYlEOHBQdNxo3y8Bi6+lsjiKAA7Nl5yQvrHK4 8RKwRPj9ndtQmoSHMQCow5v8OsVwuQh1LirPGrS9tLZMxDZ4es21AtxPtkxK5r8aEvK/ pB0zPdZB7i5cDOjU7ZXYghquFo5X1SKRzdoxmiR4OY86QsgEBhplsy5C3gbz8IFYmDId kGjg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zCcEHmz3M+QXM9yfrG2MmjRBgclPRivdHa5W96Yk5r8=; b=56etN5fMEmPbwXzTT4psDmzRHGcz6Sqm708/ssvb769jf2khBnwWrPBJJ63DnSwhcY cGpjdkDqdi71bySA7fTtTQ+iA2Q5hIkz0QTqrSzyzi+LZ97NalaGHfang8DGNzt7soZN i7U036+q1wHHiZJu4+8HtZeGX/LFaeDMJriK6nDKr3SMvd+sw7Ah80t3jLKWRHjhA3ky lJI/p83EjCpIUrwSmz0yBsbo/rVkfVD1W3dNKj1jxQ236cW8bvpLjINn7iq0wRLw9t9Z Xoi/lYJ/vTyd5GNqHiu7Bqx+eCSWJAZfg1bCUOKtsvtYN6nuIoWVVVlvW+Gxo9ezuRZe +OkQ==
X-Gm-Message-State: AJIora+D4vudeRl8hiPuN4GjJKj5XX/4jOiUi7XzOmzP2XbMZf3lCWp+ WyqiOcYnR2AUfrQAdpbxZQW++jEGQc26ooezSWhSGg==
X-Google-Smtp-Source: AGRyM1ux7HctSjIoYEK1HYEhxcKbWSZX2sIOdVansqvcQFnJshv6Snr4sJFyEEe7yF40s2UKNgDERwoDgPDDSWe83Q8=
X-Received: by 2002:a05:6512:10ca:b0:47f:795e:8dbf with SMTP id k10-20020a05651210ca00b0047f795e8dbfmr4383546lfg.423.1655967606837; Thu, 23 Jun 2022 00:00:06 -0700 (PDT)
MIME-Version: 1.0
References: <BED5A5BC-3F7F-47E2-815E-DC0483328DFD@apple.com> <Yq67WGkb0LtJIAP9@xps13> <D149DCFE-A5C9-418D-80B4-3B5F138AA497@apple.com> <Yq/mYB6FMLWn/7Oj@xps13> <1A0308B7-266A-4E12-BC6C-6D321BAFC3D3@apple.com> <1817f68e1bc.b0da438140687.4010670760181959722@zoho.com> <E0263B0F-3C3A-46F4-AA1D-7580C5ADE326@apple.com> <d32488ea-05b3-e97f-3118-3845306faa6a@gmx.de> <181808e7a44.aed3d7b643553.5952608326455145559@zoho.com> <A00C7493-2637-4C2A-B880-EC03AD82EDF3@apple.com>
In-Reply-To: <A00C7493-2637-4C2A-B880-EC03AD82EDF3@apple.com>
From: Greg Wilkins <gregw@webtide.com>
Date: Thu, 23 Jun 2022 16:59:55 +1000
Message-ID: <CAAPGdfEKqBy2cybU_cazhuMpqf445dSUW4yBk8bOg2F+zw6r7g@mail.gmail.com>
To: Guoye Zhang <guoye_zhang@apple.com>
Cc: Eric J Bowman <mellowmutt@zoho.com>, Julian Reschke <julian.reschke@gmx.de>, ietf-http-wg <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary="000000000000f72cdb05e218002d"
Received-SPF: softfail client-ip=2a00:1450:4864:20::130; envelope-from=gregw@webtide.com; helo=mail-lf1-x130.google.com
X-W3C-Hub-DKIM-Status: validation passed: (address=gregw@webtide.com domain=webtide-com.20210112.gappssmtp.com), signature is good
X-W3C-Hub-Spam-Status: No, score=-3.2
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665, T_SCC_BODY_TEXT_LINE=-0.01, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: mimas.w3.org 1o4GpA-003A9S-Mb 1edb50fc59585716d7e83b09ec8afda0
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Draft v1 Update for Resumable Uploads
Archived-At: <https://www.w3.org/mid/CAAPGdfEKqBy2cybU_cazhuMpqf445dSUW4yBk8bOg2F+zw6r7g@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/40204
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
Guoye et al, I too agree with the sentiment that it would be good to avoid multipart with text boundaries. I think the current draft would have each part being sent in a separate HTTP message. I don't particularly mind that, but there are some latency concerns that the restriction on concurrent requests would introduce latency as you would need to wait for one part to be entirely complete before starting the next. Eitherway, thinking about this made me realize that the current draft needs to discuss intermediaries and how they might handle the multiple requests in a resumed upload. If a client is connected to a cluster with non-sticky bindings to back end servers, then a query for an offset or a resumed upload might end up on a different server within a cluster. So either intermediaries need to be sticky on upload-token or other steps must be taken to ensure that a resumed conversation is sticky. cheers On Thu, 23 Jun 2022 at 10:50, Guoye Zhang <guoye_zhang@apple.com> wrote: > > > On Jun 20, 2022, at 03:00, Eric J Bowman <mellowmutt@zoho.com> wrote: > > > > > Maybe > > > https://www.ietf.org/archive/id/draft-ietf-httpbis-binary-message-05.html > could > > be used; that would avoid the multipart parsing challenge. > > > > Ha, like I needed another hobby project from this thread! I get what > you're saying, thx Julian. Section 3.8, Padding and Truncation -- > message/bhttp as a general-purpose PATCH format for BLOBs. > > -Eric > > > I’ll admit that I was confused by this exchange. I know binary message > from oblivious HTTP, but I think Julian was suggesting replacing multipart > part boundaries with binary message, not using it as the PATCH format. Let > me explain what I understood from this discussion, please correct me if I’m > wrong: > > (1) My initial blocker for adopting Content-Range was that we needed a > open range: > > :method: PATCH > upload-offset: 42 > > [Dynamically generated body with unknown length] > > This is unrepresentable by Content-Range. > > (2) Eric suggested multipart body with Content-Range, essentially: > > :method: PATCH > content-type: multipart/sometime; boundary=“boundary" > > --boundary > Content-Range: 42-50/* > > [8 bytes] > --boundary > Content-Range: 51-60/* > > [10 bytes] > --boundary > … > > (3) I expressed concerns about multipart overhead. > > (4) Julian suggested using binary message to serialize messages more > efficiently, replacing plaintext headers and textual boundaries with the > binary serialization format. > > (5) Eric suggested using binary message itself as the PATCH format to > represent arbitrary binary diff. > > My reply to (4): If we need to invent a new format anyway, I’d avoid > multipart since we don’t really need it for appending a continuous blob. > > My reply to (5): I don’t think this is what binary message is for, but I’m > really curious what you were able to achieve with this idea. Having a > general-purpose binary patching format would be great. > > Guoye > -- Greg Wilkins <gregw@webtide.com> CTO http://webtide.com
- Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Austin William Wright
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Martin J. Dürst
- Re: Draft v1 Update for Resumable Uploads gs-lists-ietf-http-wg
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Lucas Pardue
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Glenn Strauss
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Lucas Pardue
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Glenn Strauss
- Re: Draft v1 Update for Resumable Uploads Glenn Strauss
- Re: Draft v1 Update for Resumable Uploads Glenn Strauss
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Mark Nottingham
- Re: Draft v1 Update for Resumable Uploads Glenn Strauss
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Julian Reschke
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Julian Reschke
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Julian Reschke
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Eric J Bowman
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Greg Wilkins
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Greg Wilkins
- Re: Draft v1 Update for Resumable Uploads Poul-Henning Kamp
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang
- Re: Draft v1 Update for Resumable Uploads Greg Wilkins
- Re: Draft v1 Update for Resumable Uploads Guoye Zhang