Re: Draft v1 Update for Resumable Uploads

[Guoye Zhang <[e-mail]>]

> On Jun 21, 2022, at 00:24, Greg Wilkins <[e-mail]> wrote:
> 
> 
> On Tue, 21 Jun 2022 at 16:26, Guoye Zhang <[e-mail] <mailto:[e-mail]>> wrote:
>> We think parallel upload places unnecessary burden on the server code to support it, and it is generally an abuse of the network.
> 
> +1 
>>> ... Could the server use this mechanism to skip ahead? ... 
>> This is a really interesting use case. We do not prevent this in the current draft, and it will probably just work with our current experimental implementations. Is there a concrete example where this capability is useful?
> 
> I have no specific use-case.  I was just thinking through any constraints that might be on offset, such as rewinding to zero or a previous boundary.   So I thought about the skip forward case when thinking about constraints in general.
> I certainly know that range requests were/are used to skip ahead to specific sections of a PDF document to rapidly display them, so perhaps there are similar examples of a server that wants to check a signature at the back of a content before accepting the whole thing.   Thus I could image a sequence like:
> client posts a large document
> server send 104 
> server reads initial section containing indexes into the content
> server fails the transfer with a 5xx
> client requests offset
> an offset is given at a specific index deep inside the content
> the client sends the content from the deep offset
> server reads the deep offset content and is satisfied that the document is applicable
> server terminates the transfer with a 5xx
> client requests offset and server now responds with an offset just after the initial index section
> client sends the full content.
> This would be no stranger than what PDF viewers have done with range requests.   
> 
> In fact I can even think of a use-case now.   If a document system had given out a huge PDF to be signed by a human on some specific pages, then when that signed document is uploaded, the server might want to avoid uploading all the unsigned pages and just get the signature pages.
> 
>> We’ll definitely discuss it internally. If it isn’t too much trouble for client implementations, I don’t see any reason not to support it explicitly in the draft.
> 
> 
> For our client in the jetty project, we would have no problem with a skip forward semantic.  In fact we actually have more problems with a rewind semantic as a client can be configured with a content that generates bytes on the fly and cannot (or should not) be replayed.

Yes, rewinding could be a problem. It is always possible to rewind due to connection drop and packets in various buffers getting lost.

If the client is uploading a file and it’s arbitrarily seek-able, rewinding isn’t a problem at all. For dynamically generated content, my thinking is to keep a rolling buffer and hope it’s big enough. We’ll need real-world telemetry to figure out the necessary buffer size.

> Which now makes me think that the draft might need a bit more clarification on why there are no security implications for sending the body of a POST (or other non idempotent request).   Whilst a lot of security concerns are mitigated by asking the server for the offset and then using a PATCH method,  I can still see some implications for some types of client.
> 
> Consider a client that dynamically generates a one-time password/signature. Then a man-in-the-middle attack could read the entire on-time password, use it, but fail the transfer to the client and reply with a 0 offset.  The client would then resume from 0, generating a new one-time-password on the fly and the man in the middle could use that again and repeat.   I've no idea if this is really exploitable in any way, but I'm just thinking about when we might consider it safe to rewind our dynamic contents in our client.
> 
> It would be good if the security implications section could expand on the impact for non-idempotent requests and give recommendations of when content can be replayed in a resumed request. 

I’m leaning towards banning any content change during rewinding. If the client doesn’t have the data available any more and cannot generate the exact same bytes again, the upload should just fail. Then it would be no worse than retransmitting lost packets.

I filed issues for these topics: https://github.com/tus/tus-v2/issues/53 https://github.com/tus/tus-v2/issues/52 https://github.com/tus/tus-v2/issues/51

Guoye
> 
> cheers
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> Greg Wilkins <[e-mail] <mailto:[e-mail]>> CTO http://webtide.com <http://webtide.com/>