Digest, trailers and buffering was: Re: Updating Digest header RFC3230 using "selected representation"

[Lucas Pardue <[e-mail]>]

Hi Rob!


On Tue, Jul 23, 2019 Rob Sayre <[e-mail]> wrote:

> This draft looks good to me.
>
Thanks.

I think using trailers should work these days, right?
>

HTTP/2 makes carriage of trailers a bit easier but their
applicability/usage really depends on which endpoints we are considering. I
defer to Dragana Damjanovic's nice salient description [1]

"Clients have been hesitant to implement HTTP trailers for years because of
security and consistency concerns with the trailers general case. Browsers
seek to render response bytes as soon as possible, but rendering is the
result of combining the response data with the semantics of the response
headers (e.g., the content-type header informs how a response is
displayed). Generic trailers allow header information to be added after the
rendering has begun, and this has traditionally created concerns about both
correctness and security for supporting trailers."


:On Tue, Jul 23, 2019 at 1:12 PM Rob Sayre <[e-mail]> wrote:
>
>> This draft looks good to me.
>>
>> One issue that prevented the adoption of this header and related ones
>> (e.g. Content-MD5) was that HTTP/1.1 trailers had a lot of interoperability
>> problems. This made it expensive for large files, or impossible to use for
>> message bodies that couldn't be pre-calculated.
>>
>
> That last part is slightly wrong. I should have said it was impossible to
> use for message bodies that couldn't be pre-calculated that were also
> streamed. Roughly, these headers required a lot of buffering, and this
> trade-off doesn't seem to be present anymore.
>
> thanks,
> Rob
>

Speaking personally, there is still a challenge of buffering here.
Streaming transfer does not strictly mean streaming consumption.
Progressive consumption implies an ability for progressive digest
calculation, and if that cannot happen then I don't know if sending the
digest as trailer does address trade-offs.

That said, documenting trade offs could be useful for people looking to use
Digest header. At IETF 105 we were asked to add some use cases to the
document. I wonder if you might have a candidate along the lines of this
discussion that we might add for consideration.

Kind regards
Lucas

[1]  https://www.fastly.com/blog/supercharging-server-timing-http-trailers