[Wpack] On double-hashing (was: Re: About content-based origins)

[Devin Mullins <[e-mail]>]

In light of my comment about racing the Sec-CO request, a colleague pointed
out a potential hole in the double-hash scheme. There may be flaws in
below; I haven't thought about it too much. <trite tone=apologetic>It may
be worth more clearly defining the threat model.</trite>

The publisher could delay responding to the Sec-CO request. In the interim,
the page could send its own hash to //publisher.example/this-is-my-hash.
Then the publisher would respond with the preimage sent by the bundle.
This requires:

A way for the bundle to get its own hash. Ideas:

   1. document.location -- we could fix the ni scheme to be based on the
   double-hash
   2. a fetch of distributor.example/what-is-the-hash-of-referer
   3. compute client-side based on document.body, in which the publisher
   could also include an encoding of the headers
   4. somebody computes the fixmanifold of a hash function -- is that a
   thing? i can't math [1]

A way for the publisher to associate the this-is-my-hash request with the
Sec-CO request. Ideas:

   1. IP+timestamp
   2. a token as a query param to both requests

This depends on the publisher server to have an ability that most don't
have, but it wouldn't be hard to purpose-build a server that only handled
this-is-my-hash and Sec-CO requests. By segregating its workload from the
other content requests, it would be easier to scale -- its need for
session-stickiness or regional storage wouldn't affect latency of the
edge-serving of normal content.

Overall, this still requires more effort than responding `Sec-CO: yes`, but
possibly less than a distributor/publisher backchannel. Also, possibly
immune to IP privacy depending on above.

I hope first that my above napkin sketch is wrong, and secondly that the
fix for this isn't deferral of all requests until post-transfer. My guess
is that would harm performance of the instant navigation use case.

One possible mitigation is deferral of all on-origin requests. Then there
needs to be a backchannel between the this-is-my-hash origin and the state
transfer origin. This is maybe vaguely isomorphic to the
distributor/publisher backchannel? Not really, since subdomains. The
publisher could easily CNAME sec-co.publisher.example to
sec-co.distributor.example.

Actually, I wonder if CNAME is an issue regardless of everything above.
https://tools.ietf.org/html/draft-thomson-wpack-content-origin-00#section-5.1
states the UA must follow redirects. The publisher could easily encode a
redirect from publisher.example/foo to sec-co.publisher.example/foo, and
then CNAME to the distributor, and let the distributor respond with its
known preimage.

[1] "fixmanifold" being vaguely an extension of a fixpoint. A function f(x)
s.t. for all x, hash(concat(x, f(x)) == f(x). Then the bundle could encode
its own hash.

On Tue, Mar 24, 2020 at 11:35 PM Devin Mullins <[e-mail]> wrote:

> Hi Martin,
>
> Thanks for the engineering work on this. For others, I've already reviewed
> this, had some chats with others on the AMP team, and had a few
> back-and-forths with Martin (he's been very gracious in engaging his time,
> thank you). I'm trying to estimate the feasibility of this, both for the
> AMP use-case, and for the non-AMP use-case [1], which we hope to see
> flourish to the extent that publishers and users desire it, and only
> impeded by implementation costs where necessary for the common good.
>
> With the caveat that this is very early feedback, the high-level bit is
> that, with some modifications, it seems mostly feasible for AMP. However,
> there are two main downsides:
>   1. worse UX on sites that run A/B experiments
>   2. technical constraints make this harder for publishers to adopt on
> non-AMP, as compared to SXG
>
> A lot of details follow. I suggest to rename the subject if you reply to a
> detail below, in order to reduce cross-talk and make browsing the archive
> easier.
>
> Details:
>
> We would need the display URL to be the attested URL even before transfer.
> I suspect a flash in the URL bar would be frustrating for both users and
> publishers. Jeffrey had proposed using signatures for this. I'd suppose UAs
> could choose to render this with a "may be stale" indicator.
>
> In terms of publisher implementation feasibility, I think the mostly
> likely implementation would be for a publisher to produce a
> Sec-Content-Origin (Sec-CO) response for the current version of the
> resource iff it matches the Sec-CO request. I suspect that keeping state of
> past versions of the resource would be infeasible, especially as it
> couldn't be done purely at the edge; it would require something like a
> region-wide cache. The value of the above stateless implementation would be
> inversely proportional to how often the resource changes (e.g. "transient
> content" [2] such as Related Articles). I think this would be a workable
> constraint for publishers wishing to publish such bundles, but felt it's
> worth noting, nonetheless.
>
> Re: fallback behavior on failed state transfer, distributors would need a
> way to monitor failure rates. This serves two purposes:
>   1. Automatically detecting errors in the distribution pipeline. This is
> the same purpose served by
> https://wicg.github.io/webpackage/loading.html#signed-exchange-report.
>   2. Verifying that, for some fraction of navigations, bundles are meeting
> AMP's stated UX goals (by using the verified content in the bundle and not
> potentially arbitrary content after the redirect). This is a dynamic
> equivalent of what can be done mostly statically with SXG, because the
> distributor can run the same algorithm as the UA, modulo client/server skew
> in clocks and root stores. I think this is a trade-off AMP could make. The
> alternative is that the fallback behavior keeps the content-based origin
> (CBO).
>
> There is a possible vector for user ID transfer from distributor to
> publisher; filed as
> https://github.com/martinthomson/wpack-content/issues/1.
>
> This limits the ability to run session-sticky experiments (in
> https://amp.dev/documentation/components/amp-experiment/ and many non-AMP
> frameworks). The publisher has the following options:
>   1. Delay rendering content under experiment until after state transfer.
> This hurts UX for the sake of experimentation, creating a trade-off that
> otherwise doesn't exist on origin (modulo ease of implementation).
>   2. Generate a client-side UUID before state transfer, and join it with
> the pre-existing user ID after transfer. This means the session could not
> include CBO pageviews anywhere except in the first pageview.
>   3. Generate variants of the bundle for each experimental state. The
> distributor chooses which bundle to send to the user (based on what? not
> sure). For a page with M independent experiments each with N arms, that's
> N^M variants. For large enough M and N, this is likely both infeasible for
> publishers and martinthomson/wpack-content#1. (But I need some guidance
> from experts on typical ranges for M and N.)
>
> Perhaps this is a need the UA could address. A straw proposal -- a
> selectExperimentArm() API that:
>   1. only exposes as many bits as the UA deems OK, and
>   2. exposes different bits to different attested origins, so they can't
> be joined
> On the one hand, perhaps this is too much scope for wpack. On the other,
> it addresses a problem that is somewhat particular to CBOs.
>
> This impedes content management (e.g. dialogs for GDPR and CCPA), compared
> to SXG. The publisher has a few choices:
>   1. Delay rendering the dialog under after transfer. I think this would
> cause layout shift (https://web.dev/cls/).
>   2. Render the dialog, and risk discovering that it's already been
> acknowledged. At that point:
>     a. Hide it, causing layout shift.
>     b. Leave it there. This is way outside my domain expertise, but ISTR
> there are some rules or at least best practices wrt when it's okay to
> re-show an opt-out dialog.
> One of these options may be an okay trade-off; I'm not sure.
>
> Analytics providers (via
> https://amp.dev/documentation/components/amp-analytics/ and many non-AMP
> libraries) would have to do one of two things:
>   1. Delay pingback until after transfer. This risks a lower fraction of
> successful pingbacks, as e.g. the user might close the tab before transfer.
>   2. Generate a client-side UUID, send pingback before transfer, and send
> a follow-up after transfer to join with pre-existing user ID. This captures
> those otherwise lost pingbacks pseudonymously (anonymously?), though it
> requires server-side work on the pingback endpoint.
>
> For the above client-side mitigations, two feature requests seem necessary:
>   1. Store the transfer metadata (e.g. indexedDB renames by time)
> somewhere more permanent. For various reasons the event handler may not
> fire or finish, and it would be good to later scan for stragglers to merge
> or expunge.
>   2. DOM API to wait for transfer to complete.
>
> Some subresources are ACLs by Origin header (e.g. paid fonts). Likely
> mitigation is to defer loading until transfer, with a timeout whose
> duration depends on the publisher's font-display preference. (Martin
> suggests an optimization where the encrypted subresource is loaded early,
> and only the decryption key after transfer.)
>
> Last but not least, I am concerned that this is a bar that is
> disproportionately difficult for non-AMP publishers to meet. The above
> client-side mitigations are possible, but quite a bit of work.
>   1. For frameworks like AMP that strongly encourage publishers to use a
> rolling release, it is possible to upgrade many use-cases in the wild with
> minimal publisher effort.
>   2. For other frameworks, it may be possible to provide this support, but
> the requirement to upgrade versions may limit applicability. Because
> upgrades occur less often, they often require more work.
>   3. For custom JS that manages state, authors will need to modify it to
> move or copy state into indexedDB, and handle transfer events and merge
> conflicts.
>
> From a "minimal publisher effort" perspective, I am especially interested
> in making it possible to have as close to a turnkey solution as is
> reasonable, for instance at the level of the CDN or CMS. If such a solution
> exists, sites would likely contain a hybrid of all three cases above, and
> thus want to opt into such support incrementally. How should a CDN
> determine which pages to provide Sec-CO responses for?
>   1. URL patterns are the simplest answer, but I suspect hard for
> publishers to create and maintain with minimal false negatives and
> positives.
>   2. Would it be helpful to thread an "I support state transfer"
> annotation for some portion of the build journey, from individual JS
> function all the way to bundle? Would folks use this?
>
> I don't have a good solution to the issue of non-AMP developer cost, and
> thus fear this is a solution that would feature a disproportionate amount
> of AMP pages compared to the background distribution of the web. But I'm
> hopeful that other JS framework developers can chime in with respect to
> feasibility of support for CBO and state transfer.
>
> I'm leaving out some of Martin's replies to the above; no offense
> intended. It took me long enough just to summarize the above, and I'd
> better send it early enough so at least a few people have time to read it
> before the meeting.
>
> Thanks,
> Devin
>
> [1]
> https://blog.amp.dev/2019/05/22/privacy-preserving-instant-loading-for-all-web-content/
> [2]
> http://www.seobythesea.com/2011/12/how-google-might-identify-transient-content-on-webpages/
>
> On Mon, Mar 23, 2020 at 5:35 PM Martin Thomson <[e-mail]> wrote:
>
>> Ted's note prompted me to send a much-belated announcement (sorry folks,
>> I forgot).
>>
>> The draft is here:
>> https://tools.ietf.org/html/draft-thomson-wpack-content-origin-00
>>
>> A nicer version here:
>>
>> https://martinthomson.github.io/wpack-content/draft-thomson-wpack-content-origin.html
>>
>> This approach could a dramatically different approach to addressing the
>> use cases set out in our charter.
>>
>> In short, this aims to address the core question of how offline content
>> might *ultimately* be attributed to a web origin in a fundamentally
>> different way.  There are two key concepts:
>>
>> 1. Content is given its own origin, using a new system for identification.
>>
>> 2. A target origin can "accept" content and state from one of these new
>> origins.
>>
>> There are a lot of details here (read the draft), but the major advantage
>> I see is that you don't have to make an offline decision about authority,
>> and that means you can be offline for much longer (lifting the 7 day limit).
>>
>> What it does have in common with signed exchanges approach is the need
>> for a bundling format, but in its current form it is less dependent on the
>> details of the format.  That might allow that to be simpler, but I'm sure
>> that the need to mint new identifier types will more than make up for any
>> slack there.
>>
>> The draft is quite rough.  I'm sure that it has the remnants of a few bad
>> ideas still hanging around.  Ask questions if you think something is
>> unclear.
>>
>> _______________________________________________
>> Wpack mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/wpack
>>
>