Re: [Wpack] About content-based origins

Wow, that's a long list.

I think that a lot of this relates to the inherent difficulty of providing customization under a regime like was described.

This includes all the experimentation things, CCPA/GDPR consent notices [1], analytics, loading resources with referring origin-based ACLs [2], personalization of content, etc...

This is something that a signature-based system helps with because you can front-load all of this into custom bundles that are all signed with the same key, but you treat all of these as equivalent.

I think that Ekr and I both separately mentioned in on the call last week, but it's worth writing this down somewhere (because I failed to do so thus far).  There is a variant of this design that uses signature public keys as the primary identifier and considers all content signed with the corresponding private key as the one co-extant origin.  That avoids most of this mess.  There are a few challenges, but as the liveness test (Sec-Signer-Origin maybe) remains, concerns about revocation are approximately the same as it pertains to protecting the target origin.

The reputation issue remains, as the granularity of key use becomes imperative, and I don't think we should accept any signer-based origins as valid if the the same key is used for a different target origin or a certificate.  In other words, don't cross the streams: if you want to attest as example.com that this content is yours, then you will need to use a different key for that purpose.  (I suggest we continue to discuss that question on the other thread.)

The other question you raise is the difficult of implementation for this.  I don't know to what extent state is used in frameworks and sites, so I don't have a good handle on this.  The suggestion to keep a record of pre-transfer state is a good one that seems immediately useful.  However, for the general problem, I posited offline that there are a few cases in which merge strategies could be trivial:

1. Exclusive access means no merges: If a content-based origin has exclusive ownership over a particular store, then merges are trivial.  This is true for a single content-based origin that transitions to a target origin: you can likely assume that the target origin has no pre-existing state before the transfer.  You only have to worry about multiple different content-based origins that write to the same logical store, or a content-based origin that doesn't transfer to the target origin for some reason.  It is quite likely that many transfers will take information for which the target origin has no pre-existing state.

Signer-based origins are more likely to enjoy exclusive access, though we do have to admit the possibility of a single target origin having multiple signing keys and so have multiple sets of state to reconcile, but it might also be possible to have good alignment between access to stores and signing keys to avoid the need for merging.

2. Simple merge strategies: If state is trivial, then very simple merge strategies can be effective with very little effort.  Say that you had a control that set a preference.  That might be a boolean (dark mode on/off), or something more complex.  Last change wins is a perfectly workable strategy in case of conflict.  A less simple strategy is to record discrete changes with timestamps and then taking changes in time order.

Finally, for non-trivial state, my hypothesis is that servers want to record that state server-side, so the question becomes whether any sort of offline modification of that state is desirable anyway.  At the point that a site wants that capability, then yes, this is engineering.  But that is engineering they have to undertake anyway at that point.

[1] The informed consent doctrine is something I think is despicable, but I can't deny the force of law.
[2] Not secure in any real sense, but this method seems to be pretty effective in practice.

On Wed, Mar 25, 2020, at 17:35, Devin Mullins wrote:
> Hi Martin,
> 
> Thanks for the engineering work on this. For others, I've already 
> reviewed this, had some chats with others on the AMP team, and had a 
> few back-and-forths with Martin (he's been very gracious in engaging 
> his time, thank you). I'm trying to estimate the feasibility of this, 
> both for the AMP use-case, and for the non-AMP use-case [1], which we 
> hope to see flourish to the extent that publishers and users desire it, 
> and only impeded by implementation costs where necessary for the common 
> good.
> 
> With the caveat that this is very early feedback, the high-level bit is 
> that, with some modifications, it seems mostly feasible for AMP. 
> However, there are two main downsides:
>  1. worse UX on sites that run A/B experiments
>  2. technical constraints make this harder for publishers to adopt on 
> non-AMP, as compared to SXG
> 
> A lot of details follow. I suggest to rename the subject if you reply 
> to a detail below, in order to reduce cross-talk and make browsing the 
> archive easier.
> 
> Details:
> 
> We would need the display URL to be the attested URL even before 
> transfer. I suspect a flash in the URL bar would be frustrating for 
> both users and publishers. Jeffrey had proposed using signatures for 
> this. I'd suppose UAs could choose to render this with a "may be stale" 
> indicator.
> 
> In terms of publisher implementation feasibility, I think the mostly 
> likely implementation would be for a publisher to produce a 
> Sec-Content-Origin (Sec-CO) response for the current version of the 
> resource iff it matches the Sec-CO request. I suspect that keeping 
> state of past versions of the resource would be infeasible, especially 
> as it couldn't be done purely at the edge; it would require something 
> like a region-wide cache. The value of the above stateless 
> implementation would be inversely proportional to how often the 
> resource changes (e.g. "transient content" [2] such as Related 
> Articles). I think this would be a workable constraint for publishers 
> wishing to publish such bundles, but felt it's worth noting, 
> nonetheless.
> 
> Re: fallback behavior on failed state transfer, distributors would need 
> a way to monitor failure rates. This serves two purposes:
>  1. Automatically detecting errors in the distribution pipeline. This 
> is the same purpose served by 
> https://wicg.github.io/webpackage/loading.html#signed-exchange-report.
>  2. Verifying that, for some fraction of navigations, bundles are 
> meeting AMP's stated UX goals (by using the verified content in the 
> bundle and not potentially arbitrary content after the redirect). This 
> is a dynamic equivalent of what can be done mostly statically with SXG, 
> because the distributor can run the same algorithm as the UA, modulo 
> client/server skew in clocks and root stores. I think this is a 
> trade-off AMP could make. The alternative is that the fallback behavior 
> keeps the content-based origin (CBO).
> 
> There is a possible vector for user ID transfer from distributor to 
> publisher; filed as 
> https://github.com/martinthomson/wpack-content/issues/1.
> 
> This limits the ability to run session-sticky experiments (in 
> https://amp.dev/documentation/components/amp-experiment/ and many 
> non-AMP frameworks). The publisher has the following options:
>  1. Delay rendering content under experiment until after state 
> transfer. This hurts UX for the sake of experimentation, creating a 
> trade-off that otherwise doesn't exist on origin (modulo ease of 
> implementation).
>  2. Generate a client-side UUID before state transfer, and join it with 
> the pre-existing user ID after transfer. This means the session could 
> not include CBO pageviews anywhere except in the first pageview.
>  3. Generate variants of the bundle for each experimental state. The 
> distributor chooses which bundle to send to the user (based on what? 
> not sure). For a page with M independent experiments each with N arms, 
> that's N^M variants. For large enough M and N, this is likely both 
> infeasible for publishers and martinthomson/wpack-content#1. (But I 
> need some guidance from experts on typical ranges for M and N.)
> 
> Perhaps this is a need the UA could address. A straw proposal -- a 
> selectExperimentArm() API that:
>  1. only exposes as many bits as the UA deems OK, and
>  2. exposes different bits to different attested origins, so they can't 
> be joined
> On the one hand, perhaps this is too much scope for wpack. On the 
> other, it addresses a problem that is somewhat particular to CBOs.
> 
> This impedes content management (e.g. dialogs for GDPR and CCPA), 
> compared to SXG. The publisher has a few choices:
>  1. Delay rendering the dialog under after transfer. I think this would 
> cause layout shift (https://web.dev/cls/).
>  2. Render the dialog, and risk discovering that it's already been 
> acknowledged. At that point:
>  a. Hide it, causing layout shift.
>  b. Leave it there. This is way outside my domain expertise, but ISTR 
> there are some rules or at least best practices wrt when it's okay to 
> re-show an opt-out dialog.
> One of these options may be an okay trade-off; I'm not sure.
> 
> Analytics providers (via 
> https://amp.dev/documentation/components/amp-analytics/ and many 
> non-AMP libraries) would have to do one of two things:
>  1. Delay pingback until after transfer. This risks a lower fraction of 
> successful pingbacks, as e.g. the user might close the tab before 
> transfer.
>  2. Generate a client-side UUID, send pingback before transfer, and 
> send a follow-up after transfer to join with pre-existing user ID. This 
> captures those otherwise lost pingbacks pseudonymously (anonymously?), 
> though it requires server-side work on the pingback endpoint.
> 
> For the above client-side mitigations, two feature requests seem 
> necessary:
>  1. Store the transfer metadata (e.g. indexedDB renames by time) 
> somewhere more permanent. For various reasons the event handler may not 
> fire or finish, and it would be good to later scan for stragglers to 
> merge or expunge.
>  2. DOM API to wait for transfer to complete.
> 
> Some subresources are ACLs by Origin header (e.g. paid fonts). Likely 
> mitigation is to defer loading until transfer, with a timeout whose 
> duration depends on the publisher's font-display preference. (Martin 
> suggests an optimization where the encrypted subresource is loaded 
> early, and only the decryption key after transfer.)
> 
> Last but not least, I am concerned that this is a bar that is 
> disproportionately difficult for non-AMP publishers to meet. The above 
> client-side mitigations are possible, but quite a bit of work.
>  1. For frameworks like AMP that strongly encourage publishers to use a 
> rolling release, it is possible to upgrade many use-cases in the wild 
> with minimal publisher effort.
>  2. For other frameworks, it may be possible to provide this support, 
> but the requirement to upgrade versions may limit applicability. 
> Because upgrades occur less often, they often require more work.
>  3. For custom JS that manages state, authors will need to modify it to 
> move or copy state into indexedDB, and handle transfer events and merge 
> conflicts.
> 
> From a "minimal publisher effort" perspective, I am especially 
> interested in making it possible to have as close to a turnkey solution 
> as is reasonable, for instance at the level of the CDN or CMS. If such 
> a solution exists, sites would likely contain a hybrid of all three 
> cases above, and thus want to opt into such support incrementally. How 
> should a CDN determine which pages to provide Sec-CO responses for?
>  1. URL patterns are the simplest answer, but I suspect hard for 
> publishers to create and maintain with minimal false negatives and 
> positives.
>  2. Would it be helpful to thread an "I support state transfer" 
> annotation for some portion of the build journey, from individual JS 
> function all the way to bundle? Would folks use this?
> 
> I don't have a good solution to the issue of non-AMP developer cost, 
> and thus fear this is a solution that would feature a disproportionate 
> amount of AMP pages compared to the background distribution of the web. 
> But I'm hopeful that other JS framework developers can chime in with 
> respect to feasibility of support for CBO and state transfer.
> 
> I'm leaving out some of Martin's replies to the above; no offense 
> intended. It took me long enough just to summarize the above, and I'd 
> better send it early enough so at least a few people have time to read 
> it before the meeting.
> 
> Thanks,
> Devin
> 
> [1] 
> https://blog.amp.dev/2019/05/22/privacy-preserving-instant-loading-for-all-web-content/
> [2] 
> http://www.seobythesea.com/2011/12/how-google-might-identify-transient-content-on-webpages/
> 
> On Mon, Mar 23, 2020 at 5:35 PM Martin Thomson <mt@lowentropy.net> wrote:
> > Ted's note prompted me to send a much-belated announcement (sorry folks, I forgot).
> > 
> >  The draft is here:
> > https://tools.ietf.org/html/draft-thomson-wpack-content-origin-00
> > 
> >  A nicer version here:
> > https://martinthomson.github.io/wpack-content/draft-thomson-wpack-content-origin.html
> > 
> >  This approach could a dramatically different approach to addressing the use cases set out in our charter.
> > 
> >  In short, this aims to address the core question of how offline content might *ultimately* be attributed to a web origin in a fundamentally different way. There are two key concepts:
> > 
> >  1. Content is given its own origin, using a new system for identification.
> > 
> >  2. A target origin can "accept" content and state from one of these new origins.
> > 
> >  There are a lot of details here (read the draft), but the major advantage I see is that you don't have to make an offline decision about authority, and that means you can be offline for much longer (lifting the 7 day limit).
> > 
> >  What it does have in common with signed exchanges approach is the need for a bundling format, but in its current form it is less dependent on the details of the format. That might allow that to be simpler, but I'm sure that the need to mint new identifier types will more than make up for any slack there.
> > 
> >  The draft is quite rough. I'm sure that it has the remnants of a few bad ideas still hanging around. Ask questions if you think something is unclear.
> > 
> >  _______________________________________________
> >  Wpack mailing list
> > Wpack@ietf.org
> > https://www.ietf.org/mailman/listinfo/wpack