[Atp] Re: Repository and Synchronization Draft Text

[Phil Schleihauf <[e-mail]>]

> I'm not sure I understand this part. If a consumer has buffered commit
messages for revs C and D, and it fetches a full repo and gets B, that
should work out fine (because commit C will reference B in the 'since'
field). If it got anything earlier it would fail. If it got C or later it
would work.

Yeah sorry, I meant the case where the consumer buffers C and D, wasn't
aware of rev B, receives repo data at A, and fails. (eg. Tap drops commits
when it puts a repo in "desynchronized" state, and starts buffering after
the transition to "synchronizing", so imagine it drops B before it tries to
resync).

In the simple-upstream case, I suppose your simple caching scenario holds:
even with different transports (websocket vs http) a consumer that begins
buffering before emitting its getRepo request, to a server that updates its
known repo rev visibly-to-http-endpoints before emitting it on the
firehose, won't race. I think.

But, that's for the boring case --

> And further this is also an issue if alternative mirrors are being used.
Eg, sources which are "siblings" as opposed to "upstream" in the data flow.

I hope these are first-class use-cases for atp too. We have all this nice
authentication stuff, and unreliable PDSes (and no current caching relays),
and the desirable option for multiple-upstream (multi-relay, relay + some
PDSes, ...) graphs, etc.

> I agree that the current expected behavior is that the consumer would
mark the repo with status "desynchronized" (per Synchronization 2.1) and
would try to re-synchronize again later.

(i think that should be S4.6?)

(This might be getting into implementation-level details instead of spec
concerns) if a first-buffered-commit-after-resync has a `since` parameter
*after* the just-resynced data's rev, then it probably lost the race. A
consumer could immediately retry the resync (retaining the existing
buffered commits), knowing the minimum rev it actually needs, needs for it
to succeed, to avoid restarting the whole race again later.

> a new optional query parameter to the HTTP endpoint for fetching full
repos

I really like this direction! However, I think it runs into the same
awareness-of-rev-B thing -- can a consumer know which rev is its minimum at
that point?

One way out of some of this might be for consumers to track the
latest-seen-rev of (valid, authenticated) firehose commits, while dropping
them, while that repo is in desynchronized state. It would be able to make
that request, or poll, or even just abandon a resync early if it's with
too-old-data.


On Tue, Jun 30, 2026 at 10:07 PM Bryan Newbold <[e-mail]> wrote:

> On Sat, Jun 27, 2026 at 8:24 AM Phil Schleihauf <[e-mail]> wrote:
>
>> Regarding Sync 4.6, Resynchronization,
>>
>> There is a data race between fetching the repository structure and
>> buffering commits to replay after completing synchronization. Suppose a
>> producer emits revs for a repo
>>
>> A => B => C => D
>>
>> A consumer realizes it needs to resynchronize upon observing rev B. It
>> then requests the repository data structure. There are no guarantees
>> provided about which version it will get. It buffers events for revs C + D
>> for replay.
>>
>> If it receives repository data from rev B, C, or D, it can successfully
>> process the buffered (and future) events. But, it might receive data from
>> rev A, in which case, processing the buffered event for rev C will fail.
>>
>> This is rare in practice today, because upstream producers (that i'm
>> aware of) in the main production atproto network always redirect resync
>> requests to the canonical hosts, which presumably always respond with the
>> latest rev they have. Latency introduced by relays works in our favour
>> against the race. But, the this section explicitly carves out scenarios
>> where latency in the other direction should be expected:
>>
>> > Direct upstreams MAY coalesce and cache snapshot requests, or redirect
>> consumers to alternative sources
>>
>> A cached snapshot or public mirror service could return a snapshot at rev
>> A when a consumer desynchronizes at rev B, causing the resync to fail.
>>
>
> I think that this would not be an issue in the case of simple caching. If
> a producer (eg, a relay) is retaining a cache of the full repository CAR
> file for an account, it would presumably invalidate that cache as soon as
> it receives notice of a new commit revision from upstream. As long as the
> cache is invalidated before the commit (or sync) message is re-emitted to
> downstream consumers, I think there is no problem.
>
> I think you are correct in the case of coalescing: in that scenario the
> timing is spread out and the result of an earlier upstream request could be
> returned to a later downstream consumer.
>
> And further this is also an issue if alternative mirrors are being used.
> Eg, sources which are "siblings" as opposed to "upstream" in the data flow.
>
>
>> Further, in this example, the consumer *knows* that it needs a snapshot
>> for at least Rev B for the resync to succeed, but that's not necessarily
>> the case. For a "scheduled resync" (eg., if the first one failed for any
>> reason, and the consumer applies some arbitrary backoff delay), the
>> consumer might not know which "minimum rev" it needs in order to succeed on
>> the next rev following resync.
>>
>> ie., it begins resync, buffering revs C and D, but has no idea about the
>> existence of rev B.
>>
>
> I'm not sure I understand this part. If a consumer has buffered commit
> messages for revs C and D, and it fetches a full repo and gets B, that
> should work out fine (because commit C will reference B in the 'since'
> field). If it got anything earlier it would fail. If it got C or later it
> would work.
>
>
>> For all of this, the consumer's handling probably boils down to "just
>> retry". While the aggregate rate of events on the network may be high, the
>> per-repo interval is expected to be low enough to eventually succeed, even
>> in the presence of slightly stale data from the resync process. Maybe some
>> of this is worth being explicit about in the spec?
>>
>
> I agree that the current expected behavior is that the consumer would mark
> the repo with status "desynchronized" (per Synchronization 2.1) and would
> try to re-synchronize again later.
>
> It seems like some of the issues you are raising could be addressed by
> adding a new optional query parameter to the HTTP endpoint for fetching
> full repos (aka, /xrpc/com.atproto.sync.getRepo, though the current draft
> text doesn't name that endpoint). If that endpoint took a query param
> 'minimumRevision' (just spitballing), the upstream could return an error if
> it didn't have access to a full CAR with that version or higher. Or, the
> consumer could call the 'com.atproto.sync.getRepoStatus' endpoint
> repeatedly until the returned 'rev' is greater or equal to what it is
> expecting (this would be cheaper than fetching the full CAR).
>
> I think this probably warrants further discussion.
>
> --bryan
>