Re: [Tzdist] Ticket #22

[Cyrus Daboo <[e-mail]>]

Hi Lester,

--On August 27, 2014 at 5:41:57 PM +0100 Lester Caine <[e-mail]> 
wrote:

> OK this is filling in my mistakes ... I'd not considered relying only on
> the ETag caching given the qualification on it's use in the get
> definition? Re-reading it of cause , there SHOULD only be one timezone
> specified so under what condition would it not?

I think that text is left over from a time when we allowed multiple time 
zones to be returned in a single GET, but we later decided that was 
overkill and broke HTTP caching to an extent we were uncomfortable with. I 
will fix that text.

> I'm more used to data replication which is transactional. I'm normally
> doing all of this in databases. It's just different mindsets. I would
> just be replicating the change rather than resending the whole data
> package. But we are not reliant on a single data source which is what
> ETag is enforcing? Having said that, since the data IS well defined why
> would the generated ETag values be different between servers?

ETags are opaque tokens generated by the HTTP server. There is no 
requirement that different servers use the same method to generate those - 
some may choose an md5 hash, others sha1, and others something completely 
different. So ETags are not meant to represent state across servers - they 
only represent the state of the one resource they are "attached" to.

> The race condition I'm more concerned about here is that of a secondary
> server, but rethinking things, it's more a case here of what is used to
> feed that servers clients? While the server is downloading and
> processing each new get, what is it's response to it's clients? Is each
> new change used as it's downloaded, and IT'S list package changed each
> time, or as I proposed previously does the whole update get processed
> and then made active .. as I would expect the distributor to be doing.
>
> I can see now why you may have such a diverse spread of versions while
> I'm looking to maintain data that is NOT necessarily being used on the
> same machine that the normalizing has been processed on so I NEED a
> stable version identity for other machines to work with! I need
> something that can be stored with the archived data and verified later,
> which is what a properly managed last-modified provides.

By choosing to allow individual time zones to be downloaded, as opposed to 
the entire database all in one go, we do introduce the possibility of race 
conditions. In the case of a secondary server replicating what is on a 
primary, the secondary can achieve some level of integrity by using 
action=list&changedsince,action=get operations up to a point where the 
list+changedsince returns no results.

As far as clients go, they too are subject to this race - they too could 
repeat a list+changedsince immediately after doing a list/get sync to 
verify they have the latest.

>> Now, I do think the current draft is lacking in discussion of the
>> ETag-condition action=get behavior, and I propose adding text to Section
>> 6.3 to cover that. I would also be willing to add a (non-mormative)
>> appendix describing an example client sync procedure such as the one I
>> outlined above, but I really want it to be in the most general of terms
>> because client implementors really need to think carefully about that
>> themselves.
>
> This is not exactly obvious to a new reader :(
> My own experience of caching can be somewhat hit and miss as the routing
> and data proxies change. We have to design the mobile applications
> around here so they work reliably off-net until a mobile connection can
> be restored, and I'm not yet convinced that ETag can work in that
> situation? A local database/webserver maintains operation until it can
> replicate that back to the main service.

ETag together with list+changedsince is the equivalent of what we do in 
CalDAV - there clients also use ETag together with the webdav sync REPORT 
to cache data from the server. Those client are all capable of working in 
an offline mode - and in fact a more complex mode in that changes made on 
the client whilst disconnected need to be sync'd up to the server when they 
re-connect, i.e., a two-way sync is involved. So I am confident what we 
have now is sufficient for a one-way sync from primary to secondary or 
server to client, though I am not confident I have convinced you of that 
fact yet :-)

-- 
Cyrus Daboo