Re: [ldapext] Content Synchronization Operation + Transactions

[Petr Spacek <[e-mail]>]

On 29.6.2015 14:23, Howard Chu wrote:
> Petr Spacek wrote:
>> On 3.6.2015 09:35, Petr Spacek wrote:
>>> Hello,
>>>
>>> it seems to me that LDAP Transactions (RFC 5805) are not perfectly integrated
>>> into Content Synchronization Operation (RFC 4533).
>>>
>>> Mainly the client has no idea what changes were part of single transaction. As
>>> a result, client cannot replicate transactions reliably, especially when
>>> connection drops in the middle of persist phase of Content Synchronization
>>> Operation.
>>>
>>> Also some clients could use the information where LDAP transaction
>>> started/ended for application-level transactions (if the application uses the
>>> data in real-time) or possibly some optimizations (if some application-level
>>> work needs to be done after each group of updates).
>>>
>>> I'm going to sketch -00 draft which will attempt to address this, probably by
>>> adding new messages to indicate where transaction started and ended.
>>> Naturally, this makes sense only for persist phase.
>>
>> The 00 version of the draft is now available. I would really appreciate any
>> feedback!
>>
>> Particularly, I'm not very sure that "transaction identifier" in "End
>> Transaction Notification Message" is the best idea.
> 
> Assuming a unique transaction identifier per transaction, I see no reason you
> couldn't allow interleaved transactions during persist stage. (Well, you would
> need to send the txnID along with every message, so that may be excessive
> overhead.) In OpenLDAP transactions would not be interleaved at execution time
> anyway so perhaps it's a non-issue.

Main reason for using extra messages as transaction boundaries was ease of
implementation on the client side. It can be quite awful to deal with multiple
concurrent transactions on the application level. The assumption here is that
transaction commit should be quite quick so serialization should not add a big
performance penalty (SyncRepl serializes all changes anyway).

Main motivation why tnxID was exposed to SyncRepl users was ability to detect
own writes. Maybe it is better to hide tnxID completely, use empty Start/End
Transation Notification messages, and define another control for 'ignore own
writes when doing SyncRepl'.

What do you think? I'm not familiar with OpenLDAP internals, so I do not know
how hard it is to implement 'ignore own writes' on server side. I assume that
'own write' = 'write done by the same BIND DN' but this can be tricky when it
comes to proxying ...

> On a slightly related note, I intend to submit an extension to RFC5805 adding
> a Prepare Transaction request. This would enable 2 phase commit, allowing us
> to support transactions spanning multiple DSAs. With OpenLDAP's back-ldap,
> back-meta, back-relay, and glue, the capability is sorely needed.

Good idea!

Would it be appropriate & possible to clarify what level of 'isolation' is
assumed in RFC5805, possibly in the same draft? I guess that the
under-specification will be more visible with 2 phase commit.

I think that this property is very important and its under-specification can
cause incompatibilities if other servers decide to implement different levels
of isolation.

Thank you!

-- 
Petr Spacek  @  Red Hat