Re: [ldapext] Content Synchronization Operation + Transactions

Petr Spacek <> Wed, 18 November 2015 17:16 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id BDF1E1A8A63 for <>; Wed, 18 Nov 2015 09:16:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -7.486
X-Spam-Status: No, score=-7.486 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.585, SPF_HELO_PASS=-0.001] autolearn=ham
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id B5V-K5d_iSbd for <>; Wed, 18 Nov 2015 09:16:14 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 20B901A8A27 for <>; Wed, 18 Nov 2015 09:16:14 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTPS id 9ECF4C0BB2AA for <>; Wed, 18 Nov 2015 17:16:13 +0000 (UTC)
Received: from (unused [] (may be forged)) by (8.14.4/8.14.4) with ESMTP id tAIHGBIv023271 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <>; Wed, 18 Nov 2015 12:16:13 -0500
References: <> <> <> <>
From: Petr Spacek <>
Organization: Red Hat
Message-ID: <>
Date: Wed, 18 Nov 2015 18:16:11 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on
Archived-At: <>
Subject: Re: [ldapext] Content Synchronization Operation + Transactions
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: LDAP Extension Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 18 Nov 2015 17:16:21 -0000

On 8.7.2015 17:32, Petr Spacek wrote:
> On 29.6.2015 14:23, Howard Chu wrote:
>> Petr Spacek wrote:
>>> On 3.6.2015 09:35, Petr Spacek wrote:
>>>> Hello,
>>>> it seems to me that LDAP Transactions (RFC 5805) are not perfectly integrated
>>>> into Content Synchronization Operation (RFC 4533).
>>>> Mainly the client has no idea what changes were part of single transaction. As
>>>> a result, client cannot replicate transactions reliably, especially when
>>>> connection drops in the middle of persist phase of Content Synchronization
>>>> Operation.
>>>> Also some clients could use the information where LDAP transaction
>>>> started/ended for application-level transactions (if the application uses the
>>>> data in real-time) or possibly some optimizations (if some application-level
>>>> work needs to be done after each group of updates).
>>>> I'm going to sketch -00 draft which will attempt to address this, probably by
>>>> adding new messages to indicate where transaction started and ended.
>>>> Naturally, this makes sense only for persist phase.
>>> The 00 version of the draft is now available. I would really appreciate any
>>> feedback!
>>> Particularly, I'm not very sure that "transaction identifier" in "End
>>> Transaction Notification Message" is the best idea.
>> Assuming a unique transaction identifier per transaction, I see no reason you
>> couldn't allow interleaved transactions during persist stage. (Well, you would
>> need to send the txnID along with every message, so that may be excessive
>> overhead.) In OpenLDAP transactions would not be interleaved at execution time
>> anyway so perhaps it's a non-issue.
> Main reason for using extra messages as transaction boundaries was ease of
> implementation on the client side. It can be quite awful to deal with multiple
> concurrent transactions on the application level. The assumption here is that
> transaction commit should be quite quick so serialization should not add a big
> performance penalty (SyncRepl serializes all changes anyway).

After LDAPCon I decided to wipe out transaction identifiers from the draft, so
transaction boundaries are marked by empty Start & End Transaction
Notification Messages.

This would be enough for making SyncRepl useful with transactions in
applications I care about. I will think about writing separate draft for
'ignore own writes', but today I think that it belongs to a separate document.

> Main motivation why tnxID was exposed to SyncRepl users was ability to detect
> own writes. Maybe it is better to hide tnxID completely, use empty Start/End
> Transation Notification messages, and define another control for 'ignore own
> writes when doing SyncRepl'.
> What do you think? I'm not familiar with OpenLDAP internals, so I do not know
> how hard it is to implement 'ignore own writes' on server side. I assume that
> 'own write' = 'write done by the same BIND DN' but this can be tricky when it
> comes to proxying ...
>> On a slightly related note, I intend to submit an extension to RFC5805 adding
>> a Prepare Transaction request. This would enable 2 phase commit, allowing us
>> to support transactions spanning multiple DSAs. With OpenLDAP's back-ldap,
>> back-meta, back-relay, and glue, the capability is sorely needed.
> Good idea!
> Would it be appropriate & possible to clarify what level of 'isolation' is
> assumed in RFC5805, possibly in the same draft? I guess that the
> under-specification will be more visible with 2 phase commit.
> I think that this property is very important and its under-specification can
> cause incompatibilities if other servers decide to implement different levels
> of isolation.

Howard, are you going to clarify isolation level in the new drafts related to
transactions? I think that it would be really useful.

Thank you for your time!

Petr Spacek

-------- Forwarded Message --------
Subject: New Version Notification for
Date: Wed, 18 Nov 2015 09:02:37 -0800
To: Petr Spacek <>

A new version of I-D, draft-spacek-ldapext-syncrepl-transaction-01.txt
has been successfully submitted by Petr Spacek and posted to the
IETF repository.

Name:		draft-spacek-ldapext-syncrepl-transaction
Revision:	01
Title:		The Lightweight Directory Access Protocol (LDAP) Content
Synchronization Operation with Transactions
Document date:	2015-11-18
Group:		Individual Submission
Pages:		5

   This document specifies LDAP Control which extends the persist stage
   of the Content Synchronization Operation with information about LDAP
   transaction boundaries.  This information can be used to support
   application-level transactions or for application-level