Re: [ldapext] Content Synchronization Operation + Transactions

Petr Spacek <pspacek@redhat.com> Wed, 18 November 2015 17:16 UTC

Return-Path: <pspacek@redhat.com>
X-Original-To: ldapext@ietfa.amsl.com
Delivered-To: ldapext@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BDF1E1A8A63 for <ldapext@ietfa.amsl.com>; Wed, 18 Nov 2015 09:16:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.486
X-Spam-Level:
X-Spam-Status: No, score=-7.486 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.585, SPF_HELO_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id B5V-K5d_iSbd for <ldapext@ietfa.amsl.com>; Wed, 18 Nov 2015 09:16:14 -0800 (PST)
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 20B901A8A27 for <ldapext@ietf.org>; Wed, 18 Nov 2015 09:16:14 -0800 (PST)
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id 9ECF4C0BB2AA for <ldapext@ietf.org>; Wed, 18 Nov 2015 17:16:13 +0000 (UTC)
Received: from pspacek.brq.redhat.com (unused [10.34.128.7] (may be forged)) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id tAIHGBIv023271 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <ldapext@ietf.org>; Wed, 18 Nov 2015 12:16:13 -0500
To: ldapext@ietf.org
References: <556EAE47.10406@redhat.com> <55913519.3040302@redhat.com> <559138C2.6060904@highlandsun.com> <559D4274.2000400@redhat.com>
From: Petr Spacek <pspacek@redhat.com>
Organization: Red Hat
Message-ID: <564CB25B.9020702@redhat.com>
Date: Wed, 18 Nov 2015 18:16:11 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0
MIME-Version: 1.0
In-Reply-To: <559D4274.2000400@redhat.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
Archived-At: <http://mailarchive.ietf.org/arch/msg/ldapext/8Ou3bx4bEunZRdAWMLAZjEX-7ZM>
Subject: Re: [ldapext] Content Synchronization Operation + Transactions
X-BeenThere: ldapext@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: LDAP Extension Working Group <ldapext.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ldapext>, <mailto:ldapext-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ldapext/>
List-Post: <mailto:ldapext@ietf.org>
List-Help: <mailto:ldapext-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ldapext>, <mailto:ldapext-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 18 Nov 2015 17:16:21 -0000

On 8.7.2015 17:32, Petr Spacek wrote:
> On 29.6.2015 14:23, Howard Chu wrote:
>> Petr Spacek wrote:
>>> On 3.6.2015 09:35, Petr Spacek wrote:
>>>> Hello,
>>>>
>>>> it seems to me that LDAP Transactions (RFC 5805) are not perfectly integrated
>>>> into Content Synchronization Operation (RFC 4533).
>>>>
>>>> Mainly the client has no idea what changes were part of single transaction. As
>>>> a result, client cannot replicate transactions reliably, especially when
>>>> connection drops in the middle of persist phase of Content Synchronization
>>>> Operation.
>>>>
>>>> Also some clients could use the information where LDAP transaction
>>>> started/ended for application-level transactions (if the application uses the
>>>> data in real-time) or possibly some optimizations (if some application-level
>>>> work needs to be done after each group of updates).
>>>>
>>>> I'm going to sketch -00 draft which will attempt to address this, probably by
>>>> adding new messages to indicate where transaction started and ended.
>>>> Naturally, this makes sense only for persist phase.
>>>
>>> The 00 version of the draft is now available. I would really appreciate any
>>> feedback!
>>>
>>> Particularly, I'm not very sure that "transaction identifier" in "End
>>> Transaction Notification Message" is the best idea.
>>
>> Assuming a unique transaction identifier per transaction, I see no reason you
>> couldn't allow interleaved transactions during persist stage. (Well, you would
>> need to send the txnID along with every message, so that may be excessive
>> overhead.) In OpenLDAP transactions would not be interleaved at execution time
>> anyway so perhaps it's a non-issue.
> 
> Main reason for using extra messages as transaction boundaries was ease of
> implementation on the client side. It can be quite awful to deal with multiple
> concurrent transactions on the application level. The assumption here is that
> transaction commit should be quite quick so serialization should not add a big
> performance penalty (SyncRepl serializes all changes anyway).

After LDAPCon I decided to wipe out transaction identifiers from the draft, so
transaction boundaries are marked by empty Start & End Transaction
Notification Messages.

This would be enough for making SyncRepl useful with transactions in
applications I care about. I will think about writing separate draft for
'ignore own writes', but today I think that it belongs to a separate document.


> Main motivation why tnxID was exposed to SyncRepl users was ability to detect
> own writes. Maybe it is better to hide tnxID completely, use empty Start/End
> Transation Notification messages, and define another control for 'ignore own
> writes when doing SyncRepl'.
> 
> What do you think? I'm not familiar with OpenLDAP internals, so I do not know
> how hard it is to implement 'ignore own writes' on server side. I assume that
> 'own write' = 'write done by the same BIND DN' but this can be tricky when it
> comes to proxying ...
> 
>> On a slightly related note, I intend to submit an extension to RFC5805 adding
>> a Prepare Transaction request. This would enable 2 phase commit, allowing us
>> to support transactions spanning multiple DSAs. With OpenLDAP's back-ldap,
>> back-meta, back-relay, and glue, the capability is sorely needed.
> 
> Good idea!
> 
> Would it be appropriate & possible to clarify what level of 'isolation' is
> assumed in RFC5805, possibly in the same draft? I guess that the
> under-specification will be more visible with 2 phase commit.
> 
> I think that this property is very important and its under-specification can
> cause incompatibilities if other servers decide to implement different levels
> of isolation.

Howard, are you going to clarify isolation level in the new drafts related to
transactions? I think that it would be really useful.

Thank you for your time!

Petr Spacek



-------- Forwarded Message --------
Subject: New Version Notification for
draft-spacek-ldapext-syncrepl-transaction-01.txt
Date: Wed, 18 Nov 2015 09:02:37 -0800
From: internet-drafts@ietf.org
To: Petr Spacek <pspacek@redhat.com>


A new version of I-D, draft-spacek-ldapext-syncrepl-transaction-01.txt
has been successfully submitted by Petr Spacek and posted to the
IETF repository.

Name:		draft-spacek-ldapext-syncrepl-transaction
Revision:	01
Title:		The Lightweight Directory Access Protocol (LDAP) Content
Synchronization Operation with Transactions
Document date:	2015-11-18
Group:		Individual Submission
Pages:		5
URL:
https://www.ietf.org/internet-drafts/draft-spacek-ldapext-syncrepl-transaction-01.txt
Status:
https://datatracker.ietf.org/doc/draft-spacek-ldapext-syncrepl-transaction/
Htmlized:
https://tools.ietf.org/html/draft-spacek-ldapext-syncrepl-transaction-01
Diff:
https://www.ietf.org/rfcdiff?url2=draft-spacek-ldapext-syncrepl-transaction-01

Abstract:
   This document specifies LDAP Control which extends the persist stage
   of the Content Synchronization Operation with information about LDAP
   transaction boundaries.  This information can be used to support
   application-level transactions or for application-level
   optimizations.