Re: [ldapext] Content Synchronization Operation + Transactions

Petr Spacek <pspacek@redhat.com> Wed, 08 July 2015 15:32 UTC

Return-Path: <pspacek@redhat.com>
X-Original-To: ldapext@ietfa.amsl.com
Delivered-To: ldapext@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2BDFC1A004A for <ldapext@ietfa.amsl.com>; Wed, 8 Jul 2015 08:32:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.012
X-Spam-Level:
X-Spam-Status: No, score=-5.012 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, RCVD_IN_DNSWL_HI=-5, SPF_HELO_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VqL62VeR_Mmi for <ldapext@ietfa.amsl.com>; Wed, 8 Jul 2015 08:32:07 -0700 (PDT)
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7BA061A0041 for <ldapext@ietf.org>; Wed, 8 Jul 2015 08:32:07 -0700 (PDT)
Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (Postfix) with ESMTPS id 0E8D0B5B82; Wed, 8 Jul 2015 15:32:07 +0000 (UTC)
Received: from pspacek.brq.redhat.com (ovpn-204-45.brq.redhat.com [10.40.204.45]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t68FW4n6018245 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO); Wed, 8 Jul 2015 11:32:06 -0400
Message-ID: <559D4274.2000400@redhat.com>
Date: Wed, 08 Jul 2015 17:32:04 +0200
From: Petr Spacek <pspacek@redhat.com>
Organization: Red Hat
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0
MIME-Version: 1.0
To: Howard Chu <hyc@highlandsun.com>, ldapext@ietf.org
References: <556EAE47.10406@redhat.com> <55913519.3040302@redhat.com> <559138C2.6060904@highlandsun.com>
In-Reply-To: <559138C2.6060904@highlandsun.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
Archived-At: <http://mailarchive.ietf.org/arch/msg/ldapext/u49JK_SDotmbvJkK8ZltjwS-DlU>
Subject: Re: [ldapext] Content Synchronization Operation + Transactions
X-BeenThere: ldapext@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: LDAP Extension Working Group <ldapext.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ldapext>, <mailto:ldapext-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ldapext/>
List-Post: <mailto:ldapext@ietf.org>
List-Help: <mailto:ldapext-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ldapext>, <mailto:ldapext-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Jul 2015 15:32:09 -0000

On 29.6.2015 14:23, Howard Chu wrote:
> Petr Spacek wrote:
>> On 3.6.2015 09:35, Petr Spacek wrote:
>>> Hello,
>>>
>>> it seems to me that LDAP Transactions (RFC 5805) are not perfectly integrated
>>> into Content Synchronization Operation (RFC 4533).
>>>
>>> Mainly the client has no idea what changes were part of single transaction. As
>>> a result, client cannot replicate transactions reliably, especially when
>>> connection drops in the middle of persist phase of Content Synchronization
>>> Operation.
>>>
>>> Also some clients could use the information where LDAP transaction
>>> started/ended for application-level transactions (if the application uses the
>>> data in real-time) or possibly some optimizations (if some application-level
>>> work needs to be done after each group of updates).
>>>
>>> I'm going to sketch -00 draft which will attempt to address this, probably by
>>> adding new messages to indicate where transaction started and ended.
>>> Naturally, this makes sense only for persist phase.
>>
>> The 00 version of the draft is now available. I would really appreciate any
>> feedback!
>>
>> Particularly, I'm not very sure that "transaction identifier" in "End
>> Transaction Notification Message" is the best idea.
> 
> Assuming a unique transaction identifier per transaction, I see no reason you
> couldn't allow interleaved transactions during persist stage. (Well, you would
> need to send the txnID along with every message, so that may be excessive
> overhead.) In OpenLDAP transactions would not be interleaved at execution time
> anyway so perhaps it's a non-issue.

Main reason for using extra messages as transaction boundaries was ease of
implementation on the client side. It can be quite awful to deal with multiple
concurrent transactions on the application level. The assumption here is that
transaction commit should be quite quick so serialization should not add a big
performance penalty (SyncRepl serializes all changes anyway).

Main motivation why tnxID was exposed to SyncRepl users was ability to detect
own writes. Maybe it is better to hide tnxID completely, use empty Start/End
Transation Notification messages, and define another control for 'ignore own
writes when doing SyncRepl'.

What do you think? I'm not familiar with OpenLDAP internals, so I do not know
how hard it is to implement 'ignore own writes' on server side. I assume that
'own write' = 'write done by the same BIND DN' but this can be tricky when it
comes to proxying ...

> On a slightly related note, I intend to submit an extension to RFC5805 adding
> a Prepare Transaction request. This would enable 2 phase commit, allowing us
> to support transactions spanning multiple DSAs. With OpenLDAP's back-ldap,
> back-meta, back-relay, and glue, the capability is sorely needed.

Good idea!

Would it be appropriate & possible to clarify what level of 'isolation' is
assumed in RFC5805, possibly in the same draft? I guess that the
under-specification will be more visible with 2 phase commit.

I think that this property is very important and its under-specification can
cause incompatibilities if other servers decide to implement different levels
of isolation.

Thank you!

-- 
Petr Spacek  @  Red Hat