Re: [ldapext] Content Synchronization Operation + Transactions
Petr Spacek <pspacek@redhat.com> Wed, 18 November 2015 17:16 UTC
Return-Path: <pspacek@redhat.com>
X-Original-To: ldapext@ietfa.amsl.com
Delivered-To: ldapext@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BDF1E1A8A63 for <ldapext@ietfa.amsl.com>; Wed, 18 Nov 2015 09:16:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.486
X-Spam-Level:
X-Spam-Status: No, score=-7.486 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.585, SPF_HELO_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id B5V-K5d_iSbd for <ldapext@ietfa.amsl.com>; Wed, 18 Nov 2015 09:16:14 -0800 (PST)
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 20B901A8A27 for <ldapext@ietf.org>; Wed, 18 Nov 2015 09:16:14 -0800 (PST)
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id 9ECF4C0BB2AA for <ldapext@ietf.org>; Wed, 18 Nov 2015 17:16:13 +0000 (UTC)
Received: from pspacek.brq.redhat.com (unused [10.34.128.7] (may be forged)) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id tAIHGBIv023271 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <ldapext@ietf.org>; Wed, 18 Nov 2015 12:16:13 -0500
To: ldapext@ietf.org
References: <556EAE47.10406@redhat.com> <55913519.3040302@redhat.com> <559138C2.6060904@highlandsun.com> <559D4274.2000400@redhat.com>
From: Petr Spacek <pspacek@redhat.com>
Organization: Red Hat
Message-ID: <564CB25B.9020702@redhat.com>
Date: Wed, 18 Nov 2015 18:16:11 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0
MIME-Version: 1.0
In-Reply-To: <559D4274.2000400@redhat.com>
Content-Type: text/plain; charset="windows-1252"
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
Archived-At: <http://mailarchive.ietf.org/arch/msg/ldapext/8Ou3bx4bEunZRdAWMLAZjEX-7ZM>
Subject: Re: [ldapext] Content Synchronization Operation + Transactions
X-BeenThere: ldapext@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: LDAP Extension Working Group <ldapext.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ldapext>, <mailto:ldapext-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ldapext/>
List-Post: <mailto:ldapext@ietf.org>
List-Help: <mailto:ldapext-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ldapext>, <mailto:ldapext-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 18 Nov 2015 17:16:21 -0000
On 8.7.2015 17:32, Petr Spacek wrote: > On 29.6.2015 14:23, Howard Chu wrote: >> Petr Spacek wrote: >>> On 3.6.2015 09:35, Petr Spacek wrote: >>>> Hello, >>>> >>>> it seems to me that LDAP Transactions (RFC 5805) are not perfectly integrated >>>> into Content Synchronization Operation (RFC 4533). >>>> >>>> Mainly the client has no idea what changes were part of single transaction. As >>>> a result, client cannot replicate transactions reliably, especially when >>>> connection drops in the middle of persist phase of Content Synchronization >>>> Operation. >>>> >>>> Also some clients could use the information where LDAP transaction >>>> started/ended for application-level transactions (if the application uses the >>>> data in real-time) or possibly some optimizations (if some application-level >>>> work needs to be done after each group of updates). >>>> >>>> I'm going to sketch -00 draft which will attempt to address this, probably by >>>> adding new messages to indicate where transaction started and ended. >>>> Naturally, this makes sense only for persist phase. >>> >>> The 00 version of the draft is now available. I would really appreciate any >>> feedback! >>> >>> Particularly, I'm not very sure that "transaction identifier" in "End >>> Transaction Notification Message" is the best idea. >> >> Assuming a unique transaction identifier per transaction, I see no reason you >> couldn't allow interleaved transactions during persist stage. (Well, you would >> need to send the txnID along with every message, so that may be excessive >> overhead.) In OpenLDAP transactions would not be interleaved at execution time >> anyway so perhaps it's a non-issue. > > Main reason for using extra messages as transaction boundaries was ease of > implementation on the client side. It can be quite awful to deal with multiple > concurrent transactions on the application level. The assumption here is that > transaction commit should be quite quick so serialization should not add a big > performance penalty (SyncRepl serializes all changes anyway). After LDAPCon I decided to wipe out transaction identifiers from the draft, so transaction boundaries are marked by empty Start & End Transaction Notification Messages. This would be enough for making SyncRepl useful with transactions in applications I care about. I will think about writing separate draft for 'ignore own writes', but today I think that it belongs to a separate document. > Main motivation why tnxID was exposed to SyncRepl users was ability to detect > own writes. Maybe it is better to hide tnxID completely, use empty Start/End > Transation Notification messages, and define another control for 'ignore own > writes when doing SyncRepl'. > > What do you think? I'm not familiar with OpenLDAP internals, so I do not know > how hard it is to implement 'ignore own writes' on server side. I assume that > 'own write' = 'write done by the same BIND DN' but this can be tricky when it > comes to proxying ... > >> On a slightly related note, I intend to submit an extension to RFC5805 adding >> a Prepare Transaction request. This would enable 2 phase commit, allowing us >> to support transactions spanning multiple DSAs. With OpenLDAP's back-ldap, >> back-meta, back-relay, and glue, the capability is sorely needed. > > Good idea! > > Would it be appropriate & possible to clarify what level of 'isolation' is > assumed in RFC5805, possibly in the same draft? I guess that the > under-specification will be more visible with 2 phase commit. > > I think that this property is very important and its under-specification can > cause incompatibilities if other servers decide to implement different levels > of isolation. Howard, are you going to clarify isolation level in the new drafts related to transactions? I think that it would be really useful. Thank you for your time! Petr Spacek -------- Forwarded Message -------- Subject: New Version Notification for draft-spacek-ldapext-syncrepl-transaction-01.txt Date: Wed, 18 Nov 2015 09:02:37 -0800 From: internet-drafts@ietf.org To: Petr Spacek <pspacek@redhat.com> A new version of I-D, draft-spacek-ldapext-syncrepl-transaction-01.txt has been successfully submitted by Petr Spacek and posted to the IETF repository. Name: draft-spacek-ldapext-syncrepl-transaction Revision: 01 Title: The Lightweight Directory Access Protocol (LDAP) Content Synchronization Operation with Transactions Document date: 2015-11-18 Group: Individual Submission Pages: 5 URL: https://www.ietf.org/internet-drafts/draft-spacek-ldapext-syncrepl-transaction-01.txt Status: https://datatracker.ietf.org/doc/draft-spacek-ldapext-syncrepl-transaction/ Htmlized: https://tools.ietf.org/html/draft-spacek-ldapext-syncrepl-transaction-01 Diff: https://www.ietf.org/rfcdiff?url2=draft-spacek-ldapext-syncrepl-transaction-01 Abstract: This document specifies LDAP Control which extends the persist stage of the Content Synchronization Operation with information about LDAP transaction boundaries. This information can be used to support application-level transactions or for application-level optimizations.
- [ldapext] Content Synchronization Operation + Tra… Petr Spacek
- Re: [ldapext] Content Synchronization Operation +… Petr Spacek
- Re: [ldapext] Content Synchronization Operation +… Howard Chu
- Re: [ldapext] Content Synchronization Operation +… Petr Spacek
- Re: [ldapext] Content Synchronization Operation +… Petr Spacek