Re: The LDAP 'list' debate

[[e-mail]]

a few additional comments

1. Many thanks to Colin for re-centering the discussion on the real
issues:
   will LDAP, as it is, enable to design/implement
	truly working/interworking X.500 client applications
	with user acceptable performance?

2. There is a real issue about what kind of operations are needed, and w
	what kind of client algorithm will perform reasonably well
	in an open heterogeneous environment.
	I have some doubts however that a "list" operation might be a 
	solution in the long run.
	In my mind the solution is much more complex than the availability
	of a single operation, but relates to the whole algorithm used
	by (L)DAP clients and possibly on the use of Search Guides.
	There is a need to take into account not only the specific
	characteristics of a given implementation but also the
	administrative restrictions that might be decided by 
	an ADDMD or PRDMD manager (such as allowing searches only
	below a given level in the DIT).



> 
> In the last few weeks there have been a number of messages discussing
> LDAP, one thread inparticular discussed whether LDAP should support the
> X.500 "list" operation.  I don't wish to restart the debate, but do
> think the real issue has been lost in a functionality discussion.
> 
> >From my perspective, the real issue is to make sure the OSI Directory is
> capable of providing a usable and useful service to the end user.  Given
> the current DSA technology I question whether LDAP without "list" can do
> this. 
> 
> DUAs such as the PARADISE DE, and infact many ISODE and UFN based DUAs,
> rely upon being able to perform a search operation at the high levels of
> the DIT.   Irrespective of the reasons why, it is a fact that not all
> DSA implementations are capable of doing this (I will refer to these as
> non-QUIPU DSAs).  This means that if one of the ISODE based DUA connects
> to a non-QUIPU DSA it is unable to perform correctly.  From a users
> perspective this is unacceptable.

Perfectly right.

But, additionaly, as stated above the local manager may also decide (for 
whatever reason) to restrict the right to use "searches" at the upper levels
of the DIT. I may even imagine that at the same level a "list" will
be forbidden or limited to return a very small number of entries.
A commercial provider may by this try to prevent outsiders to get
a listing of its subscribers... This will limit requests to those
entities with an exact knowledge of at least one 1rst level RDN (eg ORG).
This is not completely unrealistic


> 
> There are a number of possible ways to proceed.  One is to persuade the
> implementors of the non-QUIPU DSAs to make search at these high levels
> work.  This is an unreasonable requirement and we are unlikely to 
> suceed - they implement the standard fully and meet the various profiles
> that exist. QUIPU only works because it makes a simplifying assumption.
> 

I am less pessimistic, and QUIPU is a good example which is
certainly directly or indirectly through paying customers
an incentive for all other implementors to make 
improvement in the processing of One-level-searches.
It will take time, but I am ready to bet that the few survivors
in the DSA market a few years from now will indeed all
be able to perform Copy-Allowed-One-Level_Search localy
and thus with good response time.
However, 1) it will take time, 2) there will be adminsitrative
limits. In the end I agree with Colin that it would be foolish
to base DUA algorithm on pure 'search" operations from the top of the tree.

> Another, more practical solution, is to build "implementation" knowledge
> of the situation into the DUAs.  The PARADISE DE interface (full stack
> version) does just this.  When looking below a part of the DIT held by a
> non-QUIPU DSA, it reverts to a "first generation" algorithm and uses
> the list operation.  This then provides a useful service to the end user
> in a multi-implementation universe. 

I fully back the idea of X.500 (L)DAP clients algorithm taking into account
"knowledge", but I will call it "local-DMD" knowledge rather than
implementation (even if it is true that the implementation
capabilities will play an important role for this).
Note that this is totaly in line with the idea (see previous messages
in this list and in [e-mail] list) of the importance of
preparing a "DUA guidelines" type of document.

However, my conclusion is a bit different.
Instead of reverting to "first generation" algorithms, I would suggest that 
all of us have to "design" (logically) the 3rd generation algorithms.
I have to say that I really appreciated the work on DE (by Paul Barker)
which allow DE to be usable in the french branch of the DIT with
many non-QUIPU DSAs. 
However I still have some trouble to understand why when
giving DE my 
	exact Country Name
	exact Organisation Name
	exact OU Name
	and approximate common name
it does not try a read with these attributes that will fail but return
that the firts three attribute values were correct, thus enabling
to start the search at the Department level
  intstead of Org level with the improved DE
  and instead of the country level will original DE
There is no nead for any "list" operation, a read is enough to know
where in the DIT search (in the general meaning) has to start.

A second point is once the "search-start-point" determined to
really use the appropriate algorithm for this, taking into account
the actual environment (type of implementation, allowed operations
aso...).
  if it is detected that the One-Level-Search is available and "cheap"
	then DE/ISODE type of algorithm (sequence of search-one-level)
	is certainly the most appropriate
  if no search allowed or if One-Level-Search "costly", then it may be
	appropriate to try an alternative approach (using list for example
	or using specific entries providing additional information on the
	lower levels of this subtree...)

IN my mind and for other reasons 3rd generation DUAs will certainly
have to read entries such as
	Schema
	Search-Guide
in order to go on with their algorithm.
If Search-Guide is not enough for what is needed for really good
interworking, it will probably be a good idea to define a new object
which will contain all the information enabling a well designed
and implemented DUA to behave with good performance in an open environement.
Eventualy but this is another story, this could be feed back into a
revision of the standard (probably it is stupid, but as an example why not
imagine a "list-entries-of-given-class" operation, if it appears
this would facilitate DUA design).
To remain more realistic I suspect that we have more chances to succeed
with 
	either new object entries describing the subtree specificities
	or new operational attributes in all Non-Leaf-Entries

	
> 
> When porting DE to use LDAP, I understand this functionality has had to
> be droped as LDAP does not support list.  I argue that this is a major
> omission. 
> 

The study of the logs of a non-QUIPU high level DSA clearly confirms
Colin above statement.
	DE is OK with the french master
	whereas many ports will terminate with time-outs and partial results

> Another possible solution (and perhaps somebody who has read the spec
> closer than me can comment if it is possible) is for the LDAP server to
> have this "implementation" knowledge built in, and map the LDAP search
> onto list and reads for the relevant part of the DIT (you can work out
> which parts of the DIT are affected algorithmicaly).
> 

For the reasons given above even if an intelligent LDAP server would do
no harm, I still believe that the "knowledge" information is
important at the client side where the whole algorithm can
be adapted  whereas the server will only be able to  map
a single "client" operation.

> Steve is correct in saying second generation DUAs don't use list, but
> these DUAs have to recognise that some DSAs cannot provide the 
> level of service required, and downgrade their service appropriately.
> Simply not working is not be acceptable to users, we need to ensure LDAP
> is capable of delivering a solution.

Fully agreed

This does not mean that the solution lays in the list operation.

> 
> Parting thought.  This and similar problems will increasingly become
> an issue as the number of non-QUIPU implementations in the DIT increase.
> DUA implementors have to make sure they provide an interface that will
> work across DSA implementation boundaries, otherwise X.500 is not going
> to deliver the level of service required, and will loose out to some
> other technology.  Perhaps, the interoperability service provided by the 
> PARADISE project may be able to help DUA implementors here.

This is indeed one of our goals (I am one of the people in charge of this
interworking forum and platform)
Friday we will have a discussion with Tim Howes about this
here at INRIA.
The forum and platform are open too any designers developers 
willing to discuss ideas and/or to exercise against the platform.

    reminder : 
	the list name is [e-mail]
	subscription: oifp-request     as usual


-- PAP