Re: [nfsv4] Directory delegations, take 2

[David Robinson <[e-mail]>]

The purist in me would love to have directory delegations
be symmetric with file delegations, but I grant that the
complexity and recovery semantics of revoking a write
delegation can get really ugly.

I will claim that one of the set of requirements that we are
trying to solve is the ability of a client to cache the
contents of a directory (the names and file handles) in
order to efficiently operate on part of the namespace tree.

I will argue that the ability for a client to modify a
directory without going to the server is not a requirement.
For all the arguments laid out by Dave, it will be difficult
to handle not only the error semantics, but it would imply that
the client should have a reasonable ability to determine what
the new cookies should be, otherwise we take the rare problem
of invalid cookies and make it the common case.

I would support read-only delegations of directory contents.
All directory modifying operations MUST we written through to
the server and any client having a delegation MUST be notified.
In the unlucky event that a client is unreachable, the effect is
that directory entries created or deleted by another client
may go undetected.  But I will claim that we have this problem
today in NFSv3 with normal client DNLC's and attribute caching,
this just opens the window a bit wider in case of a lost revocation.

I agree that the delegation recall (effectively change notification)
should be synchronous. It should not be a large burden on the
client as by definition there is nothing that will need to be
written back, only a flush of cached entries.  It even leaves open the
option for client to ignore the recall and continue to run out
of their caches knowing that there may be an inconsistency. Seat belts?
Don't need no stinking seat belts! To advocate async recalls will
prohibit an interesting class of clients that need to know
about changes.

> Tom Talpey (private e-mail) has raised the issue of whether change notifica-
> tion is worth it at all, and whether instead you just have a recall/revoca-
> tion event and let the client just get his delegation again and refetch the 
> modified directory contents.  This does make the feature easier to spec 
> (e.g. There is a tough case of sequencing notifications and successive 
> READDIR's when fetching a big directory, as well as a more complicated set 
> of callbacks to define).  However, I worry about very large directories and 
> the effort of refetching, when there is a modest level of exogenous
> directory change.  What do other people think?  I think I'm going to go 
> forward trying to do the notifications, unless it turns out  that the 
> complexities make this too difficult to do for v4.1. 

Assuming that as part of the addition of directory delegations, we
add in the concept of reacquiring a delegation, I see no
significant difference between a recall and a change notification.
Both must occur on a directory modifying event, I can see an
argument that a change notification can be made on a per file basis
versus a delegation on a per directory basis. Because the order,
cookies, and contents can have a ripple effect through the
directory I am not sure that a client can safely do anything other
than reread the directory. I am explicitly not addressing attribute
changes here, more on that later.

> One issue that Saadia Khan raised is the issue of directory changes made 
> by the delegate himself.  I think we have to make clear that this is 
> allowed and the client is presumed to know about directory changes it 
> makes itself.  Doing otherwise would compromise the usefulness of 
> directory delegations in the case in which a single client is modifying 
> the directory.  My assumption is that it just too difficult to do write 
> directory delegation, but exclusive use is still a very important case, 
> and we should what we can to make read directory delegations useful in 
> the exclusive use environment.  

I find this contradictory with your previous arguments against write
delegations. All the issues of revocation and unresponsive clients
exists in this scenerio as well.  The only way to make this sort of
exclusive access work is to make irrevokable delegations which will
suffer the problem of stale locks blocking any directory access.
You must require all modifying directory operations to write through.

On the argument against a separate attribute delegation it appears
to come down primarily to that of performance. One recent a-ha that
I had with respect to the old READDIRPLUS as translated to v4,
the straight forward implementation of requesting the desired
attributes as part of the READDIR operation may not be the most
efficient mechanism to process from a client perspective. With
the availablity of COMPOUND, the client can alternatively issue
a READDIR with no requested attributes (trivial decode) and then
issue a single COMPOUND of LOOKUP's for all the names returned.
(Alternatively request just the nfs_fh4 and COMPOUND GETATTRs).
With this sort of approach, if the client wanted to gather
delegations of attributes it could just COMPOUND the delegation
requests with the GETATTRS, likewise revocation could also be
COMPOUND'd. From a performance perspective this seems trivially
more work than doing it on a per directory basis.

More importantly, unless we want to ban hard links, or delegation of
directories that contain files with hard links (something I would
probably support notwithstanding Posix), determing which directory
delegations need to get revoked due to a SETATTR on a hard linked
file is incredibly hard to implement.

I have not thought of how to spec this in a protocol, but Mike's
idea's of OPENDIR/CLOSEDIR looks interesting...

While I can see how we could nicely add in symlink delegation,
I am not sure that is a "problem" that needs to be solved.

	-David


_______________________________________________
nfsv4 mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/nfsv4