Re: [nfsv4] Directory delegations, take 2

[Nicolas Williams <[e-mail]>]

On Wed, Oct 22, 2003 at 07:38:24AM -0700, Noveck, Dave wrote:
[...]
> > I would support read-only delegations of directory contents.
> > All directory modifying operations MUST we written through to
> > the server and any client having a delegation MUST be notified.
> > In the unlucky event that a client is unreachable, the effect is
> > that directory entries created or deleted by another client
> > may go undetected.  But I will claim that we have this problem
> > today in NFSv3 with normal client DNLC's and attribute caching,
> > this just opens the window a bit wider in case of a lost revocation.
> 
> The other issue is that the client doing the modification must
> wait for the unresponsive client, but since, as you point out,
> the effects of giving up on the notification, and allowing the
> update to proceed without it are not dire, we can be aggressive 
> in allowing that to happen, and so the client will not have to
> wait a very long time.

Synchronous notification delegations sound both neat and like trouble.
If we can't guarantee synchronicity (and from these arguments it's clear
that we shouldn't), what good is it to provide synchronous notification
delegations?  Applications will not be able to use the notification
mechanism for synchronization without the guarantee, so why would they
want synchronous notifications?

[...]
> > > Tom Talpey (private e-mail) has raised the issue of whether change notifica-
> > > tion is worth it at all, and whether instead you just have a recall/revoca-
> > > tion event and let the client just get his delegation again and refetch the 
> > > modified directory contents.  This does make the feature easier to spec 
> > > (e.g. There is a tough case of sequencing notifications and successive 
> > > READDIR's when fetching a big directory, as well as a more complicated set 
> > > of callbacks to define).  However, I worry about very large directories and 
> > > the effort of refetching, when there is a modest level of exogenous
> > > directory change.  What do other people think?  I think I'm going to go 
> > > forward trying to do the notifications, unless it turns out  that the 
> > > complexities make this too difficult to do for v4.1. 
> 
> > Assuming that as part of the addition of directory delegations, we
> > add in the concept of reacquiring a delegation, I see no
> > significant difference between a recall and a change notification.
> > Both must occur on a directory modifying event, I can see an
> > argument that a change notification can be made on a per file basis
> > versus a delegation on a per directory basis. Because the order,
> > cookies, and contents can have a ripple effect through the
> > directory I am not sure that a client can safely do anything other
> > than reread the directory. I am explicitly not addressing attribute
> > changes here, more on that later.
> 
> The significant difference is that you don't have to re-read the
> directory, in most (almost all?) cases, and when directories are
> large (and many are) that is a big deal indeed.
> 
> You say that it might be the case that a single change could have a
> ripple effect (order, cookies, etc.) and that is possible but not
> very common.  Such a thing would cause clients doing multiple
> READDIR's to get the whole directory to lose their mind.  The spec
> tells servers to try not to do this, and I suspect most servers don't.
> 
> I agree that you do need the option of recall if you are in a situation
> where the server for whatever reason has changed so much stuff that
> re-READDIR is the best thing.  However, in most cases, something simple
> has happened and the server knows that.  When a file is created and nothing
> else changes in the directory, it makes sense for the server, which
> knows that to tell the client, so he can get rid of is negative dnlc
> entry, without rereading the whole directory, simply because some
> other changes might have happened, when it in fact they didn't.

This is clearly a rehash of the same discussion that has surely taken
place in the design of many a database replication protocol: "we want to
be efficient for the common case of low flux rates, but we don't want
periods of large flux rates to cause replication bandwidth to zoom past
the bandwidth necessary to just send the database; during periods of
medium flux rates we want to batch up notifications close to each other
in time, and during periods of large flux rates we want to throttle down
replication and replicate the whole DB instead of incremental updates."

We're really talking about replication of lots of mostly small
databases, where the directories are the databases.

And note that where a client is interested in getting notifications for
many directories in a file system, not just one, a per-directory
notification system will be inefficient, so a way to get a notification
delegation for all directories in a file system may be a good idea.

Note that if we take this to extremes we get an [async] file system
replication protocol :)

> > > One issue that Saadia Khan raised is the issue of directory changes made 
> > > by the delegate himself.  I think we have to make clear that this is 
> > > allowed and the client is presumed to know about directory changes it 
> > > makes itself.  Doing otherwise would compromise the usefulness of 
> > > directory delegations in the case in which a single client is modifying 
> > > the directory.  My assumption is that it just too difficult to do write 
> > > directory delegation, but exclusive use is still a very important case, 
> > > and we should what we can to make read directory delegations useful in 
> > > the exclusive use environment.  
> 
> > I find this contradictory with your previous arguments against write
> > delegations. All the issues of revocation and unresponsive clients
> > exists in this scenerio as well.  The only way to make this sort of
> > xclusive access work is to make irrevokable delegations which will
> > suffer the problem of stale locks blocking any directory access.
> > You must require all modifying directory operations to write through.
> 
> Yes, all modifying directory operations must be written through.  We
> agree on that.  The question is whether the client who did that write
> loses his delegation (or gets a notification).  I'm saying he doesn't.
> 
> He is presumed to know about changes he makes himself (by getting
> the reply) and he retains the delegation and thus the ability to be
> notified (whether through change notifications or recall) about 
> changes made by other clients.

Whether it gets a notification or not is not as interesting as: making
sure that the server can know not to send a notification, if the client
must not get one, or making sure that the client can tell that it was
the source of some notification so that it can ignore it.

Cheers,

Nico
-- 

_______________________________________________
nfsv4 mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/nfsv4