Re: [nfsv4] "Courtesy locks"

[Trond Myklebust <[e-mail]>]

On Sun, 2010-09-12 at 12:21 -0400, [e-mail] wrote:
> The term "courtesy locks" is Trond's.  It refers to the practice of
> allowing locks associated with an expired lease to remain around, "as a
> courtesy to the client or as an optimization" (see section 8.6.3 or RFC
> 3530).  The text that is associated with this situation is not as clear
> or complete as it should be and a number of RFC3530bis issues are
> related to it.
> 
> In deciding how to address this, we need to find out what exactly
> clients and servers do.  A long while back I promised to send mail and
> gather up the requisite information, and now my promise has caught up
> with me.  If you can answer the following questions via email that would
> be great.  If not, I hope people will at least find out what their
> implementations do on these points so we discuss it at the upcoming
> bakeathon.
> 
> This looks pretty daunting but most people will probably say "Yes" to
> (S1) and/or easily deal with (C1), (C2), and (C3).
> 
> Make sure you use black pen and completely fill in the circles :-) 
> 
> 
> If you have a client and not a server, skip to (C1)
> 
> S1) When a lease expires, do you simply release all
>     the associated locks?
> 
>     If so, go to (C1), or if you don't have a client you 
>     are all done.
> 
> S2) If you do maintain locks for a longer time, when and if
>     there is a conflict before the lease being reactivated,
>     what do you do?
> 
>     A) Release all the locks associated with that client
>     B) Release all the locks held by that client for that 
>        same fh.
>     C) Something more limited than (B) that includes the lock
>        that had the conflict.
> 
>     If you answered (A) or (B), skip down to (S7).
> 
> S3) If the lock that had a conflict is a byte-range lock,
> 
>     S3.1) Is the associated open released as well?
>     S3.2) Are all byte-range locks associated with the
>           same open released as well?
>     S3.3) Are all byte range locks associated with the same
>           client/lock-owner released as well?
>     S3.4) Are all byte-range locks associated with the 
>           same open and owned by the client/lock-owner
>           released as well?
>     S3.5) If the lock is associated with an open that is
>           subordinate to a delegation, is the delegation
>           released as well?
>     S3.6) Are there any other locks that are released?
> 
> S4) If the lock that had a conflict is an open?
> 
>     S4.1) Are byte-range locks associated with this open
>           released as well?
>     S4.2) If the open is associated with a delegation,
>           is the delegation released as well.
> 
> S5) If the lock that had the conflict a delegation?
> 
>     S5.1) Are opens associated with that delegation
>           released as well?
>     S5.2) Are all opens for that client and for that fh
>           released as well?
>     S5.3) Are all byte-range locks associated with open
>           that are associated with that delegation released
>           as well.
>     S5.4) Are all byte-range locks for that client and for
>           released as well?
> 
> S6) Are there any other situations in which release of a 
>     lock due to a conflict will cause other locks to be
>     released?
> 
> S7) After releasing locks due to a conflict, what happen if
>     the associated stateid is referenced, assuming that 
>     happens relatively quickly?
> 
>     S7.1) The client gets NFS4ERR_EXPIRED?
>     S7.2) The client gets some other error?  What?
>     S7.3) Can it be different for different types of stateids?
> 
> S8) Assuming that the release does not cause immediate freeing
>     of the stateid (and returning NFS4ERR_BAD_STATEID), what 
>     provision is made for eventual deletion of such stateids?
> 
>     a) There is an LRU mechanism to eventually delete them.
>     b) They are kept around until the client or server 
>        reboots.
>     c) It depends on the type of lock.
> 
> If you have no client, you are done.
> 
> C1) When you get NFS4ERR_EXPIRED, what happens?
> 
>     a) Assume all locks associated with lease have been released
>        that the client needs to create a new clientid?
>     b) Assume that it means one or more locks have been lost and
>        the client needs to determine what as gone and which are
>        still valid.
>     c) Something else.  What?

C) We try to send a RENEW. If that fails, we try to send a SETCLIENTID
that uses the same clientid and verifier as when we originally mounted
the filesystem.
After that, we try to replay all OPEN and LOCK calls.

> C2) If you didn't answer (a) to (C1), are there any assumptions
>     (of the sort mentioned in (S3) to (S6)) whose violation would
>     cause problems? 

In principle not. We are supposed to be able to recover all locks on a
per-open_stateid basis.

IOW: if an open stateid, lock stateid or delegation stateid returns an
error, we should be able to replay just the OPEN+LOCK requests that are
required to allow the application to proceed.

Note that we do not implement a SIGLOST feature in the case where the
application has lost a lock (although we probably should). We never did
it for NFSv3, and since nobody has really requested it for NFSv4 either,
we have deferred implementing it there too.

> C3) How do you respond to getting the following errors?  Specifically
>     would it be problematic if that error were returned when a
>     courtesy lock was released due to a conflict:
> 
>     C3.1) NFS4ERR_ADMIN_REVOKED
>     C3.2) NFS4ERR_BAD_STATEID

We replay the OPEN+LOCK requests, and if that fails, we try again using
the zero stateid.

Cheers
  Trond