[nfsv4] Re: RFC: new MUTEX_BEGIN/MUTEX_END operations

[Jeff Layton <[e-mail]>]

On Thu, 2024-08-08 at 13:36 -0700, Rick Macklem wrote:
> Hi,
> 
> Over the years, I've run into cases where it would be really
> nice to be able to perform multiple NFSv4 operations on a
> file without other operations done by other clients
> "gumming up the works" by changing the file's data/metadata
> between the operations in the compound.
> 
> So, what do others think about an extension to NFSv4.2 that
> adds 2 new operations:
>   MUTEX_BEGIN(CFH)
>   MUTEX_END(CFH)
> Both would use the CFH as argument, and no other client would
> be allowed to perform operations on the CFH between the MUTEX_BEGIN
> and MUTEX_END.
> 
> I think there would need to be a couple of properties for these:
> - There would need to be an "implicit" MUTEX_END when any operation
>   between MUTEX_BEGIN and MUTEX_END returns a status other than NFS_OK.
> - I think you would want a restriction of only one mutex for one CFH
>   at a time in a compound. Without that, there could easily be deadlocks
>   caused by other compounds acquiring mutexes on the same CFHs in a
>   different order.
> - Only one compound can hold a mutex on a given CFH at any time.
> - MUTEX_BEGIN/MUTEX_END can only be used in compounds where SEQUENCE
>   is the first operation.
> - All mutexes are discarded by a server when it crashes/recovers.
>   (Any time a client receives a NFS4ERR_STALE_CLIENTID.)
>   That way RPC retries after a server reboot should work ok, I think?
> 
> I am not sure what the semantics for reading data/metadata should be,
> but I was thinking that would be allowed to be done by compounds for
> other clients for the CFH. If a client wanted to serialize against
> other compounds for the CFH, it could do a MUTEX_BEGIN/MUTEX_END.
> 
> I see this as useful in a variety of ways:
> - The example in the previous email of:
>   MUTEX_BEGIN
>   NVERIFY acl_truform ACL_MODEL_NFS4
>   SETATTR posix_access_acl
>   MUTEX_END
> - Append writing:
>   MUTEX_BEGIN
>   VERIFY size "offset in WRITE that follows"
>   WRITE "offset" etc
>   MUTEX_END
> - A bunch of cases where NFSv4 lacks the postop_attributes
>   that were in NFSv3.
>   MUTEX_BEGIN
>   WRITE
>   GETATTR size, change,..
>   MUTEX_END
> 
> So, what do others think?
> (This was obviously not possible without sessions.)
> 

It's an interesting idea. Quite a bit different than LOCK/LOCKU for
sure.

The difficulty here will be in defining what activity this mutex
blocks. Just other MUTEX_BEGINs, or is this a mandatory lock for all
access to that filehandle? Assuming the latter, would READ or GETATTR
activity also be blocked? If not, then we'll have to have a list of
operations that are blocked by the mutex, and new operations will have
to have their semantics spelled out clearly vs. MUTEX_BEGIN.

-- 
Jeff Layton <[e-mail]>