[nfsv4] Re: RFC: new MUTEX_BEGIN/MUTEX_END operations

[Pali Rohár <[e-mail]>]

On Thursday 08 August 2024 13:36:25 Rick Macklem wrote:
> Hi,
> 
> Over the years, I've run into cases where it would be really
> nice to be able to perform multiple NFSv4 operations on a
> file without other operations done by other clients
> "gumming up the works" by changing the file's data/metadata
> between the operations in the compound.
> 
> So, what do others think about an extension to NFSv4.2 that
> adds 2 new operations:
>   MUTEX_BEGIN(CFH)
>   MUTEX_END(CFH)
> Both would use the CFH as argument, and no other client would
> be allowed to perform operations on the CFH between the MUTEX_BEGIN
> and MUTEX_END.
> 
> I think there would need to be a couple of properties for these:
> - There would need to be an "implicit" MUTEX_END when any operation
>   between MUTEX_BEGIN and MUTEX_END returns a status other than NFS_OK.
> - I think you would want a restriction of only one mutex for one CFH
>   at a time in a compound. Without that, there could easily be deadlocks
>   caused by other compounds acquiring mutexes on the same CFHs in a
>   different order.
> - Only one compound can hold a mutex on a given CFH at any time.
> - MUTEX_BEGIN/MUTEX_END can only be used in compounds where SEQUENCE
>   is the first operation.
> - All mutexes are discarded by a server when it crashes/recovers.
>   (Any time a client receives a NFS4ERR_STALE_CLIENTID.)
>   That way RPC retries after a server reboot should work ok, I think?
> 
> I am not sure what the semantics for reading data/metadata should be,
> but I was thinking that would be allowed to be done by compounds for
> other clients for the CFH. If a client wanted to serialize against
> other compounds for the CFH, it could do a MUTEX_BEGIN/MUTEX_END.
> 
> I see this as useful in a variety of ways:
> - The example in the previous email of:
>   MUTEX_BEGIN
>   NVERIFY acl_truform ACL_MODEL_NFS4
>   SETATTR posix_access_acl
>   MUTEX_END
> - Append writing:
>   MUTEX_BEGIN
>   VERIFY size "offset in WRITE that follows"
>   WRITE "offset" etc
>   MUTEX_END
> - A bunch of cases where NFSv4 lacks the postop_attributes
>   that were in NFSv3.
>   MUTEX_BEGIN
>   WRITE
>   GETATTR size, change,..
>   MUTEX_END
> 
> So, what do others think?
> (This was obviously not possible without sessions.)
> 
> rick

Hello,

Now I'm thinking more about this and those mutexes looks to be
performance killer for any NFS4 server which allows thousands of
parallel operations. And this can open also vectors for DDOS attacks
if servers are implemented not securely where malicious clients take
mutexes and blocks any operations.

Would not it be better to address existing problems by separate
mechanisms? For example append issue can be solved by new append
operation which NFS4 server can implement more optimized (e.g. if its
storage or API already provides append operation, which applies for all
POSIX systems via open/O_APPEND).

Also this mutex mechanism does not solve atomicity of operations in
multiprotocol environment where other protocols without this kind of
mutex co-exist together on the same storage (e.g. interop with SMB or
NFS3). If you have implementation of NFS4 and Samba in different
userspace processes then what can result is that the NFS4 mutex will
hold other NFS4 compound operations, but would not hold Samba
operations. So atomicity would not be guaranteed at all.

NFS4 servers do not have to be implemented in kernel where they can lock
inode to prevent any other operation by other system parts (which
implements mutex on filehandle). And from POSIX userspace such inode
locking is highly impossible to implement.

I just cannot imagine to how implement NFS4 server with support for this
mutex with system fs storage just by using POSIX API.

Pali