Re: [nfsv4] RFC: New DelegReturn_NoFH operation for NFSv4.2?

On Tue, Apr 16, 2024 at 5:21 AM David Noveck <davenoveck@gmail.com> wrote:
>
>
>
> On Mon, Apr 15, 2024, 8:06 PM Trond Myklebust <trondmy@hammerspace.com> wrote:
>>
>>
>>
>> On Apr 15, 2024, at 18:31, Rick Macklem <rick.macklem@gmail.com> wrote:
>>
>> Hi,
>>
>> I am wondering if others have found this a pita and are interested in
>> a new operation for NFSv4.2 to get around it.
>>
>> Right now DelegReturn requires a CFH and delegation stateid.
>> I do not know why the CFH is needed, since it seems that the
>> delegation stateid should be sufficient?
>>
>> This new operation would be the same as DelegReturn except that
>> it would not require a CFH argument.
>>
>> Why am I interested in this?
>> - Lets assume a NFSv4.2 server that supports persistent FHs and
>>  multiple hard links (as most extant servers do).
>> - A client mounted to this server is removing a file for which it holds
>>  a write delegation and a bunch of dirty data.
>> Right now, the client must:
>> - Write the dirty data back to the server.
>> - DelegReturn
>> - Remove
>> because the client has no way of knowing if the file will be deleted
>> by the Remove (or alternately is just removing one of several hard links).
>>
>>
>> If a client is holding a delegation, it will presumably have done a GETATTR at some point while holding that delegation so should know how many links remain. While it is true that the unlink might invalidate the filehandle, that’s not a situation you are able to fix with the new operation.
>>
>> With a DelegReturn_NoFH, the client could:
>> - Do a compound RPC with
>>  Remove
>>  PutFH of file
>> - if the above returns NFS_OK, the file still exists and the client
>>  can continue to use the delegation, writing the dirty data back
>>  to the server sometime later (or maybe never, if the file gets
>>   deleted by a subsequent Remove).
>> or
>> - If the above returns NFS4ERR_STALE for the PutFH, the file
>>  has been deleted. It can throw the dirty data away and
>>  DelegReturn_NoFH the delegation.
>>
>> A similar situation exists for Rename where the destination
>> already exists.
>>
>> I find this scenario happens frequently during software builds.
>>
>> So, what do others think  w.r.t. this new operation? rick
>>
>>
>> Why would you need to do a DELEGRETURN if the PUTFH returns NFS4ERR_STALE?
>
>
> To make sure the client and server agree on the state of things.
>
> > If the server isn’t willing to just invalidate the delegation on the final unlink of the file,
>
> I'm not sure about "unwilling",  but I am uncomfortable with unilateral stare changes like this, even if the spec indicated that needed to be done, which is harder to effect than an extension such as Rick is proposing
>
>  > then it can and should recall the delegation (and any pNFS layouts that might be outstanding).
>
> If it did recall it/them, then there is no way to return them without something like Rick's proposed extension.
I assumed Trond meant "recall during the REMOVE" while the FH is still valid.

rick

>
> It seems to me we need something like CB_REVOKED (aka ? CB_RECALL_WITH_EXTREME_PREJUDICE :-) or some equivalent that might be dealt with in rfc5661bis-05
>
> I think we should reserve 5-10 minutes of the 4/23 interim to discuss possible approaches to dealing with the problem of getting rid of locking state for removed files.
>
>>
>> _________________________________
>> Trond Myklebust
>> Linux NFS client maintainer, Hammerspace
>> trond.myklebust@hammerspace.com
>>
>> _______________________________________________
>> nfsv4 mailing list
>> nfsv4@ietf.org
>> https://www.ietf.org/mailman/listinfo/nfsv4