Re: [nfsv4] RFC: New DelegReturn_NoFH operation for NFSv4.2?

[Trond Myklebust <[e-mail]>]

On Tue, 2024-04-16 at 07:06 -0700, Rick Macklem wrote:
> On Tue, Apr 16, 2024 at 5:21 AM David Noveck <[e-mail]>
> wrote:
> > 
> > 
> > 
> > On Mon, Apr 15, 2024, 8:06 PM Trond Myklebust
> > <[e-mail]> wrote:
> > > 
> > > 
> > > 
> > > On Apr 15, 2024, at 18:31, Rick Macklem <[e-mail]>
> > > wrote:
> > > 
> > > Hi,
> > > 
> > > I am wondering if others have found this a pita and are
> > > interested in
> > > a new operation for NFSv4.2 to get around it.
> > > 
> > > Right now DelegReturn requires a CFH and delegation stateid.
> > > I do not know why the CFH is needed, since it seems that the
> > > delegation stateid should be sufficient?
> > > 
> > > This new operation would be the same as DelegReturn except that
> > > it would not require a CFH argument.
> > > 
> > > Why am I interested in this?
> > > - Lets assume a NFSv4.2 server that supports persistent FHs and
> > >  multiple hard links (as most extant servers do).
> > > - A client mounted to this server is removing a file for which it
> > > holds
> > >  a write delegation and a bunch of dirty data.
> > > Right now, the client must:
> > > - Write the dirty data back to the server.
> > > - DelegReturn
> > > - Remove
> > > because the client has no way of knowing if the file will be
> > > deleted
> > > by the Remove (or alternately is just removing one of several
> > > hard links).
> > > 
> > > 
> > > If a client is holding a delegation, it will presumably have done
> > > a GETATTR at some point while holding that delegation so should
> > > know how many links remain. While it is true that the unlink
> > > might invalidate the filehandle, that’s not a situation you are
> > > able to fix with the new operation.
> > > 
> > > With a DelegReturn_NoFH, the client could:
> > > - Do a compound RPC with
> > >  Remove
> > >  PutFH of file
> > > - if the above returns NFS_OK, the file still exists and the
> > > client
> > >  can continue to use the delegation, writing the dirty data back
> > >  to the server sometime later (or maybe never, if the file gets
> > >   deleted by a subsequent Remove).
> > > or
> > > - If the above returns NFS4ERR_STALE for the PutFH, the file
> > >  has been deleted. It can throw the dirty data away and
> > >  DelegReturn_NoFH the delegation.
> > > 
> > > A similar situation exists for Rename where the destination
> > > already exists.
> > > 
> > > I find this scenario happens frequently during software builds.
> > > 
> > > So, what do others think  w.r.t. this new operation? rick
> > > 
> > > 
> > > Why would you need to do a DELEGRETURN if the PUTFH returns
> > > NFS4ERR_STALE?
> > 
> > 
> > To make sure the client and server agree on the state of things.
> > 
> > > If the server isn’t willing to just invalidate the delegation on
> > > the final unlink of the file,
> > 
> > I'm not sure about "unwilling",  but I am uncomfortable with
> > unilateral stare changes like this, even if the spec indicated that
> > needed to be done, which is harder to effect than an extension such
> > as Rick is proposing
> > 
> >  > then it can and should recall the delegation (and any pNFS
> > layouts that might be outstanding).
> > 
> > If it did recall it/them, then there is no way to return them
> > without something like Rick's proposed extension.
> I assumed Trond meant "recall during the REMOVE" while the FH is
> still valid.

Yes. This is a clarification of how the server can deal with the
problem (if it thinks this is a problem) within the confines of the
existing spec.

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
[e-mail]