Re: [nfsv4] RFC: New DelegReturn_NoFH operation for NFSv4.2?

[Rick Macklem <[e-mail]>]

On Tue, Apr 16, 2024 at 7:44 AM Trond Myklebust <[e-mail]> wrote:
>
> On Tue, 2024-04-16 at 07:06 -0700, Rick Macklem wrote:
> > On Tue, Apr 16, 2024 at 5:21 AM David Noveck <[e-mail]>
> > wrote:
> > >
> > >
> > >
> > > On Mon, Apr 15, 2024, 8:06 PM Trond Myklebust
> > > <[e-mail]> wrote:
> > > >
> > > >
> > > >
> > > > On Apr 15, 2024, at 18:31, Rick Macklem <[e-mail]>
> > > > wrote:
> > > >
> > > > Hi,
> > > >
> > > > I am wondering if others have found this a pita and are
> > > > interested in
> > > > a new operation for NFSv4.2 to get around it.
> > > >
> > > > Right now DelegReturn requires a CFH and delegation stateid.
> > > > I do not know why the CFH is needed, since it seems that the
> > > > delegation stateid should be sufficient?
> > > >
> > > > This new operation would be the same as DelegReturn except that
> > > > it would not require a CFH argument.
> > > >
> > > > Why am I interested in this?
> > > > - Lets assume a NFSv4.2 server that supports persistent FHs and
> > > >  multiple hard links (as most extant servers do).
> > > > - A client mounted to this server is removing a file for which it
> > > > holds
> > > >  a write delegation and a bunch of dirty data.
> > > > Right now, the client must:
> > > > - Write the dirty data back to the server.
> > > > - DelegReturn
> > > > - Remove
> > > > because the client has no way of knowing if the file will be
> > > > deleted
> > > > by the Remove (or alternately is just removing one of several
> > > > hard links).
> > > >
> > > >
> > > > If a client is holding a delegation, it will presumably have done
> > > > a GETATTR at some point while holding that delegation so should
> > > > know how many links remain. While it is true that the unlink
> > > > might invalidate the filehandle, that’s not a situation you are
> > > > able to fix with the new operation.
> > > >
> > > > With a DelegReturn_NoFH, the client could:
> > > > - Do a compound RPC with
> > > >  Remove
> > > >  PutFH of file
> > > > - if the above returns NFS_OK, the file still exists and the
> > > > client
> > > >  can continue to use the delegation, writing the dirty data back
> > > >  to the server sometime later (or maybe never, if the file gets
> > > >   deleted by a subsequent Remove).
> > > > or
> > > > - If the above returns NFS4ERR_STALE for the PutFH, the file
> > > >  has been deleted. It can throw the dirty data away and
> > > >  DelegReturn_NoFH the delegation.
> > > >
> > > > A similar situation exists for Rename where the destination
> > > > already exists.
> > > >
> > > > I find this scenario happens frequently during software builds.
> > > >
> > > > So, what do others think  w.r.t. this new operation? rick
> > > >
> > > >
> > > > Why would you need to do a DELEGRETURN if the PUTFH returns
> > > > NFS4ERR_STALE?
> > >
> > >
> > > To make sure the client and server agree on the state of things.
> > >
> > > > If the server isn’t willing to just invalidate the delegation on
> > > > the final unlink of the file,
> > >
> > > I'm not sure about "unwilling",  but I am uncomfortable with
> > > unilateral stare changes like this, even if the spec indicated that
> > > needed to be done, which is harder to effect than an extension such
> > > as Rick is proposing
> > >
> > >  > then it can and should recall the delegation (and any pNFS
> > > layouts that might be outstanding).
> > >
> > > If it did recall it/them, then there is no way to return them
> > > without something like Rick's proposed extension.
> > I assumed Trond meant "recall during the REMOVE" while the FH is
> > still valid.
>
> Yes. This is a clarification of how the server can deal with the
> problem (if it thinks this is a problem) within the confines of the
> existing spec
Just fyi, I should be able to test the stuff I've posted here recently
at the Bakeathon next week.
- I have a client setup that tries to do Open/Claim_deleg_prev_fh
  before ReclaimComplete. It also does not DelegReturn a delegation
  for a file it is about to Remove.
- I'll have the server configured so that it does not bother to CB_RECALL
  a delegation issued to the same client as the one doing the Remove or
  Rename.

I'll report here if the testing finds anything interesting, rick

>
> --
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace
> [e-mail]
>
>