Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done before/after Reclaim_complete?

[Rick Macklem <[e-mail]>]

On Fri, Apr 5, 2024 at 8:02 PM Trond Myklebust <[e-mail]> wrote:
>
> On Fri, 2024-04-05 at 16:13 -0700, Rick Macklem wrote:
> > On Thu, Apr 4, 2024 at 9:18 PM Trond Myklebust
> > <[e-mail]> wrote:
> > >
> > > On Thu, 2024-04-04 at 16:23 -0700, Rick Macklem wrote:
> > > > Hi,
> > > >
> > > > I am in the process of implementing Open/Claim_Delegate_Prev and
> > > > have
> > > > run
> > > > into a question I cannot seem to answer from reading RFC8881.
> > > >
> > > > There is this snippet:
> > > >    enum open_claim_type4 {
> > > >            /*
> > > >             * Not a reclaim.
> > > >             */
> > > >            CLAIM_NULL              = 0,
> > > >
> > > >            CLAIM_PREVIOUS          = 1,
> > > >            CLAIM_DELEGATE_CUR      = 2,
> > > >            CLAIM_DELEGATE_PREV     = 3,
> > > >
> > > >            /*
> > > >             * Not a reclaim.
> > > >             *
> > > >             * Like CLAIM_NULL, but object identified
> > > >             * by the current filehandle.
> > > >             */
> > > >            CLAIM_FH                = 4, /* new to v4.1 */
> > > > The comments in the above seem to "hint" that CLAIM_DELEGATE_PREV
> > > > is
> > > > a
> > > > reclaim type operation.
> > > > If that is the case, I'd assume it/should be performed by the
> > > > client
> > > > before it does a
> > > > Reclaim_Complete?
> > > >
> > > > Then there is this snippet:
> > > >    For OPEN requests that reach the server during the grace
> > > > period,
> > > > the
> > > >    server returns an error of NFS4ERR_GRACE.  The following claim
> > > > types
> > > >    are exceptions:
> > > >
> > > >    *  OPEN requests specifying the claim type CLAIM_PREVIOUS are
> > > > devoted
> > > >       to reclaiming opens after a server restart and are
> > > > typically
> > > > only
> > > >       valid during the grace period.
> > > >
> > > >    *  OPEN requests specifying the claim types CLAIM_DELEGATE_CUR
> > > > and
> > > >       CLAIM_DELEG_CUR_FH are valid both during and after the
> > > > grace
> > > >       period.  Since the granting of the delegation that they are
> > > >       subordinate to assures that there is no conflict with locks
> > > > to
> > > > be
> > > >       reclaimed by other clients, the server need not return
> > > >       NFS4ERR_GRACE when these are received during the grace
> > > > period.
> > > > This clearly states that CLAIM_DELEGATE_PREV cannot be performed
> > > > during the server's grace period.
> > > >
> > > > The above two statements are not necessarily contradictory but it
> > > > does seem
> > > > that, if the server is in its grace period, the Reclaim_Complete
> > > > needs to be
> > > > done before any Open/Claim_Delegate_Prev, since Reclaim_Complete
> > > > tells
> > > > the server when to end its grace period.
> > > > Note that, usually, a client would not be rebooting when a server
> > > > is
> > > > in its
> > > > grace period, but that could happen.
> > > >
> > > > Bottom line...
> > > > Should a client perform Open/Claim_Delegate_Prev before or after
> > > > Reclaim_Complete?
> > > > And, if the answer is "after", how does the server know when a
> > > > client
> > > > is done recovering opens/delegations via
> > > > Open/Claim_Delegate_Prev?
> > > >
> > > > Thanks for any help with this, rick
> > >
> > > CLAIM_DELEGATE_PREV has now survived at least 24 years as part of a
> > > "standard" and has yet to be properly described. I can find
> > > references
> > > to it in RFC3010 and all NFSv4 descriptions since then, but I have
> > > yet
> > > to meet anybody who can claim to have implemented it in a product.
> > >
> > > IMO, it is time to drop it, and start afresh with something that
> > > has
> > > actually been road tested before someone started putting pen to
> > > paper.
> > Well, I am going to respond to David's post in some more detail, but
> > I will
> > note here that, with the exception of whether to do it before or
> > after
> > Reclaim_complete,
> > I have not found serious issues with it. (Btw, the FreeBSD server has
> > had a "dormant"
> > implementation for about 20years, although I am just now trying to
> > use it by the
> > FreeBSD client. I was wondering if I would find any other server
> > implementation at
> > the Bakeathon in a couple of weeks.)
>
> I see. So how do you plan to deal with the case where the server denies
> the request for a delegation? Will your client just toss away the dirty
> data?
After thinking about this a little more, my current plan (which could change
at any time) is:
- When a delegation cannot be recovered after a client reboot, the cached
  data for the file will be marked Defunct.
  - This means the cached data remains in the non-volatile cache, but
     cannot be used as cached data.
- Then I plan on implementing a couple of commands (I call this cache
   a packrat, in case you wondered about the names):
   packrat_defunct <file-path>
   - This command checks to see if there is defunct cached data for the
      file and whether or not any of it is dirty.
   packrat_flush <file-path>
   - This command flushes defunct cached data that is dirty back to the
      file on the server.
I think these two commands can be used with "find" to deal with dirty data
where there is no longer a delegation and, as such, cannot be written
back to the server automatically. If a file needs to be merged, all the user
needs to do is a:
$ cp foo foo.bak
- before doing the
$ packrat-flush foo

If the above commands are coded carefully, they should be usable by
non-root users, since they will only work on files that the user has write
access to.

rick



>
> --
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace
> [e-mail]
>
>