Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done before/after Reclaim_complete?

[Rick Macklem <[e-mail]>]

On Sat, Apr 6, 2024 at 5:33 PM Rick Macklem <[e-mail]> wrote:
>
> On Fri, Apr 5, 2024 at 12:50 PM David Noveck <[e-mail]> wrote:
> >
> >
> >
> > On Thu, Apr 4, 2024 at 7:23 PM Rick Macklem <[e-mail]> wrote:
> >>
> >> Hi,
> >>
> >> I am in the process of implementing Open/Claim_Delegate_Prev and have run
> >> into a question I cannot seem to answer from reading RFC8881.
> >
> >
> > Let me see if I can make this clearer in rfc5661bis-04.  Fingers croosed.
> >>
> >>
> >> There is this snippet:
> >>    enum open_claim_type4 {
> >>            /*
> >>             * Not a reclaim.
> >>             */
> >>            CLAIM_NULL              = 0,
> >>
> >>            CLAIM_PREVIOUS          = 1,
> >>            CLAIM_DELEGATE_CUR      = 2,
> >>            CLAIM_DELEGATE_PREV     = 3,
> >>
> >>            /*
> >>             * Not a reclaim.
> >>             *
> >>             * Like CLAIM_NULL, but object identified
> >>             * by the current filehandle.
> >>             */
> >>            CLAIM_FH                = 4, /* new to v4.1 */
> >> The comments in the above seem to "hint" that CLAIM_DELEGATE_PREV is a
> >> reclaim type operation.
> >> If that is the case, I'd assume it/should be performed by the client
> >> before it does a
> >> Reclaim_Complete?
> >
> >
> > I think that's right.
> >>
> >>
> >> Then there is this snippet:
> >>    For OPEN requests that reach the server during the grace period, the
> >>    server returns an error of NFS4ERR_GRACE.  The following claim types
> >>    are exceptions:
> >>
> >>    *  OPEN requests specifying the claim type CLAIM_PREVIOUS are devoted
> >>       to reclaiming opens after a server restart and are typically only
> >>       valid during the grace period.
> >>
> >>    *  OPEN requests specifying the claim types CLAIM_DELEGATE_CUR and
> >>       CLAIM_DELEG_CUR_FH are valid both during and after the grace
> >>       period.  Since the granting of the delegation that they are
> >>       subordinate to assures that there is no conflict with locks to be
> >>       reclaimed by other clients, the server need not return
> >>       NFS4ERR_GRACE when these are received during the grace period.
> >> This clearly states that CLAIM_DELEGATE_PREV cannot be performed
> >> during the server's grace period.
> >>
> >
> > It seems to me that this is an oversight.
> It may be, but I am now thinking it is a feature and not a bug.
> Why?
> Well, I now think that it is preferable for the client to use
> Open/Claim_previous instead of Open/Claim_delegate_prev when
> the server is in grace. (One reason is that the client is guaranteed to
> get a delegation for Open/Claim_previous, which would allow it to write
> dirty data back to the server if the reply specifies an immediate recall.
>
> Here is how I am currently thinking the algorithms could go:
> On the server...
> - When the server receives a ExchangeID/Create_session with the
>   same co_ownerid but different co_verifier as a clientid it already has
>   - Mark all extant delegations OLD
>
> - When the server receives a Open/Claim_delegate_prev
>    - if in grace
>          - reply NFS4ERR_GRACE
>    - search the delegation list for a matching one (same FH) marked OLD
>      - if found
>          - clear the OLD mark and keep it
>          - reply to Open successfully
>      - else
>          - reply NFS4ERR_BAD_RECLAIM
Oops, my bad...it's NFS4ERR_RECLAIM_BAD, no pun intended, rick

>
> - When the server receives Reclaim_complete, discard all delegations
>    marked OLD
>
> On the client when it reboots and has delegations it wishes to recover...
> - Do a ExchangeID/Create_session with same co_ownerid and
>   different co_verifier
>   - use_claim_previous = false
>   - for each delegation it wishes to recover
>  tryagain:
>     - if use_claim_previous
>        - do an Open/Claim_previous
>     - else
>        - do an Open/Claim_delegate_prev
>          - if the reply is NFS4ERR_GRACE
>            - use_claim_previous = true
>            - goto tryagain
>     - if the reply is NFS4ERR_BAD_RECLAIM
>         - deal with ugly case of no recovered delegation
> - Do a Reclaim_complete
>
> Basically, having Open/Claim_delegate_prev reply NFS4ERR_GRACE is
> an easy way for the client to test if the server is in grace.
>
> >
> >>
> >> The above two statements are not necessarily contradictory but it does seem
> >> that, if the server is in its grace period, the Reclaim_Complete needs to be
> >> done before any Open/Claim_Delegate_Prev, since Reclaim_Complete tells
> >> the server when to end its grace period.
> >
> >
> > Yes.
> >
> >>
> >> Note that, usually, a client would not be rebooting when a server is in its
> >> grace period, but that could happen.
> >
> >
> > Yes it can.  It wouldn't matter if you said "MUST NOT" or "IST STRENG VERBOTEN".
> >>
> >>
> >> Bottom line...
> >> Should a client perform Open/Claim_Delegate_Prev before or after
> >> Reclaim_Complete?
> >
> >
> > Before.
> >
> >>
> >> And, if the answer is "after", how does the server know when a client
> >> is done recovering opens/delegations via Open/Claim_Delegate_Prev?
> >>
> > That's why "after" is not a good answer.
> >
> >>
> >> Thanks for any help with this, rick
> >>
> > Thanks for bringing this up.
> One more item. The text for Sec. 18.16 allows Open/Claim_delegate_prev
> to be used with Create. I cannot think why a Create case would be useful?
>
> rick
>
> >
> >>
> >> _______________________________________________
> >> nfsv4 mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/nfsv4