Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done before/after Reclaim_complete?

[Rick Macklem <[e-mail]>]

On Fri, Apr 5, 2024 at 12:50 PM David Noveck <[e-mail]> wrote:
>
>
>
> On Thu, Apr 4, 2024 at 7:23 PM Rick Macklem <[e-mail]> wrote:
>>
>> Hi,
>>
>> I am in the process of implementing Open/Claim_Delegate_Prev and have run
>> into a question I cannot seem to answer from reading RFC8881.
>
>
> Let me see if I can make this clearer in rfc5661bis-04.  Fingers croosed.
>>
>>
>> There is this snippet:
>>    enum open_claim_type4 {
>>            /*
>>             * Not a reclaim.
>>             */
>>            CLAIM_NULL              = 0,
>>
>>            CLAIM_PREVIOUS          = 1,
>>            CLAIM_DELEGATE_CUR      = 2,
>>            CLAIM_DELEGATE_PREV     = 3,
>>
>>            /*
>>             * Not a reclaim.
>>             *
>>             * Like CLAIM_NULL, but object identified
>>             * by the current filehandle.
>>             */
>>            CLAIM_FH                = 4, /* new to v4.1 */
>> The comments in the above seem to "hint" that CLAIM_DELEGATE_PREV is a
>> reclaim type operation.
>> If that is the case, I'd assume it/should be performed by the client
>> before it does a
>> Reclaim_Complete?
>
>
> I think that's right.
>>
>>
>> Then there is this snippet:
>>    For OPEN requests that reach the server during the grace period, the
>>    server returns an error of NFS4ERR_GRACE.  The following claim types
>>    are exceptions:
>>
>>    *  OPEN requests specifying the claim type CLAIM_PREVIOUS are devoted
>>       to reclaiming opens after a server restart and are typically only
>>       valid during the grace period.
>>
>>    *  OPEN requests specifying the claim types CLAIM_DELEGATE_CUR and
>>       CLAIM_DELEG_CUR_FH are valid both during and after the grace
>>       period.  Since the granting of the delegation that they are
>>       subordinate to assures that there is no conflict with locks to be
>>       reclaimed by other clients, the server need not return
>>       NFS4ERR_GRACE when these are received during the grace period.
>> This clearly states that CLAIM_DELEGATE_PREV cannot be performed
>> during the server's grace period.
>>
>
> It seems to me that this is an oversight.
It may be, but I am now thinking it is a feature and not a bug.
Why?
Well, I now think that it is preferable for the client to use
Open/Claim_previous instead of Open/Claim_delegate_prev when
the server is in grace. (One reason is that the client is guaranteed to
get a delegation for Open/Claim_previous, which would allow it to write
dirty data back to the server if the reply specifies an immediate recall.

Here is how I am currently thinking the algorithms could go:
On the server...
- When the server receives a ExchangeID/Create_session with the
  same co_ownerid but different co_verifier as a clientid it already has
  - Mark all extant delegations OLD

- When the server receives a Open/Claim_delegate_prev
   - if in grace
         - reply NFS4ERR_GRACE
   - search the delegation list for a matching one (same FH) marked OLD
     - if found
         - clear the OLD mark and keep it
         - reply to Open successfully
     - else
         - reply NFS4ERR_BAD_RECLAIM

- When the server receives Reclaim_complete, discard all delegations
   marked OLD

On the client when it reboots and has delegations it wishes to recover...
- Do a ExchangeID/Create_session with same co_ownerid and
  different co_verifier
  - use_claim_previous = false
  - for each delegation it wishes to recover
 tryagain:
    - if use_claim_previous
       - do an Open/Claim_previous
    - else
       - do an Open/Claim_delegate_prev
         - if the reply is NFS4ERR_GRACE
           - use_claim_previous = true
           - goto tryagain
    - if the reply is NFS4ERR_BAD_RECLAIM
        - deal with ugly case of no recovered delegation
- Do a Reclaim_complete

Basically, having Open/Claim_delegate_prev reply NFS4ERR_GRACE is
an easy way for the client to test if the server is in grace.

>
>>
>> The above two statements are not necessarily contradictory but it does seem
>> that, if the server is in its grace period, the Reclaim_Complete needs to be
>> done before any Open/Claim_Delegate_Prev, since Reclaim_Complete tells
>> the server when to end its grace period.
>
>
> Yes.
>
>>
>> Note that, usually, a client would not be rebooting when a server is in its
>> grace period, but that could happen.
>
>
> Yes it can.  It wouldn't matter if you said "MUST NOT" or "IST STRENG VERBOTEN".
>>
>>
>> Bottom line...
>> Should a client perform Open/Claim_Delegate_Prev before or after
>> Reclaim_Complete?
>
>
> Before.
>
>>
>> And, if the answer is "after", how does the server know when a client
>> is done recovering opens/delegations via Open/Claim_Delegate_Prev?
>>
> That's why "after" is not a good answer.
>
>>
>> Thanks for any help with this, rick
>>
> Thanks for bringing this up.
One more item. The text for Sec. 18.16 allows Open/Claim_delegate_prev
to be used with Create. I cannot think why a Create case would be useful?

rick

>
>>
>> _______________________________________________
>> nfsv4 mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/nfsv4