Re: [nfsv4] more re: admin revoke and edge conditions

[Spencer Shepler <[e-mail]>]

On Sat, [e-mail] wrote:
> [Spencer's good stuff snipped, which helped clarify some of my confusion]
> 
> > In these cases, not all client state would be removed ot need to be
> > removed.  However, the client should be told of the revocation and
> > hence the NFS4ERR_ADMIN_REVOKED.  The reason for returning this error
> > on RENEW is similar to the NFS4ERR_CB_PATH_DOWN such that the client
> > is active on other parts of the server's resources but not the part
> > where the state has been revoked.  The client could be told earlier of
> > the ADMIN_REVOKED than later and it could then in turn notify the
> > client environment as per its policies.  The idea was that the client
> > could "check" its state to determine what had been revoked; no clear
> > guidance was obviously given but something like a zero length read on
> > open files may suffice.
> 
> Ok, so Renew should return NFS4ERR_ADMIN_REVOKED whenever any of the state
> (doesn't have to be all of it) has been revoked?
> 
> And, it sounds like this Renew Op that returns NFS4ERR_ADMIN_REVOKED does
> Renew the client's lease. Is that correct?

Yes. It does renew; again, similarly as with the CB_PATH_DOWN error.

> 
> > If the server has rebooted, it just needs to return things like
> > NFS4ERR_STALE_CLIENTID/NFS4ERR_STALE_STATEID.  The ADMIN_REVOKED is
> > not required because all state has been removed for the client.
> 
> I did my usual crappy job of explaining what my concern was, so I think
> Spencer didn't follow it. I'll try again with an example.
> (Dave might have understood or he might be thinking
> of something else. I haven't looked at it closely enough to see what
> effect sessions might have on this. I have a hunch that they will help,
> but it's just a hunch.)
> For example:
> 1 - client A has lock on xxx
> 2 - administrator revokes this lock on xxx for client A
> 3 - client B gets lock on xxx that would have conflicted with (1)
> 4 - server crashes/reboots
> 5 - both clients see STALECLIENTID or STALESTALEID and start reclaiming locks
> 6 - client A requests reclaim of lock on xxx
> 7 - client B requests reclaim of lock on xxx, that would conflict with (6)
> 
> Since, if (1)->(4) occurs in less than a lease time (or client A is network
> partitioned until (5)), it did not see NFS4ERR_ADMIN_REVOKED.
> 
> Now, the question is, what is the server's correct response to (6) and (7)?
> If the server doesn't keep anything about the revoke at (2) in stable storage,
> it would be:
> (6) - NFS4_OK
> (7) - NFS4ERR_RECLAIM_CONFLICT
> and this doesn't seem correct to me.
> 
> It seems to me that the server needs to use stable storage such that, after
> reboot, it would reply:
> (6) - NFS4ERR_xxx
> (7) - NFS4_OK
> which then leads to what NFS4ERR_xxx should be? I was thinking NFS4ERR_NOGRACE,
> but will a client expect that error for only some of the reclaims and not all
> of them?

Example good.  Agreed, the server should record something in its
stable storage with respect to the occurance of ADMIN_REVOKED and upon
reboot, the client should be treated as though it did not hold state
prior to the reboot and therefore receive NOGRACE.

> 
> I was also thinking that I would handle admin revoke using the same stable
> storage mechanism as for lease expiry (essentially as described in 8.6.3), but
> that has a couple of negative implications:
> a) - client A can't reclaim any of its state for the above scenario
>
> b) - once I note admin revoke has happenned to client A in stable storage,
>      I can't come up with a way, short of the client doing a new SetClientID
>      of recognizing that it is ok to let client A do reclaims and this doesn't
>      seem like a good thing. (This was what I was trying to ask about in
>      the last post, believe it or not:-)
>      For example:
>      1 - client A has lock on xxx
>      2 - administrator revokes that lock, which is noted in stable storage
>      3 - client A sees NFS4ERR_ADMIN_REVOKED, but doesn't do a SetClientID,
>          since it doesn't see NFS4ERR_EXPIRED
> 	 (this assumes that some Op is renewing the lease, even though
> 	  the lock has been admin revoked. If no Op can renew the lease,
> 	  then everything will expire and you might as well just revoke all
> 	  the state for a clientid at once, because it will expire soon
> 	  anyhow.)
>      4 - client A chugs merrily along for weeks and has lots of other valid
>          state
>      5 - server crashes/reboots
>      6 - client A sees STALECLIENTID or STALESTATEID
>      7 - client A tries to reclaim state and gets NFS4ERR_NOGRACE for all of
>          it
> 
> The more I look at it, it seems that there are only two alternatives (at
> least until sessions are in place, which I haven't looked at closely
> enough yet, to understand how they might help).
> 
> 1 - forget about the cases where not all of the clientid's state is admin
>     revoked and just revoke it all at once, then let the lease expire.
> OR
> 2 - figure out a more sophisticated stable storage structure that handles
>     the edge cases, so that the NFS4ERR_xxx replied to A's reclaim above,
>     can be ADMIN_REVOKED.

Given what we have to work with and the fact that the ADMIN_REVOKED is
supposed to represent a rare intervention by the administrator, a simple
recording in stable storage that the revocation occurred and NO_GRACE
error on reboot is sufficient.  Obviously a server tool that allows
for revocation is a power tool and should come with the appropriate
warnings about its use (e.g., the sys admin should see about "resetting"
the client in a way to have it generate a new SETCLIENTID sequence).

> > In the case of server reboot, all of this is moot.  The server doesn't
> > need to worry about partial revocation before the reboot because all
> > of it is gone now.
> 
> But, only if the client knows the state was revoked and doesn't try to reclaim
> it, right?
> 
> Am I making sense or still out in left field? rick

As you have demonstrated with your example, the server, if it supports
administrative revocation, needs to record revocations such that
the client will not be subject to potential reclaim conflicts.

Spencer

_______________________________________________
nfsv4 mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/nfsv4