[nfsv4] Preventing an NFSv4.1 client from destroying a migrated lease after TSM

[Chuck Lever <[e-mail]>]

One of Oracle's engineers, Xuan Qi, has been experimenting
with NFSv4.1 Transparent State Migration. The
implementations under test are a Solaris-based prototype
NFS server, and a recent vintage migration-capable Linux
NFS client. Xuan observed the following:

+ While a simple workload is running, there is an FS
  migration event from server A to server B. Transparent
  State Migration is done as part of this migration. The
  session that had been established between the client and
  server A is NOT migrated.

+ The client has never interacted with server B before the
  migration event. After the client retrieves fs_locations
  from server A, it begins trunking discovery with server B.

+ In the initial EXCHANGE_ID request with server B, the
  client presents the same client owner and boot verifier
  it previously presented to server A.

+ In the initial EXCHANGE_ID reply, server B reports the
  presence of the migrated lease by returning the same
  clientID the client got from server A and asserting
  EXCHGID4_FLAG_CONFIRMED_R.

+ The client thus believes this lease is an old one it
  forgot to clean up. It purges the migrated lease by
  sending a second EXCHANGE_ID with the same client owner
  but an arbitrary boot verifier. Server B returns a
  second distinct client ID.

+ The client issues a CREATE_SESSION with the second
  client ID. This client ID and session is not used for
  subsequent operations.

+ The client issues a DESTROY_SESSION and DESTROY_CLIENTID
  to server A, using the original session and client ID.
  How polite!

+ The client then establishes a fresh lease with server
  B by sending a third EXCHANGE_ID with the same client
  owner and the original boot verifier. Server B returns
  a third distinct client ID. The client then uses this
  new client ID for a CREATE_SESSION. This session is used
  to issue RECLAIM_COMPLETE with no OPEN state recovery.

+ The first WRITE operation to server B fails with
  NFS4ERR_BAD_STATEID. The client performs no-grace
  stateid recovery, and the workload continues normally.

+ Both servers report the same server scope, but different
  server owners. The scope string is

    "Solaris NFSv4.1 Server Scope"

RFC 5661 and draft-ietf-nfsv4-migration-issues are not
clear about how servers and clients are supposed to
interact after Transparent State Migration. For instance:

1. Since the client has had no previous contact with
server B, should the migrated lease be considered
confirmed or not confirmed on server B?

2. How should the client determine whether or not
Transparent State Migration has occurred?

3. If the session was migrated (which is not the case
in the current example), how should the client determine
it must perform BIND_CONN_TO_SESSION instead of
CREATE_SESSION? (I guess just try BC2S first?)

4. If TSM is detected, the client performs no state
recovery. Should it avoid sending RECLAIM_COMPLETE in
this case?


--
Chuck Lever