Re: [nfsv4] persistent sessions and compound atomicity

["J. Bruce Fields" <[e-mail]>]

On Thu, Dec 10, 2020 at 09:23:52PM -0500, Trond Myklebust wrote:
> On Thu, 2020-12-10 at 17:07 -0500, David Noveck wrote:
> > 
> > 
> > On Wed, Dec 9, 2020, 5:59 PM J. Bruce Fields <[e-mail]>
> > wrote:
> > > On Wed, Dec 09, 2020 at 04:43:59PM +0000, Rick Macklem wrote:
> > > > I would say that the ATOMIC refers to the individual operations
> > > of the
> > > > compound. In other words, it must cache the reply to each
> > > operation
> > > > when it is completed, in the persistent cache.
> > > > 
> > > > Then, if a retry of the compound is received after a server
> > > reboot,
> > > > the reply for the first K operations that were completed prior to
> > > the
> > > > reboot are generated from the cache and the remaining N-K are
> > > > performed on the file system to complete the reply. (At least for
> > > the
> > > > non-idempotent operations in the compound.)
> > > > 
> > > > For this to all work correctly, I think the server file system
> > > would
> > > > need to perform each non-idempotent operation ATOMICALLY as well,
> > > such
> > > > that after the reboot the file system is in either the pre-
> > > operation
> > > > or post-operation state.
> > > 
> > > Great, sounds like everyone agrees, I'll calm down.
> > 
> > I don't see the need for individual ops to be atomic especially since
> > they (e.g. writes) are not defined as atomic in general.
> > 
> > I don't think we have total agreement but we are moving in the same
> > direction.  No reason to be anything but calm.
> 
> in Section 2.10.6.5 of RFC5661, there is the following snippet:
> 
>    One way to
>    view the problem is as a single transaction consisting of each
>    operation in the COMPOUND followed by storing the result in
>    persistent storage, then finally a transaction commit.  If there is a
>    failure before the transaction is committed, then the server rolls
>    back the transaction.  If the server itself fails, then when it
>    restarts, its recovery logic could roll back the transaction before
>    starting the NFSv4.1 server.
> 
> Why shouldn't a client implementer reading the above be able to
> conclude that a NFS4ERR_DEADSESSION error implies that the operation
> failed with no side-effects?

Yes--they can conclude that the one operation failed.  But previous
operations may have succeeded and had side effects.

I had that wrong--I thought the spec said somewhere that DEADSESSION
could only be returned to the initial SEQUENCE op, but actually it's
quite explicit that it can be returned later:

	https://tools.ietf.org/html/rfc5661#section-18.46.4

	In the case of a partially executed COMPOUND, processing may
	reach an operation not processed during the earlier server
	instance, making this operation a new one and not performable on
	the existing session.  In this case, NFS4ERR_DEADSESSION will be
	returned from that operation.

	https://tools.ietf.org/html/rfc5661#section-15.1.11.5

	15.1.11.5.  NFS4ERR_DEADSESSION (Error Code 10078)

	The specified session is a persistent session that is dead and
	does not accept new requests or perform new operations on
	existing requests (in the case in which a request was partially
	executed before server restart).

The table in https://tools.ietf.org/html/rfc5661#section-8.4.2 gives it
as a valid error return for any operation, I think, except GETFH.

So actually, forget DELAY, the server returns DEADSESSION at the point
it stopped processing.  (With the one exception of GETFH as otherwise it
would be impossible to implement stuff like O_CREAT reliably.)

Thus the 2.10.6.5 language was clearly meant to treat each op (and
storage of its encoded result in the persistent DRC) as a transaction,
not to treat the entire compound as a transaction.

--b.