Re: [nfsv4] persistent sessions and compound atomicity

[Trond Myklebust <[e-mail]>]

On Fri, 2020-12-11 at 11:57 -0500, J. Bruce Fields wrote:
> On Thu, Dec 10, 2020 at 09:23:52PM -0500, Trond Myklebust wrote:
> > On Thu, 2020-12-10 at 17:07 -0500, David Noveck wrote:
> > > 
> > > 
> > > On Wed, Dec 9, 2020, 5:59 PM J. Bruce Fields
> > > <[e-mail]>
> > > wrote:
> > > > On Wed, Dec 09, 2020 at 04:43:59PM +0000, Rick Macklem wrote:
> > > > > I would say that the ATOMIC refers to the individual
> > > > > operations
> > > > of the
> > > > > compound. In other words, it must cache the reply to each
> > > > operation
> > > > > when it is completed, in the persistent cache.
> > > > > 
> > > > > Then, if a retry of the compound is received after a server
> > > > reboot,
> > > > > the reply for the first K operations that were completed
> > > > > prior to
> > > > the
> > > > > reboot are generated from the cache and the remaining N-K are
> > > > > performed on the file system to complete the reply. (At least
> > > > > for
> > > > the
> > > > > non-idempotent operations in the compound.)
> > > > > 
> > > > > For this to all work correctly, I think the server file
> > > > > system
> > > > would
> > > > > need to perform each non-idempotent operation ATOMICALLY as
> > > > > well,
> > > > such
> > > > > that after the reboot the file system is in either the pre-
> > > > operation
> > > > > or post-operation state.
> > > > 
> > > > Great, sounds like everyone agrees, I'll calm down.
> > > 
> > > I don't see the need for individual ops to be atomic especially
> > > since
> > > they (e.g. writes) are not defined as atomic in general.
> > > 
> > > I don't think we have total agreement but we are moving in the
> > > same
> > > direction.  No reason to be anything but calm.
> > 
> > in Section 2.10.6.5 of RFC5661, there is the following snippet:
> > 
> >    One way to
> >    view the problem is as a single transaction consisting of each
> >    operation in the COMPOUND followed by storing the result in
> >    persistent storage, then finally a transaction commit.  If there
> > is a
> >    failure before the transaction is committed, then the server
> > rolls
> >    back the transaction.  If the server itself fails, then when it
> >    restarts, its recovery logic could roll back the transaction
> > before
> >    starting the NFSv4.1 server.
> > 
> > Why shouldn't a client implementer reading the above be able to
> > conclude that a NFS4ERR_DEADSESSION error implies that the
> > operation
> > failed with no side-effects?
> 
> Yes--they can conclude that the one operation failed.  But previous
> operations may have succeeded and had side effects.
> 
> I had that wrong--I thought the spec said somewhere that DEADSESSION
> could only be returned to the initial SEQUENCE op, but actually it's
> quite explicit that it can be returned later:
> 
>         https://tools.ietf.org/html/rfc5661#section-18.46.4
> 
>         In the case of a partially executed COMPOUND, processing may
>         reach an operation not processed during the earlier server
>         instance, making this operation a new one and not performable
> on
>         the existing session.  In this case, NFS4ERR_DEADSESSION will
> be
>         returned from that operation.
> 
>         https://tools.ietf.org/html/rfc5661#section-15.1.11.5
> 
>         15.1.11.5.  NFS4ERR_DEADSESSION (Error Code 10078)
> 
>         The specified session is a persistent session that is dead
> and
>         does not accept new requests or perform new operations on
>         existing requests (in the case in which a request was
> partially
>         executed before server restart).
> 
> The table in https://tools.ietf.org/html/rfc5661#section-8.4.2 gives
> it
> as a valid error return for any operation, I think, except GETFH.
> 
> So actually, forget DELAY, the server returns DEADSESSION at the
> point
> it stopped processing.  (With the one exception of GETFH as otherwise
> it
> would be impossible to implement stuff like O_CREAT reliably.)
> 
> Thus the 2.10.6.5 language was clearly meant to treat each op (and
> storage of its encoded result in the persistent DRC) as a
> transaction,
> not to treat the entire compound as a transaction.

Sure. That's how I've been reading it as well.

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
[e-mail]