Re: [nfsv4] DESTROY_SESSION and clientid trunking

["J. Bruce Fields" <[e-mail]>]

On Mon, Jul 26, 2010 at 06:02:59PM -0400, [e-mail] wrote:
> Leaving aside the case of read-only data, everybody needs true EOS,
> whether they are attentive enough to want it or not.  But v4.1 does not
> make true EOS with persistent sessions a mandatory feature, probably
> because of implementation difficulty.  
> 
> Now in our example of an unresponsive server-minor, if there is support
> to update the session reply cache on disk or on another backup piece of
> hardware, then we have EOS in this case.  It doesn't show up as
> persistent sessions but rather as connections becoming unresponsive and
> when they are re-established, they will wind up on an alternate piece of
> hardware with the correct reply cache being available, whether from disk
> or otherwise.
> 
> But just as we deal with servers that don't have persistent sessions in
> the server-major case, we should deal as well as we can, when we have
> the case of a server-minor which does not have the ability to store its
> reply cache information persistently.  I agree that people need EOS for
> this case, but we do have to deal with the case in which we don't have
> the needed infrastructure.  In the case, I'm thinking about a case in
> which we have unresponsive connections to a server-minor and we need to
> be able assure that operations previously started on these are either
> done or not and what is critical is that they are not done in the
> future.  

OK, so we could call the sort of failover you're talking about
cross-minor_id-failover, since it occurs between two servers with
different minor_id's (but the same major_id)?

And we're assuming a server implementation for which:

	- It is hard to implemented shared sessions (shared minor_id)
	- It is easy for one server to fence off another (meaning, it
	  can perform some operation on completion of which it knows no
	  further writes from the other server will hit the filesystem).
	- Full EOS across cross-minor_id-failover is not required.
	- Preventing out-of-order writes over such failover is.

With "hard" and "easy" defined however strongly necessary to justify the
exercise.  OK, makes sense, thanks for the patient explanation!

My concerns:

	- Just one data point: looking at the linux 4.1 server, I
	  haven't been personally worrying about making DESTROY_SESSION
	  wait for anything.  I don't consider the implementation done,
	  so that's fixable, but maybe it's some evidence that (given a
	  sufficiently dim implementor) readers of the current spec
	  won't be doing this, and hence it may be difficult to just
	  slip this in as a clarification of the semantics of
	  DESTROY_SESSION.
	- Your initial suggestion seemed to be that we would eventually
	  build on whatever solution we chose here as a way of providing
	  full EOS across this sort of failover.  I worry that the
	  result would no longer be significantly easier to implement
	  than shared sessions, so wonder if it would be worth the
	  additional protocol.

> You could do this explicitly  with an op that take a server-owner or you
> could to this at the session level.  It seems to me that doing that as
> part of DESTROY_SESSION makes sense, but you would have to deal with the
> current spec issues that make this difficult or impossible.  My feeling
> is DESTROY_SESSION needs to assure people that ops started under that
> session are not going to execute after the DESTROY_SESSION completes,

Right, so that part wasn't obvious to me: I assumed the DESTROY_SESSION
was the client's way of saying: "I give up; I no longer care what
happens to these operations; let's just drop everything and get on with
our lives".  Also, if we've lost contact with a disk it may be nice to
be able to destroy the state and let the client continue unmounting (or
whatever it's doing) without having to wait forever.  I don't know.

> and I would like to understand if there is a good reason for the current
> restriction as to what connection the op can be issued on.

Yeah, I did a little work on implementing that but honestly I never
understood the point either.  My first thought was that it was to ensure
that a rogue client wanting to destroy the session would be required to
do a tcp injection attack.  But surely it could just associate another
connection first instead.

--b.