Re: [nfsv4] DESTROY_SESSION and clientid trunking

[<[e-mail]>]

> I don't get it--isn't it the *connections* associated with the session
> that become unresponsive, not the session itself?

OK, but if the single connection associated with a session or all the
connections associated with a session becomes unresponsive, then you
still have the same issue whether you are saying the session is
unresponsive or the connections.

I think I neglected to mention one important piece of context.  If you
have two sessions associated with a single clientid and this is the
result of the client on its own deciding on doing this form of clientid
trunking, then what you say is true.  

However, if the two sessions are established because the server returned
different so_minor_id values, then we have a different situation.  In
this case the server is telling you that two different IP's are not
capable of supporting a single session.  If that is the case, typically
it is because you have a clustered server such that the two IP's
terminate in different hardware which is incapable of sharing memory.
With shared memory, you would simply be able to support a common session
and its associated replay cache.

In such cases, it is very likely that a session or rather the set of all
sessions sharing a given so_minor_id, will become unresponsive, as the
result of various hardware problems in addition to bugs that you can
simply fix. 

> If you wanted EOS across sessions, then it seems to me that you'll end
> up creating another session abstraction on top of sessions.

That may seem to be a flaw in terms of abstraction management, but it is
in response to a practical requirement.  Clients want an EOS guarantee
for their requests.  It is simple (by which I mean we have already done
it) to provide it within the context of a single session.  However, if
one is forced to move requests from one session to another, and if one
can be forced into either doing this or having the requests wait
forever, then if one is not willing to accept the possibility of data
corruption, one has to address the issue of how to make sure the old
request cannot come to life.

Note that this doesn't fully address the issue of EOS across sessions
for non-idempotent requests and I think some v4.2 changes would be
required, but v4.1 does allow different so_minor_id value for clustered
servers, and it seems kind of rough to say you can only do this if you
accept the possibility of data corruption.

-----Original Message-----
From: J. Bruce Fields [mailto:[e-mail]] 
Sent: Monday, July 19, 2010 5:42 PM
To: Noveck, David
Cc: [e-mail]
Subject: Re: [nfsv4] DESTROY_SESSION and clientid trunking

On Mon, Jul 19, 2010 at 05:22:01PM -0400, [e-mail] wrote:
> This mail concerns the use of DESTROY_SESSION in the context of
clientid
> trunking.  Specifically, there are going to be cases in which a client
> will encounter a situation in which connections to one session are
> unresponsive, whether due to client-side networking issues, a wire
being
> cut, server-side networking issues, or a node of a clustered server
> being down.  No matter what the cause the client will need to transfer
> work from one session to another.  For simplicity, let's assume we
have
> clientid CL and with two sessions S1 and S2 and that S1 becomes
> unresponsive making it necessary to transfer work so that everything
for
> the client is being done on a single session S2.

I don't get it--isn't it the *connections* associated with the session
that become unresponsive, not the session itself?

If you wanted EOS across sessions, then it seems to me that you'll end
up creating another session abstraction on top of sessions.

And just fixing whatever bug is causing the session "unresponsiveness"
would seem simpler than the infinite regress....  But I probably don't
understand what you mean: how could a session become unresponsive?

--b.

> 
> The problem I'd like to consider is that of modifying-idempotent
> requests such as WRITE.  The question of non-idempotent requests such
as
> RENAME is harder and probably requires some work in v4.2, but the
> simpler case is important since WRITEs are typically much more common.
> 
> So the issue is that if you have a request issued on S1 and we'll use
> WRITE throughout as the example, and you get no response, then you
would
> like to issue the WRITE on the other session.  Since WRITEs are, in
the
> strict sense, idempotent, you don't have a problem if two WRITEs are
> issued but what is a problem is if there were a WRITE issued on S1,
and
> it was just lazily hanging around somewhere, and then if you did the
> WRITE on S2, and it completed then if the WRITE on S1 were to spring
> back to life, you have the likelihood of data corruption.
> 
> What you need is the assurance that the first WRITE (the one on S1),
> either succeed or failed but is not sitting around waiting to be done
at
> a later point.  You want to draw a line under all requests initiated
as
> part of S1.  I'm assuming the right way to do that is to destroy S1.
> Does anybody have any other ways to do that?  Things like closing the
> file will assure that the writes are done but they have unsatisfactory
> locking issues in that someone can open the file with a deny-mode and
> you might not be allowed to open it again.
> 
> However, there are some problems in the way that DESTROY_SESSION is
> specified and I think they need to be addressed, whether as errata in
> the v4.1 context or in the v4.2 context.
> 
> The first problem is that you would expect to have such a guarantee
> specified in the definition of DESTROY_SESSION and it isn't there.
You
> could argue that since it specifies that the reply information is
> dropped, it would be bad for requests to terminate in a situation
where
> there is session reply information to update, but that isn't ironclad.
> I think it makes sense for the initial paragraph to be changed to
> something like this:
> 
>    The DESTROY_SESSION operation closes the session and discards the
>    session's reply cache, if any.  All pending operations initiated on

>    the session are terminated and no additional ones can be started.
>    Thus, the requester is assured, once it receives the response, that
>    no operations initiated on the session will modify any file data 
>    or attributes or locking state associated with the client.
> 
>    Any remaining connections associated with the session are
immediately
> 
>    disassociated.  If the connection has no remaining associated
> sessions, 
>    the connection MAY be closed by the server.  Locks, delegations,
> layouts, 
>    wants, and the lease, which are all tied to the client ID, are not 
>    affected by DESTROY_SESSION.
> 
> What do people think?  Is this reasonable?  Is there some way we can
get
> by without it?
> 
> The other issue is that the spec says:
> 
>    DESTROY_SESSION MUST be invoked on a connection that is associated
>    with the session being destroyed.
> 
> So here I have a couple of questions:
> *	Is there a good reason for this?  Does it make sense?
> *	Is this a big problem for the issue we are talking about?
> 
> I believe the answers to both are "no", but I'm not sure.
> 
> The problem I'm worried about is that if you are doing a
DESTROY_SESSION
> to clean up from a non-responsive session, if you have to issue it on
> connections associated with that session.  So the problem would be
that
> the DESTROY_SESSION can only be sent when you don't need it to be
sent,
> i.e. when the destination IP's are functioning, but if they are, you
> don't want to send it.
> 
> There may be security issues but the aren't the security issues dealt
> with by the text that follows the text above.  Would there really be
any
> security issue allowing this on any connection that is associated with
a
> session that is part of the same clientid as the session being
> destroyed?  After all, you can do an EXCHANGE_ID with a new client
> verifier and get rid of all of all sessions as long as you can confirm
> it my creating one new session.  Given that you can doing that why
can't
> you destroy the session.
> 
> So one possible way of getting around this would be to associate
> connections that are part of S2 (let's call them with C2A and C2B)
with
> S1 just so they would be allowed in the context in which you need
them.
> While the text talks about the security issues of making sure that we
> have the right server, it doesn't talk about how you might reject the
> BIND_CONN_TO_SESSION because it is for a connection that is for the
> wrong so_minor_id.  Is the server obliged to reject this?  If it isn't
> then it might have a connection associated with a different session,
> e.g. C2A with S1.  If it allows that, is the server obliged to accept
S1
> requests (in general) on the same basis as it accepts S2 requests?  Or
> can it treat these specially, i.e. slow for normal requests (because
> they are routed over a cluster interconnect to the node normally
> handling S2), but capable of using clustering knowledge to respond to
> DESTROY_SESSION even when the S1 node is not functioning?
> 
> Is there anything in the spec that prevents this?  
> 
> 
> 
> _______________________________________________
> nfsv4 mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/nfsv4