Re: [nfsv4] disposition of inflight NFSv4.1 operations when a session or client ID is destroyed

<david.noveck@emc.com> Thu, 14 October 2010 10:41 UTC

Return-Path: <david.noveck@emc.com>
X-Original-To: nfsv4@core3.amsl.com
Delivered-To: nfsv4@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 0C1E13A67A7 for <nfsv4@core3.amsl.com>; Thu, 14 Oct 2010 03:41:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.677
X-Spam-Level:
X-Spam-Status: No, score=-6.677 tagged_above=-999 required=5 tests=[AWL=-0.078, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2F35ofZy93-t for <nfsv4@core3.amsl.com>; Thu, 14 Oct 2010 03:41:09 -0700 (PDT)
Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by core3.amsl.com (Postfix) with ESMTP id 6CDC63A6868 for <nfsv4@ietf.org>; Thu, 14 Oct 2010 03:41:09 -0700 (PDT)
Received: from hop04-l1d11-si02.isus.emc.com (HOP04-L1D11-SI02.isus.emc.com [10.254.111.55]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id o9EAgP7R013546 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 14 Oct 2010 06:42:25 -0400
Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.226]) by hop04-l1d11-si02.isus.emc.com (RSA Interceptor); Thu, 14 Oct 2010 06:42:20 -0400
Received: from corpussmtp3.corp.emc.com (corpussmtp3.corp.emc.com [10.254.169.196]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id o9EAfrEn008997; Thu, 14 Oct 2010 06:41:54 -0400
Received: from CORPUSMX50A.corp.emc.com ([128.221.62.41]) by corpussmtp3.corp.emc.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 14 Oct 2010 06:41:54 -0400
X-MimeOLE: Produced By Microsoft Exchange V6.5
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Date: Thu, 14 Oct 2010 06:41:52 -0400
Message-ID: <BF3BB6D12298F54B89C8DCC1E4073D80028515CB@CORPUSMX50A.corp.emc.com>
In-Reply-To: <9a0c3fae91b577f987b0209ac1b91753.squirrel@webmail.eisler.com>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Thread-Topic: [nfsv4] disposition of inflight NFSv4.1 operations when a session or client ID is destroyed
Thread-Index: ActrJ/QPOEI3LPXdR6Ss6zduYsqnlQAXd15Q
References: <9a0c3fae91b577f987b0209ac1b91753.squirrel@webmail.eisler.com>
From: david.noveck@emc.com
To: mre-ietf@eisler.com, nfsv4@ietf.org
X-OriginalArrivalTime: 14 Oct 2010 10:41:54.0547 (UTC) FILETIME=[6B07FC30:01CB6B8C]
Subject: Re: [nfsv4] disposition of inflight NFSv4.1 operations when a session or client ID is destroyed
X-BeenThere: nfsv4@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: NFSv4 Working Group <nfsv4.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/nfsv4>
List-Post: <mailto:nfsv4@ietf.org>
List-Help: <mailto:nfsv4-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Oct 2010 10:41:11 -0000

This issue has been vaguely alluded to before but there is no previous
that actually tackles it.  Your message is not a dup.

I don't see how a client can go on without the assurance that operations
started under a previous session or clientid, cannot just start up again
and screw up any assumptions you are making.  The protocol is designed
so that if such messages were stuck in the network, they would be
discarded.  To me, the fact that this issue is not mentioned for
operations in-process on the server is just a plain oversight.  I'd fix
it via the errata mechanism.

This oversight is clearly either your fault or my fault or Spencer's
fault.  We could have a big finger-pointing session in Beijing :-).  Or
we could go have a drink and celebrate the fact that instances of this
sort of oversight have been pretty small given the size and complexity
of the spec.  I vote for the latter. 

If this is judged too big/substantive for the errata mechanism, then I
would favor the capability as second best.  But the question is what do
you say about clients and servers that don't provide this.   The
following is what I would want to, but couldn't, say:

     "Those writing clients that do not set this bit or 
      that are allowed to interact with servers that don't 
      set it, should immediately contact a mental health
      professional." 


> Another question is, even if you think the server MUST 
> abort/wait for in-flight requests, does your NFSv4.1 server 
> do that?

I don't know.  I intend to find out.

The fact that this is not mentioned in RFC5661 does raise the chances
that implementations did not address this.  While, once you raise the
question, the answer should be clear, someone asked to implement the
spec might very well not think of this issue.

So this gives rise to another reason for the capability bit.  It creates
the occasion for someone to really look at the implementation and decide
to say "we have this Halloween server and if you dig being afraid of
weird protocol interactions with dead sessions or server instances, this
is for you."  or "we've looked at this and you can relax.  No need to
take anxiolytic drugs.", presumably making whatever fixes are necessary
to be able to make the second choice.
  
-----Original Message-----
From: nfsv4-bounces@ietf.org [mailto:nfsv4-bounces@ietf.org] On Behalf
Of Mike Eisler
Sent: Wednesday, October 13, 2010 6:41 PM
To: nfsv4@ietf.org
Subject: [nfsv4] disposition of inflight NFSv4.1 operations when a
session or client ID is destroyed

Apologies if this is a dup.

What do people think should happen to operations that are in flight when
DESTROY_SESSION kills a session or EXCHANGE_ID/CREATE_SESSION
causes an existing client ID to be destroyed (thus destroying
the sessions of s previous client instance)?

- Undefined, or
- Server MUST abort or wait for in flight operations to complete

The internet-draft Margaret Susairaj and I wrote proposes a new
capability to EXCHANGE_ID to ensure that the latter happens. See
http://www.ietf.org/id/draft-eisler-nfsv4-enterprise-apps-00.txt .

Excerpt from the I-D:

   A client MAY request the EXCHGID4_FLAG_SUPP_FENCE_OPS capability when
   it sends an EXCHANGE_ID operation.  The server MAY set this
   capability in the EXCHANGE_ID request whether the client requests it
   or not.  If the client ID is created with this capability then the
   following will occur:

   o  The server will not reply to DESTROY_SESSION until all operations
      in progress are completed or aborted.

   o  The server will not reply to subsequent EXCHANGE_ID invoked on the
      same Client Owner with a new verifier until all operations in
      progress on the Client ID's session are completed or aborted.

   o  When DESTROY_CLIENTID is invoked, if there are sessions (both idle
      and non-idle), opens, locks, delegations, layouts, and/or wants
      (Section 18.49) associated with the client ID are removed.
      Pending operations will be completed or aborted before the
      sessions, opens, locks, delegations, layouts, and/or wants are
      deleted.

   o  The NFS server SHOULD support client ID trunking, and if it does
      and the EXCHGID4_FLAG_SUPP_FENCE_OPS capability is enabled, then a
      session ID created on one node of the storage cluster MUST be
      destroyable via DESTROY_SESSION.  In addition, DESTROY_CLIENTID
      and an EXCHANGE_ID with a new verifier affects all sessions
      regardless what node the sessions were created on.

=======================================================

But it raises the question if this is necessary.

Another question is, even if you think the server MUST abort/wait for
inflight requests, does your NFSv4.1 server do that?

--
Mike Eisler, Senior Technical Director, CR & NFS
+1 719 599 9026 - blogs.netapp.com/eislers_nfs_blog/


_______________________________________________
nfsv4 mailing list
nfsv4@ietf.org
https://www.ietf.org/mailman/listinfo/nfsv4