[nfsv4] Review of 3530bis "Multi-Server Namespace" Chapter

James Lentini <jlentini@netapp.com> Wed, 22 September 2010 23:30 UTC

Return-Path: <jlentini@netapp.com>
X-Original-To: nfsv4@core3.amsl.com
Delivered-To: nfsv4@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4E95C28C0D0 for <nfsv4@core3.amsl.com>; Wed, 22 Sep 2010 16:30:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.46
X-Spam-Level:
X-Spam-Status: No, score=-4.46 tagged_above=-999 required=5 tests=[AWL=-0.873, BAYES_00=-2.599, J_CHICKENPOX_46=0.6, J_CHICKENPOX_74=0.6, RCVD_IN_DNSWL_MED=-4, SARE_SPEC_REPLICA_OBFU=1.812]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id afFix2wIeG-s for <nfsv4@core3.amsl.com>; Wed, 22 Sep 2010 16:30:49 -0700 (PDT)
Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by core3.amsl.com (Postfix) with ESMTP id 69EEA28C0F0 for <nfsv4@ietf.org>; Wed, 22 Sep 2010 16:30:49 -0700 (PDT)
X-IronPort-AV: E=Sophos;i="4.57,220,1283756400"; d="scan'208";a="456325063"
Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 22 Sep 2010 16:31:17 -0700
Received: from jlentini-linux.hq.netapp.com (jlentini-linux.hq.netapp.com [10.97.16.21]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id o8MNVGRT002594; Wed, 22 Sep 2010 16:31:16 -0700 (PDT)
Date: Wed, 22 Sep 2010 19:31:15 -0400
From: James Lentini <jlentini@netapp.com>
X-X-Sender: jlentini@jlentini-linux.nane.netapp.com
To: Thomas Haynes <thomas@netapp.com>
Message-ID: <alpine.LFD.2.00.1009221101060.21841@jlentini-linux.nane.netapp.com>
User-Agent: Alpine 2.00 (LFD 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset="US-ASCII"
Cc: nfsv4@ietf.org
Subject: [nfsv4] Review of 3530bis "Multi-Server Namespace" Chapter
X-BeenThere: nfsv4@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: NFSv4 Working Group <nfsv4.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/nfsv4>
List-Post: <mailto:nfsv4@ietf.org>
List-Help: <mailto:nfsv4-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 22 Sep 2010 23:31:00 -0000

At the last 3530bis meeting, I volunteered to review the "Muli-Server 
Namespace" chapter. Below are my comments on the version in 
draft-ietf-nfsv4-rfc3530bis-04.

RFC 5661 added a significant amount of text in this area, and I think 
the strategy of pulling this text back into 3530bis is the right one. 
Overall, it looks pretty good. Here are my detailed comments on the 
text:

One reference to v4.1's change_policy attribute slipped into Chapter 
7. Since change_policy isn't supported in v4, the reference to it 
should be removed from Section 7.6 "Additional Client-side 
Considerations" on page 83.

I have a few comments related to the file system equivalence classes 
in 7.7.

Section 7.7.5 introduces the concept a change equivalence class. 
Later, in Section 7.9.1, the text states that file system instances 
should always be considered to be of different change classes for v4. 
I'd consider removing section 7.7.5 since it isn't possible to 
communicate that file system instances are in an equivalent change 
class.

For the write-verifier class in Section 7.7.7, Section 7.9.1 
recommends assuming file system instances are of a different class in 
the case of a failover. Is there a recommendation for the 
NFS4ERR_MOVED case? I realize that the recommendations in 7.9.1 are 
taken almost verbatim from RFC 5661, but it seems that there is a case 
missing here.

No mention is made of the readdir class after Section 7.7.8. I 
recommend that either (A) Section 7.9.1 should suggest a way to infer 
the readdir class (it would be safest just to assume file system 
instances were not in the same readdir class) or (B) remove section 
7.7.8 and the readdir equivalence class concept (just as RFC 5661's 
"Simultaneous Use and Transparent Transitions" section describing the 
simultaneous-use class was removed in the 3530bis version).

One minor note on the typographical representation of the equivalence 
class identifiers. Throughout Chapter 7 equivalence class foo is 
represented as "_foo_" where the '_' characters are meant to denote an 
emphasis. Early drafts of v4.1 used this notation, but it was dropped 
in the published version of RFC 5661. I'd suggest removing the XML 
emphasis tags ("<spanx style='emph'>foo</spanx>") to match RFC 5661.

In Section 7.9's definition of a struct fs_location4, the server<> 
field's type was changed from utf8str_cis to utf8val_must. It seems 
that the original versions case-insensitive behavior is no longer 
being recorded. I suggest noting the case-insensitivity in the text 
description of an fs_location4 or creating a new type 
(utf8cis_val_must?).

Throughout Chapter 7, I noticed a few minor typos that I remembered 
sending corrections for in the run up to RFC 5661. Comparing a diff of 
Chapter 11 in RFC 5661 and Chapter 7 in the 3530bis draft leads me to 
conclude that the text in 3530bis was not taken from the final version 
of RFC 5661. As a results, it looks like some fixes from the WG and 
RFC Editor are missing in the 3530bis version. In the diff below, I've 
pulled the missing corrections into the 3530bis git repo's XML source 
file for Chapter 7. These changes are mostly just punctuation, missing 
words, etc.

That's all I see. Again, I think it looks pretty good overall.
-james

--- nfsv4_middle_mars.xml	2010-09-22 10:40:49.278584000 -0400
+++ nfsv4_middle_mars.xml	2010-09-22 14:40:03.703588000 -0400
@@ -6,8 +6,8 @@
     beyond the boundaries of a single server.  It is RECOMMENDED
     that clients and servers support construction of such
     multi-server namespaces.  Use of such multi-server namespaces 
-    is OPTIONAL however, and for many purposes,
-    single-server namespace are perfectly acceptable.  Use of
+    is OPTIONAL, however, and for many purposes,
+    single-server namespaces are perfectly acceptable.  Use of
     multi-server namespaces can provide many advantages, however, by
     separating a file system's logical position in a namespace from
     the (possibly changing) logistical and administrative
@@ -37,9 +37,9 @@
       a multi-server namespace) can have a number of file system instance 
       locations associated with it via the fs_locations
       attribute.  There may also be an actual current file system at 
-      that location, accessible via normal namespace operations (e.g.
+      that location, accessible via normal namespace operations (e.g.,
       LOOKUP).  In this case, the file system is said to be 
-      "present" at that position in the namespace and clients will 
+      "present" at that position in the namespace, and clients will 
       typically use it, reserving use of additional locations 
       specified via the location-related attributes to situations in
       which the principal location is no longer available.
@@ -56,7 +56,7 @@
     </t>
     <t>
       While the error name suggests that we have a case of a file system
-      which once was present, and has only become absent later, this is 
+      that once was present, and has only become absent later, this is 
       only one possibility.  A position in the namespace may be permanently
       absent with the set of file system(s) designated by the location 
       attributes being the only realization.  
@@ -70,7 +70,7 @@
       later), when the 
       current filehandle at the start of an operation is within an 
       absent file system, that operation is not performed and the error
-      NFS4ERR_MOVED returned, to indicate that the file system is
+      NFS4ERR_MOVED is returned, to indicate that the file system is
       absent on the current server.
     </t>
     <t>
@@ -119,9 +119,10 @@
         those that are normally REQUIRED, will not be available on an
         absent file system.  In addition to the fs_locations attribute,
         the following
-        attributes SHOULD be available on absent file systems, in the
-        case of RECOMMENDED attributes at least to the same degree that
-        they are available on present file systems.
+        attributes SHOULD be available on absent file systems.  In the
+        case of RECOMMENDED attributes, they should be available at 
+        least to the same degree that they are available on present 
+        file systems.
       </t>
       <t>
        <list style='hanging'>
@@ -136,8 +137,8 @@
         </t>
         <t hangText="mounted_on_fileid:"> 
           For objects at the top of an absent
-          file system this attribute needs to be available.  Since
-          the fileid is one which is within the present parent file
+          file system, this attribute needs to be available.  Since
+          the fileid is within the present parent file
           system, there should be no need to reference the absent file
           system to provide this information.
         </t>
@@ -152,7 +153,7 @@
       <t>
         When a GETATTR operation includes a bit mask for the
         attribute fs_locations, but
-        where the bit mask includes attributes which are not supported,
+        where the bit mask includes attributes that are not supported,
         GETATTR will not return an error, but will return the mask
         of the actual attributes supported with the results.
       </t>
@@ -162,7 +163,7 @@
         the error NFS4ERR_MOVED will result.  It differs in
         that any appearance in the attribute mask of an attribute not 
         supported for an absent file system (and note that this will
-        include some normally REQUIRED attributes), will also cause
+        include some normally REQUIRED attributes) will also cause
         an NFS4ERR_MOVED result.
       </t>
     </section>
@@ -192,7 +193,7 @@
           If the attribute set requested does not include 
           fs_locations, then if the
           rdattr_error attribute is requested, each directory entry for
-          the root of an absent file system, will report 
+          the root of an absent file system will report 
           NFS4ERR_MOVED as the value of the rdattr_error attribute.
         </t>
         <t>
@@ -226,8 +227,8 @@
       in the event of server failures, communications problems, 
       or other difficulties that make continued access to the current
       file system impossible or otherwise impractical.
-      Under some circumstances multiple alternative locations
-      may be used simultaneously to provide higher performance 
+      Under some circumstances, multiple alternative locations
+      may be used simultaneously to provide higher-performance 
       access to the file system in question.  
       Provision of
       such alternate locations is referred to as "replication"
@@ -240,8 +241,8 @@
       given the opportunity to have continued access to their data,
       at an alternate location.  In this case, a continued attempt
       to use the data in the now-absent file system will result 
-      in an NFS4ERR_MOVED error and at that point the successor 
-      locations (typically only one but multiple choices are possible)
+      in an NFS4ERR_MOVED error and, at that point, the successor 
+      locations (typically only one although multiple choices are possible)
       can be fetched and used to continue access.  Transfer of the
       file system contents to the new location is referred to as 
       "migration", but it should be kept in mind that there are cases
@@ -288,13 +289,17 @@
         reflect alternate paths to the same server or provide 
         for the use of various forms of server
         clustering in which multiple servers provide alternate 
-        ways of accessing the same physical file system.
+        ways of accessing the same physical file system. How these
+        different modes of file system transition are represented 
+        within the fs_locations attribute and how the client deals 
+        with file system transition issues will be discussed in 
+        detail below.
       </t>
       <t>
         Multiple server addresses, whether they are derived from 
         a single entry with a DNS name representing a set of IP
-        addresses, or from multiple entries each with its own 
-        server address may correspond to the same actual
+        addresses or from multiple entries each with its own 
+        server address, may correspond to the same actual
         server.
       </t>
     </section>
@@ -314,15 +319,19 @@
         does not specify how the file system will be moved between 
         servers.  It is anticipated that a number of different 
         server-to-server transfer mechanisms might be used with the
-        choice left to the server implementer.  The NFSv4 protocol
+        choice left to the server implementor.  The NFSv4 protocol
         specifies the method used to communicate the migration
         event between client and server.
       </t>
       <t>
         The new location may be an alternate
-        communication path to the same server, or, in the case of
+        communication path to the same server or, in the case of
         various forms of server clustering, another server providing
-        access to the same physical file system.
+        access to the same physical file system. The client's 
+        responsibilities in dealing with this transition depend on 
+        the specific nature of the new access path as well as how and 
+        whether data was in fact migrated.  These issues will be 
+        discussed in detail below.
       </t>
       <t>
         When an alternate location is designated as the target for
@@ -336,8 +345,8 @@
         propagation of updates.  Any change visible in the original
         file system must already be effected on all migration targets,
         to avoid any 
-        possibility, that a client in effecting a transition to 
-        the migration target will see any reversion in file system state.  
+        possibility that a client, in effecting a transition to 
+        the migration target, will see any reversion in file system state.  
       </t>
     </section>
     <section anchor="referrals" title="Referrals">
@@ -367,7 +376,7 @@
       <t>
         Use of multi-server namespaces is enabled by NFSv4 but is not
         required.  The use of multi-server namespaces and their scope
-        will depend on the applications used, and system administration
+        will depend on the applications used and system administration
         preferences. 
       </t>
       <t>
@@ -376,8 +385,8 @@
         included file systems.  Alternatively, a single multi-server
         namespace may be administratively segmented with separate
         referral file systems (on separate servers) for each
-        separately-administered portion of the namespace.  Any
-        segment or the top-level referral file system may use
+        separately administered portion of the namespace. The 
+        top-level referral file system or any segment may use
         replicated referral file systems for higher availability.  
       </t>
       <t>
@@ -392,10 +401,10 @@
            anchor="loc_server_id">
     <t>
       As mentioned above, a single location entry may have a server
-      address target in the form of a DNS name which may represent 
+      address target in the form of a DNS name that may represent 
       multiple 
       IP addresses, while multiple location entries may have their 
-      own server address targets, that reference the same server.
+      own server address targets that reference the same server.
     </t>
     <t>
       When multiple addresses for the same server exist, the client 
@@ -412,27 +421,27 @@
     <t>
       If a single location entry designates multiple server IP
       addresses, the client cannot assume that these addresses
-      are multiple paths to the same server.  In most case they 
+      are multiple paths to the same server.  In most cases, they 
       will be, but the client MUST verify that before acting on
       that assumption.  When two server addresses are designated
       by a single location entry and they correspond to different
       servers, this normally indicates some sort of misconfiguration,
-      and so the client should avoid use such location entries
+      and so the client should avoid using such location entries
       when alternatives are available.  When they are not, 
       clients should pick one of IP addresses and use it,
       without using others that are not directed to the same 
       server.
     </t> 
   </section>
-  <section title="Additional Client-side Considerations">
+  <section title="Additional Client-Side Considerations">
     <t>
       When clients make use of servers that implement referrals,
       replication, and
-      migration, care should be taken so that a user who mounts a given
+      migration, care should be taken that a user who mounts a given
       file system that includes a referral or a relocated file system
       continues to see a coherent picture of that user-side file system
       despite the fact that it contains a number of server-side
-      file systems which may be on different servers.
+      file systems that may be on different servers.
     </t>
     <t>
       One important issue is upward navigation from the root of a
@@ -441,12 +450,12 @@
       result of referral, migration, or a transition as a result of
       replication.  When the client is at such a point, and it needs to ascend to
       the parent, it must go back to the parent as seen within the
-      multi-server namespace rather issuing a LOOKUPP call to the
+      multi-server namespace rather than sending a LOOKUPP operation to the
       server, which would result in the parent within that server's
       single-server namespace.  In order to do this, the client
       needs to remember the filehandles that represent such
-      file system roots, and use these instead of issuing a 
-      LOOKUPP to the current server.  This will allow the client
+      file system roots and use these instead of sending a 
+      LOOKUPP operation to the current server.  This will allow the client
       to present to applications a consistent namespace, where 
       upward navigation and downward navigation are consistent.
     </t>
@@ -454,9 +463,9 @@
       Another issue concerns refresh of referral locations.  When
       referrals are used extensively, they may change as server
       configurations change.  It is expected that clients will cache
-      information related to traversing referrals so that future client
-      side requests are resolved locally without server communication.
-      This is usually rooted in client-side name lookup caching. Clients
+      information related to traversing referrals so that future client-side
+      requests are resolved locally without server communication.
+      This is usually rooted in client-side name look up caching. Clients
       should periodically purge this data for referral points in order to
       detect changes in location information.  When the change_policy
       attribute changes for directories that hold referral entries 
@@ -469,9 +478,9 @@
            title="Effecting File System Transitions">
     <t>
       Transitions between file system instances, whether due to
-      switching between replicas upon server unavailability, or 
-      in response to server-initiated migration events are best
-      dealt with together.  This is so even though for the server,
+      switching between replicas upon server unavailability or 
+      to server-initiated migration events, are best
+      dealt with together.  This is so even though, for the server,
       pragmatic considerations will normally force different 
       implementation strategies for planned and unplanned transitions. 
       Even though the prototypical use cases
@@ -494,17 +503,19 @@
     <t>
       The NFSv4 protocol does not impose choices on clients and
       servers with regard to that spectrum of transition methods.
+      In fact, there are many valid choices, depending on client and application 
+      requirements and their interaction with server implementation choices.
       The NFSv4.0 protocol does not provide the servers a means of
-      communicating the transiation methods. In the NFSv4.1 protocol
+      communicating the transition methods. In the NFSv4.1 protocol
       <xref target="RFC5661" />, an additional attribute "fs_locations_info"
       is presented, which will define the
       specific choices that can be made, how these choices are 
-      communicated to the client and how the client is to deal with
+      communicated to the client, and how the client is to deal with
       any discontinuities.
     </t>
     <t>
       In the sections below, references will be made to various possible
-      server issues as a way of illustrating the transition
+      server implementation choices as a way of illustrating the transition
       scenarios that clients may deal with.  The intent here is not to
       define or limit server implementations but rather to illustrate
       the range of issues that clients may face. Again, as the NFSv4.0
@@ -516,15 +527,15 @@
     </t>
     <t>
       In the discussion below, references will be made to a file system
-      having a particular property or of two file systems
+      having a particular property or to two file systems
       (typically the source and destination) belonging to a common
       class of any of several types.  Two file systems that belong to
-      such a class share some important aspect of file system behavior
+      such a class share some important aspects of file system behavior
       that clients may depend upon when present, to easily effect a
       seamless transition between file system instances.  Conversely,
       where the file systems do not belong to such a common class, the
       client has to deal with various sorts of implementation 
-      discontinuities which may cause performance or other issues in
+      discontinuities that may cause performance or other issues in
       effecting a transition.
     </t> 
     <t>
@@ -536,10 +547,10 @@
       In cases in which one server is expected to
       accept opaque values from the client that originated
       from another server, the servers SHOULD
-      encode the "opaque" values in big endian byte order.
+      encode the "opaque" values in big-endian byte order.
       If this is done, servers acting as replicas or immigrating 
       file systems will be able to parse values like stateids, directory cookies,
-      filehandles, etc. even if their native byte order is different from
+      filehandles, etc., even if their native byte order is different from
       that of other servers cooperating in the replication and migration 
       of the file system.
     </t>
@@ -547,19 +558,19 @@
              title="File System Transitions and Simultaneous Access">
       <t>
         When a single file system may be accessed at multiple locations,
-        whether this is because of an indication of file system identity
+        either because of an indication of file system identity
         as reported by the fs_locations attribute, the client
         will, depending on specific circumstances as discussed below, either:
       </t>
       <t>
        <list style='symbols'>
         <t>
-          The client accesses multiple instances simultaneously, as 
-          representing alternate paths to the same data and metadata.  
+          Access multiple instances simultaneously, each of which 
+          represents an alternate path to the same data and metadata.  
         </t>
         <t>
-          The client accesses one instance (or set of instances) and then
-          transitions to an alternative instance (or set of instances) 
+          Acesses one instance (or set of instances) and then
+          transition to an alternative instance (or set of instances) 
           as a result of network issues, server unresponsiveness, or
           server-directed migration.
         </t>
@@ -576,16 +587,16 @@
         effect some sort of continuity of file system handling.
       </t>
       <t>
-        When there is no such co-operation in filehandle assignment,
+        When there is no such cooperation in filehandle assignment,
         the two file systems are reported as being in different 
         <spanx style='emph'>handle</spanx> classes.  In this case,
         all filehandles are assumed to expire as part of the 
         file system transition.  Note that this behavior does not
-        depend on fh_expire_type attribute and depends on the specification
+        depend on the fh_expire_type attribute and depends on the specification
         of the FH4_VOL_MIGRATION bit.
       </t>
       <t>
-        When there is co-operation in filehandle assignment,
+        When there is cooperation in filehandle assignment,
         the two file systems are reported as being in the same
         <spanx style='emph'>handle</spanx> classes.  In this case,
         persistent filehandles remain valid after the file system
@@ -599,25 +610,25 @@
       <t>
         The issue of continuity of fileids in the event
         of a file system transition needs to be addressed.  The general 
-        expectation had been that in situations in
+        expectation is that in situations in
         which the two file system instances are created by a single vendor
         using some sort of file system image copy, fileids will be
-        consistent across the transition while in the analogous 
+        consistent across the transition, while in the analogous 
         multi-vendor transitions they will not.  This poses difficulties, 
         especially for the client without special knowledge  
         of the transition mechanisms adopted by the server.  Note
         that although fileid is not a REQUIRED attribute, many servers
-        support fileids and many clients provide API's that depend on fileids.
+        support fileids and many clients provide APIs that depend on fileids.
       </t>
       <t>
         It is important to note that while clients themselves may have no
         trouble with a fileid changing as a result of a file system
         transition event, applications do typically have access to the
-        fileid (e.g. via stat), and the result of this is that an
+        fileid (e.g., via stat). The result is that an
         application may work perfectly well if there is no file system
         instance transition or if any such transition is among instances
         created by a single vendor, yet be unable to deal with the
-        situation in which a multi-vendor transition occurs, at the wrong
+        situation in which a multi-vendor transition occurs at the wrong
         time.
       </t>
       <t>
@@ -656,7 +667,7 @@
         that files it was accessing before the transition are the 
         same objects.  It is forced to assume that no object has been 
         renamed, and, unless there are guarantees that provide this
-        (e.g. the file system is read-only), problems for applications
+        (e.g., the file system is read-only), problems for applications
         may occur.  Therefore, use of such configurations should be 
         limited to situations where the problems that this may cause
         can be tolerated.
@@ -683,7 +694,7 @@
         since it would invalidate all cached change attributes, requiring
         refetching.  Even more disruptive, the absence of any assured
         continuity for the change attribute means that even if the same
-        value is retrieved on refetch no conclusions can drawn as to whether
+        value is retrieved on refetch, no conclusions can be drawn as to whether
         the object in question has changed.  The identical change 
         attribute could be merely an artifact of a modified file with
         a different change attribute construction algorithm, with that
@@ -732,19 +743,19 @@
         the transitioned file system instance.
       </t>
       <t>
-        File systems co-operating in state management may actually
+        File systems cooperating in state management may actually
         share state or simply divide the identifier space so as to recognize
         (and reject as stale) each other's stateids and client IDs.
-        Servers which do share state may not do so under all conditions
-        or at all times.  The requirement for the server is that if it
-        cannot be sure in accepting a client ID that it reflects the locks
-        the client was given, it must treat all associated state as 
+        Servers that do share state may not do so under all conditions
+        or at all times.  If the server cannot be sure when accepting a 
+        client ID that it reflects the locks the client was given, the 
+        server must treat all associated state as 
         stale and report it as such to the client.
       </t>
       <t>
         The client must establish a new client ID on the destination, if
         it does not have one already, and reclaim locks if 
-        possible.  In this case, old stateids and client IDs should
+        allowed by the server.  In this case, old stateids and client IDs should
         not be presented to the new server since there is no assurance
         that they will not conflict with IDs valid on that server. 
       </t>
@@ -765,7 +776,7 @@
         be avoided by securely recording lock state as part of state 
         migration).  Unless the destination server can guarantee that
         locks will not be incorrectly granted, the destination server
-        should not allow lock reclaims and avoid establishing a grace
+        should not allow lock reclaims and should avoid establishing a grace
         period. (See <xref target="sec:mig_state" /> for further details.)
       </t>
       <t>
@@ -784,7 +795,7 @@
       </t>
       <t>
         The consequences of having no facilities available to 
-        reclaim locks on the sew server will depend on the type
+        reclaim locks on the new server will depend on the type
         of environment.  In  
         some environments, such as the transition between read-only
         file systems, such denial of locks should not pose large 
@@ -802,7 +813,7 @@
       <section anchor="transition_lease_time"
                title="Transitions and the Lease_time Attribute">
         <t>
-          In order that the client may appropriately manage its leases 
+          In order that the client may appropriately manage its lease 
           in the case of a file system transition, the destination server must 
           establish proper values for the lease_time attribute.
         </t>
@@ -810,11 +821,11 @@
           When state is transferred transparently, that state 
           should include the correct value of the lease_time 
           attribute.  The lease_time attribute on the destination 
-          server must never be less than that on the source since 
-          this would result in premature expiration of leases
+          server must never be less than that on the source, since 
+          this would result in premature expiration of a lease
           granted by the source server.  Upon transitions in which 
           state is transferred transparently, the client is under 
-          no obligation to re-fetch the lease_time attribute and 
+          no obligation to refetch the lease_time attribute and 
           may continue to use the value
           previously fetched (on the source server).
         </t>
@@ -822,9 +833,9 @@
           If state has not been transferred transparently
           because the client ID is rejected when presented to
           the new server, the client should fetch the value
-          of lease_time on the new (i.e. destination) server, and 
-          use it for subsequent locking requests.  However the server 
-          must respect a grace period at least as long as the lease_time
+          of lease_time on the new (i.e., destination) server, and 
+          use it for subsequent locking requests.  However, the server 
+          must respect a grace period of at least as long as the lease_time
           on the source server, in order to ensure that clients have ample time to 
           reclaim their lock before potentially conflicting 
           non-reclaimed locks are granted.  
@@ -880,9 +891,9 @@
       <t>
         When multiple replicas exist and are used simultaneously or in
         succession by a client, applications using them will normally expect
-        that they contain data the same data or data which is consistent with
+        that they contain either the same data or data that is consistent with
         the normal sorts of changes that are made by other clients
-        updating the data of the file system.
+        updating the data of the file system
         (with metadata being the same to the degree inferred by the
         fs_locations attribute).  However, when multiple file systems are 
         presented as replicas of one another, the precise relationship 
@@ -893,9 +904,9 @@
         have problems dealing with the transition between replicas.  The 
         namespace will typically be constructed so that applications can 
         choose an appropriate level of support, so that in one position in 
-        the namespace a varied set of replicas will be listed while in 
+        the namespace a varied set of replicas will be listed, while in 
         another only those that are up-to-date may be considered replicas.  
-        The protocol does define three special cases of the relationship among 
+        The protocol does define four special cases of the relationship among 
         replicas to be specified by the server and relied upon by clients:
       </t>
       <t>
@@ -916,8 +927,8 @@
             or the visibility of that change on any of the associated
             replicas.  This allows a client to use these replicas
             simultaneously without any special adaptation to the fact
-            that there are multiple replicas.  In this case, locks,
-            whether shared or byte-range, and delegations obtained one
+            that there are multiple replicas.  In this case, locks
+            (whether share reservations or byte-range locks), and delegations obtained on one
             replica are immediately reflected on all replicas, even
             though these locks will be managed under a set of client
             IDs.
@@ -926,9 +937,9 @@
             When one replica is designated as the successor instance to another
             existing instance after return NFS4ERR_MOVED (i.e. the case of 
             migration), the client may depend on the fact that all changes
-            securely made to data (uncommitted writes are dealt with in 
-            <xref target="transition_verifier" />) on the original instance
-            are made to the successor image.
+            written to stable storage on the original instance are written
+            to stable storage of the successor (uncommitted writes are dealt
+            with in <xref target="transition_verifier" />).
           </t>
           <t>
             Where a file system is not writable but represents a read-only 
@@ -940,10 +951,10 @@
             in order to 
             avoid any possibility that a client, in effecting a transition to a
             replica, will see any reversion in file system state.
-            Since these file systems are presumed not to be  
-            suitable for simultaneous use, there is no specification of how 
-            locking is handled and it generally will be the case that locks 
-            obtained one file system will be separate from those on others.  
+            Since these file systems are presumed to be  
+            unsuitable for simultaneous use, there is no specification of how 
+            locking is handled; in general, locks 
+            obtained on one file system will be separate from those on others.  
             Since these are going to be read-only file systems, this is not 
             expected to pose an issue for clients or applications.
           </t>
@@ -958,7 +969,7 @@
       and one or more alternate locations are made available by the
       fs_locations attribute.  The client will
       typically get an NFS4ERR_MOVED error, fetch the appropriate 
-      location information and proceed to access the file system on
+      location information, and proceed to access the file system on
       a different server, even though it retains its logical position
       within the original namespace.  Referrals differ from migration
       events in that they happen only when the client has not 
@@ -969,7 +980,7 @@
     <t>
       The examples given in the sections below are somewhat artificial in
       that an actual client will not typically do a multi-component
-      lookup, but will have cached information regarding the upper levels
+      look up, but will have cached information regarding the upper levels
       of the name hierarchy.  However, these example are chosen to make
       the required behavior clear and easy to put within the scope of a
       small number of requests, without getting unduly into details of
@@ -982,7 +993,7 @@
         environment in which /this/is/the/path is absent from the
         target server.  This may be for a number of reasons.  It may 
         be the case that the 
-        file system has moved, or, it may be the case that
+        file system has moved, or it may be that
         the target server is functioning mainly, or solely, to refer
         clients to the servers on which various file systems are located.
       </t>
@@ -1007,7 +1018,7 @@
           GETFH
         </t>
         <t>
-          GETATTR fsid,fileid,size,time_modify
+          GETATTR (fsid,fileid,size,time_modify)
         </t>
        </list>
       </t>
@@ -1040,10 +1051,10 @@
         <t>
           GETFH --> NFS4ERR_MOVED.
           Fails because current fh is in an absent file system at the start of
-          the operation and the spec makes no exception for GETFH.
+          the operation, and the specification makes no exception for GETFH.
         </t>
         <t>
-          GETATTR fsid,fileid,size,time_modify.
+          GETATTR (fsid,fileid,size,time_modify).
           Not executed because the failure of the GETFH stops processing
           of the COMPOUND.
         </t>
@@ -1052,13 +1063,13 @@
       <t>
         Given the failure of the GETFH, the client has the job of
         determining the root of the absent file system and where to find
-        that file system, i.e. the server and path relative to that
+        that file system, i.e., the server and path relative to that
         server's root fh.  Note here that in this example, the client did
-        not obtain filehandles and attribute information (e.g. fsid) for
+        not obtain filehandles and attribute information (e.g., fsid) for
         the intermediate directories, so that it would not be sure where
         the absent file system starts.  It could be the case, for example,
         that /this/is/the is the root of the moved file system and that
-        the reason that the lookup of "path" succeeded is that the
+        the reason that the look up of "path" succeeded is that the
         file system was not absent on that operation but was moved between the last
         LOOKUP and the GETFH (since COMPOUND is not atomic).  Even if we
         had the fsids for all of the intermediate directories, we could
@@ -1071,7 +1082,7 @@
         fsids so we can be sure where the appropriate file system boundaries are.
         The client could choose to get fs_locations at the same time but in
         most cases the client will have a good guess as to where file system
-        boundaries are (because of where and where not NFS4ERR_MOVED was
+        boundaries are (because of where NFS4ERR_MOVED was, and was not,
         received) making fetching of fs_locations unnecessary.
       </t>
       <t>
@@ -1155,18 +1166,18 @@
           absent file system, but ...
         </t>
         <t hangText='- '>
-          The client will never see the value of that fh
+          The client will never see the value of that fh.
         </t>
         <t hangText='OP13:'>
           GETATTR(fsid, fs_locations)  --> NFS_OK
         </t>
         <t hangText='- '>
           We are getting the fsid to know where the file system boundaries are.
-          In this operation the fsid will be different than that of the
+          In this operation, the fsid will be different than that of the
           parent directory (which in turn was retrieved in OP10).
           Note that the
           fsid we are given will not necessarily be preserved at the new
-          location.  That fsid might be different and in fact the fsid
+          location.  That fsid might be different, and in fact the fsid
           we have for this file system might be a valid fsid of a different 
           file system on that new server.
         </t>
@@ -1176,19 +1187,19 @@
           since we have the fsid of the latter and it is that of the
           pseudo-fs, which presumably cannot move.  However, in other
           examples, we might not have this kind of information to rely
-          on (e.g. /this/is/the might be a non-pseudo file system
+          on (e.g., /this/is/the might be a non-pseudo file system
           separate from /this/is/the/path), so we need to have
-          another reliable source information on the boundary of the file system
-          which is moved.  If, for example, the file system "/this/is"
-          had moved we would have a case of migration rather than
-          referral and once the boundaries of the migrated file system
+          other reliable source information on the boundary of the file system
+          that is moved.  If, for example, the file system /this/is
+          had moved, we would have a case of migration rather than
+          referral, and once the boundaries of the migrated file system
           was clear we could fetch fs_locations.
         </t>
         <t hangText='- '>
           We are fetching fs_locations because the fact that we got an
-          NFS4ERR_MOVED at this point means that it most likely that
+          NFS4ERR_MOVED at this point means that it is most likely that
           this is a referral and we need the destination.  Even if it is
-          the case that "/this/is/the" is a file system which has
+          the case that /this/is/the is a file system thathatt has
           migrated, we will still need the location information for that
           file system.
         </t>
@@ -1197,7 +1208,7 @@
         </t>
         <t hangText='- '>
           Fails because current fh is in an absent file system at the start of
-          the operation and the spec makes no exception for GETFH.  Note
+          the operation, and the specification makes no exception for GETFH.  Note
           that this means the server will never send the client a
           filehandle from within an absent file system.
         </t>
@@ -1205,7 +1216,7 @@
       </t>
       <t>
         Given the above, the client knows where the root of the absent file
-        system is (/this/is/the/path), by noting where the change of fsid
+        system is (/this/is/the/path) by noting where the change of fsid
         occurred (between "the" and "path").  The fs_locations attribute
         also gives the client the actual location of
         the absent file system, so that the referral can proceed.  The
@@ -1213,8 +1224,8 @@
         absent file system so that there will be very little scope for
         problems of conflict between information sent by the referring
         server and information of the file system's home.  No filehandles
-        and very few attributes are present on the referring server and the
-        client can treat those it receives as basically transient
+        and very few attributes are present on the referring server, and the
+        client can treat those it receives as transient
         information with the function of enabling the referral.
       </t>
     </section>
@@ -1222,7 +1233,7 @@
              title="Referral Example (READDIR)">
       <t>
         Another context in which a client may encounter referrals is when
-        it does a READDIR on directory in which some of the sub-directories
+        it does a READDIR on a directory in which some of the sub-directories
         are the roots of absent file systems.
       </t>
       <t>
@@ -1242,14 +1253,14 @@
         <t>
           LOOKUP "the"
         </t>
-        <t>
+        <t
           READDIR (fsid, size, time_modify, mounted_on_fileid)
         </t>
        </list>
       </t>
       <t>
         In this case, because rdattr_error is not requested, fs_locations
-        is not requested, and some of attributes cannot be provided, the
+        is not requested, and some of the attributes cannot be provided, the
         result will be an NFS4ERR_MOVED error on the READDIR, with the
         detailed results as follows:
       </t>
@@ -1274,7 +1285,7 @@
         <t>
           READDIR (fsid, size, time_modify, mounted_on_fileid) -->
           NFS4ERR_MOVED.  Note that the same error would have been 
-          returned if /this/is/the had migrated, when in fact it is because the
+          returned if /this/is/the had migrated, but it is returned because the
           directory contains the root of an absent file system.
         </t>
        </list>
@@ -1385,7 +1396,7 @@
       </t>
       <t>
          The attributes for the directory entry with the
-         component named "path" will only contain
+         component named "path" will only contain:
       </t>
       <t>
        <list style='symbols'>
@@ -1428,7 +1439,7 @@
       same path within their namespaces, an array of server names may 
       be provided.  An entry in the server array is a UTF-8 string
       and represents one of a traditional DNS host name, IPv4 address,
-      or IPv6 address, or an zero-length string.  
+      IPv6 address, or a zero-length string.  
       A zero-length string SHOULD be used to indicate the current address 
       being used for the RPC call. It is not
       a requirement that all servers that share the same rootpath 
@@ -1443,14 +1454,14 @@
      constructed differently, the "fs_root" field is provided.  The 
      path represented
      by fs_root represents the location of the file system in the 
-     current server's namespace, i.e. that  of the
+     current server's namespace, i.e., that  of the
      server from which the fs_locations attribute was obtained.  The
      fs_root path is meant to aid the client by clearly referencing
      the root of the file system whose locations are being reported,
      no matter what object within the current file system the 
      current filehandle designates.  The fs_root is simply the
-     pathname the client used to reach the object on the current server,
-     the object being that the fs_locations attribute applies to.
+     pathname the client used to reach the object on the current server
+     (i.e., the object to which the fs_locations attribute applies).
     </t>
     <t>
      When the fs_locations attribute
@@ -1462,30 +1473,30 @@
      As an example, suppose there is a replicated file system located 
      at two
      servers (servA and servB).  At servA, the file system is located at
-     path "/a/b/c".  At, servB the file system is located at path "/x/y/z".
+     path /a/b/c.  At, servB the file system is located at path /x/y/z.
      If the client were to obtain the fs_locations value for the
-     directory at "/a/b/c/d", it might not necessarily know  
+     directory at /a/b/c/d, it might not necessarily know  
      that the file system's root is located in servA's namespace 
-     at "/a/b/c".  When the client switches to servB, it will need
+     at /a/b/c.  When the client switches to servB, it will need
      to determine that the directory it first referenced at servA is now
-     represented by the path "/x/y/z/d" on servB.  To facilitate this, the
-     fs_locations attribute provided by servA would have a fs_root value
-     of "/a/b/c" and two entries in fs_locations.  One entry in fs_locations
+     represented by the path /x/y/z/d on servB.  To facilitate this, the
+     fs_locations attribute provided by servA would have an fs_root value
+     of /a/b/c and two entries in fs_locations.  One entry in fs_locations
      will be for itself (servA) and the other will be for servB with a
-     path of "/x/y/z".  With this information, the client is able to
-     substitute "/x/y/z" for the "/a/b/c" at the beginning of its access
-     path and construct "/x/y/z/d" to use for the new server.
+     path of /x/y/z.  With this information, the client is able to
+     substitute /x/y/z for the /a/b/c at the beginning of its access
+     path and construct /x/y/z/d to use for the new server.
    </t>
    <t>
      Note that: there is no requirement that the number
      of components in each rootpath be the same; there
      is no relation between the number of components in
-     rootpath or fs_root; and the none of the components
+     rootpath or fs_root, and none of the components
      in each rootpath and fs_root have to be the same. In
      the above example, we could have had a third element
-     in the locations array, with server equal to "servC",
+     in the locations array, with server equal to "servC"
      and rootpath equal to "/I/II", and a fourth element in
-     locations with server equal to "servD", and rootpath
+     locations with server equal to "servD" and rootpath
      equal to "/aleph/beth/gimel/daleth/he".
    </t>
    <t>
@@ -1495,22 +1506,22 @@
      indicated in rootpath for the new server.
    </t>
    <t>
-     For an example for a referred or migrated file
+     For an example of a referred or migrated file
      system, suppose there is a file system located
      at serv1. At serv1, the file system is located at
-     "/az/buky/vedi/glagoli". The client finds that object
-     at "glagoli" has migrated (or is a referral).  The
+     /az/buky/vedi/glagoli. The client finds that object
+     at glagoli has migrated (or is a referral).  The
      client gets the fs_locations attribute, which contains
-     an fs_root of "/az/buky/vedi/glagoli", and one element
-     in the locations array, with server equal to "serv2",
-     and rootpath equal to "/izhitsa/fita". The client
-     replaces "/az/buky/vedi/glagoli" with "/izhitsa/fita",
-     and uses the latter pathname on "serv2".
+     an fs_root of /az/buky/vedi/glagoli, and one element
+     in the locations array, with server equal to serv2,
+     and rootpath equal to /izhitsa/fita. The client
+     replaces /az/buky/vedi/glagoli with /izhitsa/fita,
+     and uses the latter pathname on serv2.
    </t>
    <t>
      Thus, the server MUST return an fs_root that is equal
-     to the path the client used to reach the object the
-     fs_locations attribute applies to. Otherwise the
+     to the path the client used to reach the object to which the
+     fs_locations attribute applies. Otherwise, the
      client cannot determine the new path to use on the new server.
    </t>
     <section anchor="fs_locations_trans" title="Inferring Transition Modes">
@@ -1537,7 +1548,7 @@
        </t> 
        <t>
          All listed file system instances should be considered as of the 
-         same <spanx style='emph'>fileid</spanx> class, if and only if, the 
+         same <spanx style='emph'>fileid</spanx> class if and only if the 
          fh_expire_type attribute indicates persistent filehandles and 
          does not include the FH4_VOL_MIGRATION
          bit.  Note that in the case of referral, fileid issues do
@@ -1559,8 +1570,8 @@
      <t>
       <list style='symbols'>
        <t>
-         When the transition is due to migration, that is the client was
-         directed to new file system after receiving an NFS4ERR_MOVED error,
+         When the transition is due to migration, that is, the client was
+         directed to a new file system after receiving an NFS4ERR_MOVED error,
          the target should be
          treated as being of the same  
          <spanx style='emph'>write-verifier</spanx> class as the source.
@@ -1576,7 +1587,7 @@
      </t>
      <t>
        The specific choices reflect typical implementation patterns for
-       failover and controlled migration respectively.
+       failover and controlled migration, respectively.
      </t>
       <t>
        See <xref target="sec:security" /> for a