Re: [nfsv4] Benjamin Kaduk's Discuss on draft-ietf-nfsv4-mv0-trunking-update-03: (with DISCUSS and COMMENT)

[David Noveck <[e-mail]>]

The following is the response to your DISCUSS.   The response to your
additional COMMENTs
wIll be sent soon.

> That said, this document (along with the pieces of 7530 and 7931 that I
> read along the way) still leave me uncertain about how some things are
> supposed to work.

That's unfortunate.  It shouldn't happen.

> (If it's clarified in parts of those documents that I
> didn't read, I'll happily clear and apologize for the disruption, of
> course.)

There is no disruption.   This discussion seems to me fully in line with
the way that the process needs to work.

I believe there are some relevant clarifications that derive from existing
documents, but it does not appear to me, that you missed anything.   The
relevant text does not, on its own, provide sufficient clarification, so it
appears that we will need to clarify things so that future readers do not
run into the difficulties that you saw.

> To start with, I'm still lacking a clear high-level picture of why a
client
> needs to care about trunking detection vs. just treating all listed
> addresses as replicas.  There are some parts in the body where we talk
> about, e.g., lock state and similar maintenance, but I don't have a clear
> picture of what the risks and benefits of (not) tracking trunking are, and
> this would be a fine opportunity to add some text.

The issues that arise when you are not aware of the trunking relationship
between two paths do have to with management of locking state.   RFC7530
(following on from similar text in RFC3530) cited the confusions regarding
locking that would arise if a client thought it was talking to two
different servers (i.e. replicas) when it was in fact talking to a single
system over two different paths.   Basically, a client who is unaware of
this distinction, might work successfully  sometimes, but in most cases you
have to know about trunking relationships that exist and it would be
extremely hard to write locking code that worked without being aware of
trunking relationships.

In fact, this is why:

   - RFC5661 added a means of trunking detection to NFSv4.1
   - RFCs7530 and 3530 advised NFSv4.0 clients to actively prevent the
   occurrence of trunking by adopting what is described in RFC7931 as the
   "non-uniform client string model".
   - RFC7931 provided a means of trunking detection usable by NFSv4.0

-04 will need to clearly reference the importantance of state/locking
considerations in making it necessary that the client and server agree as
to trunking relationships.  I anticipate we will do this by replacing the
third paragraph of the introduction by the following two paragraphs;

As part of addressing this need, [RFC7931] introduces trunking into
NFS version 4.0 along with a trunking detection mechanism.  A trunking
detection mechanism enables a client to determine whether two distinct
network addresses are connected to the same NFS version 4.0 server
instance.  This knowledge is necessary since, without it, a client
unaware of a trnking relation ship between paths it is using
simultaneous is likely to become confused in ways described in
[RF7530].

NFSv4.1 was defined with an integral means of trunking detection
described in [RFC5661], while NFSv4.0 initially did not have one, with
it being added by [RFC7931].  Nevertheless, the use of the concept of
server-trunkability is the same in both protocol versions.


> Specifically, in
> Section 5.2.1, we just say  that "[a] client may use file system location
> elements simultaneously to provide higher-performance access to the target
> file system"; most of the focus of this document makes me think that this
> statement was intended to apply only to trunking,

That was the intention and given that this is a trunking-motivated
document, it seemed reasonable.  However, as you point out, there are cases
in which those those words could reasonably be read as applying to
replication.

> but I also think there
> are supposed to be replication-only scenarios that provide performance
> gains.  I'm not sure if we need to clarify the distinction in that
location
> as well as the high-level overview.

There are such scenarios but they are not as important because:

   - They only apply to a subset of workloads, basically read-only file
   systems, and very low-change read-write filesystems where the application
   takes special care to propagate updates to all replicas.
   - The client (or the application) needs to be aware of the fact that
   spcial state mangement actions need to be taken.  Foe example, if you are
   to stripe your read from a given file between/amon a set of replicas, there
   needs to be.

It's also relevant to consider the original purpose of the replication
feature in NFSv4.0.   The intention was to provide alternative locations
that could be used if the primary location became unavailable.   This has
not prevented its use to provide parallel acces to multiple replicas to
increase performance, in a limited set of contexts, primarily to provide
read-only access, but it is important in deciding how to describe this
option in -04.

I believe we can address this issue in -04 by replacing the current fourth
paragraph of section 5.2.1 by the following two paragraphs:

A client may use file system location elements simultaneously to
provide higher-performance access to the target file system.  This is
most simply be done using trunling although the use of multiple
replicas simultaneously is possible. To enable this simultaneous
acces, the client utilizes trunking detection and/or discovery,
further described in Section 5.2.2 of the current document, to
determine a set of network paths that are server-trunkable with the
one currently being used to access the file system.   Once this
determination is made, requests may be routed across multiple paths,
using the existing state management mechanism

Multiple replicas may also be used simultaneously, typicalls in
handling read-only file systems.   In this case, each replica has its
own state management which the client needs to be aware of, doin
multiple file opens to enable a file to be read simultaneoudly from
multiple replicas.

> It's also unclear to me what parts of migration flows are under the
control
> of the client vs. the server.  It's clear that the server has to initiate
> migration via NFS4ERR_MOVED,

The server has to precipitate the client's response to the migration event
by informing it of the unusability of the (previously) current instance by
returning NFS4ERR_MOVED.  However, in almost all cases, this is not the
start of the server's work.

The server must, before informing the client of the shift, make sure that
when the client acts on it, the successor server (or servers) are prepared
to respond appropriately:

   - The destination server needs to have a fully up-to-date copy of the
   source file system.  Where the file system is read-write, this means that
   all update made to the sotrce file system must be propagated to the
   destination(s).
   - When transparent state migration is implemented, locking data needs to
   be similarly propagated.

> but my current understanding is just that this
> prompts the client to look at fs_locations, and the client has control
over
> which alternate location to move to.

When multiple locations are provided, the client has this choice.

> But there's also a lot of discussion
> in all three documents about the servers migrating state along with
> migration, so it seems like the server should be controlling where the
> client goes.  Is this just supposed to be by limiting the fs_locations
data
> to the specific migration target chosen by the server?

As a practical matter there are difficulties propagating locking state and
other updates to multiple desinations so that I expect most implementations
to designate a single successor, at least for a while.  However, the
protocol does allow multiple choices to be provided.  If the server does
so, then it would have to propagate information to multiple targets and be
prepared for any valid choice the client might make.

> (If so, this would
> probably have potential for poor interaction with the implicit filesystem
> discovery described in Section 5.3.)

I'm not sure of what problems you foresee.  Perhaps I need to better understand
your interpretation of section 5.3.

> On the other hand, Section 5.2.6
> talks about the server putting entries "that represent addresses usable
> with the current server or a migration target before those associated with
> replicas", which seems to imply that there is some other way to know what
> the migration target is.

Not sure what you mean by "some other way".

I think the server has to know what addresses are usable with current
server because it is the one that provides the appropriate service.  With
regard to migration targets, as we have seen, the sending server has to
know this because he is typically involved in preparing (through
information progation) the destination server to provide the needed
service.

> Section 5.2.6 also tells the client to rely on that ordering:
>
>                                   To keep this process as short as
>   possible, Servers are REQUIRED to place file system location entries
>   that represent addresses usable with the current server or a
>   migration target before those associated with replicas.  A client can
>   then cease scanning for trunkable file system location entries once
>   it encounters a file system location element whose fs_name differs
>
> but I don't think a client actually can do so, since the client has no way
> to know that the server implements this document as opposed to stock
> 7530+7931 (at least, no way that I saw).

The basic assumption behind this paragraph as written is that no previous
document has provided guidance as to the use of the location attribute to
provide information relevant to trunking discovery.   However, in
considering interactions with previous servers as you have done (nice
catch!) it appears that it would be possible for a server to present two
trunkable aaddresses even without any awareness of the fact that they are
trunkable, so that a client following the guidance above might deny himself
knowledge of some piece of information related to trunking discovery.

In light of this, it appears that -04 will have to rewrite the material in
the last paragraph of section 5.2.6 to be something like the following:

Because a file system location attribute may include entries relating
to the current server, the migration destination, and possible
replicas to use, scanning for available network addresses thst might
be trunkable with addresses already seen could potentially be a long
process.  In order keep this process as short as possible, Servers
that provide infornation about trunkable network paths when the exist
are REQUIRED to place file system location entries that represent
addresses usable with the current server or a migration target before
those associated with replicas.

This ordering allows a client to cease scanning for trunkable file
system location entries once it encounters a file system location
element whose fs_name differs from the current fs_name, or whose
address is not server-trunkable with the one it is currently using.
While the possibility exists, that a client might prematurely cease
scanning for trunkable addresses when receiving a location attribute
from a an older server not following the order constaint above, the
harm is expected to be limited since such servers would not be
expected to present infomation about trunkble server access paths.


> Finally, removing the last paragraph of Section 8.5 of RFC 7530 could have
> negative operational impact if updated clients interact with non-updated
> servers/environments that are misconfigured in the described fashion.
It's
> probably worth stating in the top-level Section 5 that such misconfigured
> servers are believed to no longer exist (if that's in fact true, of
> course; if not, we'd need to reconsider the change).

It turns out this deletion was inadvertant.  The paragraph will be added
back in -04.

On Wed, Jan 9, 2019 at 2:17 PM Benjamin Kaduk <[e-mail]> wrote:

> Benjamin Kaduk has entered the following ballot position for
> draft-ietf-nfsv4-mv0-trunking-update-03: Discuss
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-nfsv4-mv0-trunking-update/
>
>
>
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
>
> First off, thanks for the work on this document; it's important to get this
> behavior clarified and functional even for NFSv4.0.
>
> That said, this document (along with the pieces of 7530 and 7931 that I
> read along the way) still leave me uncertain about how some things are
> supposed to work.  (If it's clarified in parts of those documents that I
> didn't read, I'll happily clear and apologize for the disruption, of
> course.)
>
> To start with, I'm still lacking a clear high-level picture of why a client
> needs to care about trunking detection vs. just treating all listed
> addresses as replicas.  There are some parts in the body where we talk
> about, e.g., lock state and similar maintenance, but I don't have a clear
> picture of what the risks and benefits of (not) tracking trunking are, and
> this would be a fine opportunity to add some text.  Specifically, in
> Section 5.2.1, we just say that "[a] client may use file system location
> elements simultaneously to provide higher-performance access to the target
> file system"; most of the focus of this document makes me think that this
> statement was intended to apply only to trunking, but I also think there
> are supposed to be replication-only scenarios that provide performance
> gains.  I'm not sure if we need to clarify the distinction in that location
> as well as the high-level overview.
>
> It's also unclear to me what parts of migration flows are under the control
> of the client vs. the server.  It's clear that the server has to initiate
> migration via NFS4ERR_MOVED, but my current understanding is just that this
> prompts the client to look at fs_locations, and the client has control over
> which alternate location to move to.  But there's also a lot of discussion
> in all three documents about the servers migrating state along with
> migration, so it seems like the server should be controlling where the
> client goes.  Is this just supposed to be by limiting the fs_locations data
> to the specific migration target chosen by the server?  (If so, this would
> probably have potential for poor interaction with the implicit filesystem
> discovery described in Section 5.3.)  On the other hand, Section 5.2.6
> talks about the server putting entries "that represent addresses usable
> with the current server or a migration target before those associated with
> replicas", which seems to imply that there is some other way to know what
> the migration target is.
>
> Section 5.2.6 also tells the client to rely on that ordering:
>
>                                    To keep this process as short as
>    possible, Servers are REQUIRED to place file system location entries
>    that represent addresses usable with the current server or a
>    migration target before those associated with replicas.  A client can
>    then cease scanning for trunkable file system location entries once
>    it encounters a file system location element whose fs_name differs
>
> but I don't think a client actually can do so, since the client has no way
> to know that the server implements this document as opposed to stock
> 7530+7931 (at least, no way that I saw).
>
> Finally, removing the last paragraph of Section 8.5 of RFC 7530 could have
> negative operational impact if updated clients interact with non-updated
> servers/environments that are misconfigured in the described fashion.  It's
> probably worth stating in the top-level Section 5 that such misconfigured
> servers are believed to no longer exist (if that's in fact true, of
> course; if not, we'd need to reconsider the change).
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> Section 1
>
>    As part of addressing this need, [RFC7931] introduces trunking into
>    NFS version 4.0 along with a trunking detection mechanism.  This
>    enables a client to determine whether two distinct network addresses
>    are connected to the same NFS version 4.0 server instance.
>    Nevertheless, the use of the concept of server-trunkability is the
>    same in both protocol versions.
>
> Er, what are the two protocol versions in question?  (I assume 4.0 and 4.1,
> but you don't say 4.1 anywhere.)
>
>    o  To provide NFS version 4.0 with a means of trunking discovery,
>       compatible with the means of trunking detection introduced by
>       [RFC7931].
>
> We haven't yet mentioned that the distinction between "detection" and
> "discovery" is important, so it's probably worth a forward reference to the
> text below.
>
> Section 5.1
>
>    The fs_locations attribute (described as "RECOMMENDED" in [RFC7530])
>
> If you're going to describe this section as "replacing Section 8.1 of
> [RFC7530]", then it needs to stand on its own without reference to the
> current Section 8.1 of RFC 7530.  That is, if the "RECOMMENDED" nature is
> to remain, then it should be described as such de novo in this text.
>
>       Clients use the existing means for NFSv4.0 trunking detection,
>       defined in [RFC7931], to confirm that such addresses are connected
>       to the same server.  The client can ignore addresses found not to
>       be so connected.
>
> nit: I would suggest phrasing this as "use the NFSv4.0 trunking detection
> mechanism [RFC7931] to confirm [...]", as temporal refernces like
> "existing" may not age well.
> not-nit: "ignore" is pretty strong; does this imply that a client is free
> to ignore things like migration, replication, and referrals?
>
>       location entries.  If a file system location entry specifies a
>       network address, there is only a single corresponding location
>       element.  When a file system location entry contains a host name,
>       the client resolves the hostname, producing one file system
>       location element for each of the resulting network addresses.
>       Issues regarding the trustworthiness of hostname resolutions are
>       further discussed in Section 7.
>
> nit(?) this is confusing if we read "Section 7" as being "Section 7 of RFC
> 7530", which is a tempting reading since this text is supposed to replace
> text in that document.  Perhaps "Section 7 of [[this document]]" would make
> more sense (but I also forget the RFC Editor's policy on such
> self-references).
>
> Section 5.2.1
>
>                                                                  The
>    client utilizes trunking detection and/or discovery, further
>    described in Section 5.2.2 of the current document, to determine a
>
> nit(?) perhaps s/the current document/[[this document]]/ as above (for
> update by the RFC Editor).  I'll stop commenting this construction, though
> of course if such changes are made they should be done globally.
>
> Section 5.2.3
>
>    Because of the need to support multiple connections, clients face the
>
> What need?  Where is this need articulated?
>
>                     As a result, clients supporting multiple connection
>    types need to attempt to establish a connection on various connection
>    types allowing it to determine which connection types are supported.
>
> nit: maybe describe this as a "trial and error" approach to connection type
> support determination?
>
>    To avoid waiting when there is at least one viable network path
>    available, simultaneous attempts to establish multiple connection
>    types are possible.  Once a viable connection is established, the
>    client discards less-preferred connections.
>
> It's probably worth referencing the "happy eyeballs" technique used
> elsewhere (e.g., RFC 8305) as being analogous.
>
> Section 5.2.5
>
>    Such migration can help provide load balancing or general resource
>    reallocation.  [...]
>
> side note: is this load balancing generally going to be just of a "move a
> filesystem or ten to a different server when load gets too high" or are
> people also doing "send different clients to different replicas for the
> same filesystem" live load-balancing?
>
> Section 5.2.6
>
>    When the set of network addresses designated by a file system
>    location attribute changes, NFS4ERR_MOVED might or might not result.
>    In some of the cases in which NFS4ERR_MOVED is returned migration has
>    occurred, while in others there is a shift in the network addresses
>    used to access a particular file system with no migration.
>
> I got pretty confused when I first read this, thinking there was some
> implication that a server could introduce a fleeting NFS4ERR_MOVED as a
> notification that addresses changed, even if the server could otherwise
> continue handling the client's requests.  Perhaps:
>
> % When the set of network addresses on a server change in a way that would
> % affect a file system location attribute, there are several possible
> % outcomes for clients currently accessing that file system.  NFS4ERR_MOVED
> % is returned only when the server cannot satisfy a request from the
> client,
> % whether because the file system has been migrated to a different server,
> is
> % only accessible at a different trunked address on the same server, or
> some
> % other reason.
>
> Similarly, we may want to clarify that (e.g.) case (1) is not going to
> result in an NFS4ERR_MOVED.
>
>    2.  When the list of network addresses is a subset of that previously
>        in effect, immediate action is not needed if an address missing
>        in the replacement list is not currently in use by the client.
>        The client should avoid using that address in the future, whether
>        the address is for a replica or an additional path to the server
>        being used.
>
> "avoid using that address in the future" needs to be scoped to this
> filesystem; it's not going to work if clients treat it as a global
> blacklisting.
>
>    Although significant harm cannot arise from this misapprehension, it
>    can give rise to disconcerting situations.  For example, if a lock
>    has been revoked during the address shift, it will appear to the
>    client as if the lock has been lost during migration, normally
>    calling for it to be recoverable via an fs-specific grace period
>    associated with the migration event.
>
> I think this example needs to be clarified more or rewritten to describe
> what behavior fo which participant that normally happens does not happen
> (specifically, the "normally ..." clause).
>
>    from the current fs_name, or whose address is not server-trunkable
>    with the one it is currently using.
>
> nit: does it make more sense to put the address clause first, since fs_name
> is only valid within the scope of a given address/server?
>
> Section 5.3
>
>    As mentioned above, a single file system location entry may have a
>    server address target in the form of a DNS host name that resolves to
>    multiple network addresses, while multiple file system location
>    entries may have their own server address targets that reference the
>    same server.
>
> nit: I'm not sure that "while" is the right word here.  Perhaps "and
> conversely"?
>
>    When server-trunkable addresses for a server exist, the client may
>    assume that for each file system in the namespace of a given server
>    network address, there exist file systems at corresponding namespace
>    locations for each of the other server network addresses.  It may do
>
> Pretty sure you need to say "trunkable" here, too.
>
>    this even in the absence of explicit listing in fs_locations.  Such
>
> I may be confused, but we're talking about different file systems within a
> single server's single-server namespace, right?  So there is not even a way
> for them to be listed in the fs_locations for queries on FHs in the current
> filesystem (unless the server exports the same filesystem under different
> paths in its namespace for some reason).  So, we should probably be saying
> more about how these are fs_locations results returned for queries against
> different filesystems hosted on the same server...
>
>    corresponding file system locations can be used as alternative
>    locations, just as those explicitly specified via the fs_locations
>    attribute.
>
> ... (and possibly some related tweaks in this part too).
>
> Section 7
>
> We probably need to reiterate the privacy considerations inherent in the
> UCS approach, mentioned at the end of Section 5.6 of RFC 7931.
>
>       o  When DNS is used to convert NFS server host names to network
>          addresses and DNSSEC [RFC4033] is not available, the validity
>          of the network addresses returned cannot be relied upon.
>          However, when the client uses RPCSEC_GSS [RFC7861] to access
>          NFS servers, it is possible for mutual authentication to detect
>          invalid server addresses.  Other forms of transport layer
>
> nit: It seems to only sort-of be the case that the mutual authentication
> detects invalid addresses.  I tend to think of the property involved as
> ensuring that I am talking to who I think I am, which encompasses both the
> intended network address and the stuff on the other end.  On the other
> hand, one could imagine some bizzare deployments that share kerberos keys
> across servers where GSS could succeed (if the acceptor didn't have strict
> host name checking in place) but the address would still be unintended.
> If I had to rephrase this (unclear that it's really necessary), I might go
> with something like "to increase confidence in the correctness of server
> addresses", but there are lots of valid things to say here and it's not a
> big deal.
>
>       o  Fetching file system location information SHOULD be performed
>          using RPCSEC_GSS with integrity protection, as previously
>
> I forget if we have to say "integrity protection or better" or if this
> phrasing also includes the confidentiality protection case.
>
>       When a file system location attribute is fetched upon connecting
>       with an NFSv4 server, it SHOULD, as stated above, be done using
>       RPCSEC_GSS with integrity protection.
>
> It looks like this is now three places where this normative requirement is
> stated (7530's security considerations, and earlier in this section).
> Usually we try to stick to just one, to avoid risk of conflicting
> interpretations, and restate requirements non-normatively when needed.
> (It's not even clear that this duplication is needed, though.)
>
>                                                                For
>       example, if a range of network addresses can be determined that
>       assure that the servers and clients using AUTH_SYS are subject to
>       appropriate constraints (such as physical network isolation and
>       the use of administrative controls within the operating systems),
>       then network adresses in this range can be used with others
>       discarded or restricted in their use of AUTH_SYS.
>
> I'd strongly suggest adding a comma or something here to avoid the
> misparsing of "used with others".
>
>       To summarize considerations regarding the use of RPCSEC_GSS in
>       fetching file system location information, consider the following
>       possibilities for requests to interrogate location information,
>       with interrogation approaches on the referring and destination
>       servers arrived at separately:
>
> I don't understand what this is trying to say, especially in light of the
> following bullet points being essentially recommendations for behavior (in
> one case, limited to a specific situation where disrecommended behavior is
> unavoidable).
>
> I do appreciate the good discussions about the provenance and reliability
> of location data -- it seems to be pretty complete, so thank you!
>
>
>
On Wed, Jan 9, 2019 at 2:17 PM Benjamin Kaduk <[e-mail]> wrote:

> Benjamin Kaduk has entered the following ballot position for
> draft-ietf-nfsv4-mv0-trunking-update-03: Discuss
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
>
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
>
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-nfsv4-mv0-trunking-update/
>
>
>
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
>
> First off, thanks for the work on this document; it's important to get this
> behavior clarified and functional even for NFSv4.0.
>
> That said, this document (along with the pieces of 7530 and 7931 that I
> read along the way) still leave me uncertain about how some things are
> supposed to work.  (If it's clarified in parts of those documents that I
> didn't read, I'll happily clear and apologize for the disruption, of
> course.)
>
> To start with, I'm still lacking a clear high-level picture of why a client
> needs to care about trunking detection vs. just treating all listed
> addresses as replicas.  There are some parts in the body where we talk
> about, e.g., lock state and similar maintenance, but I don't have a clear
> picture of what the risks and benefits of (not) tracking trunking are, and
> this would be a fine opportunity to add some text.  Specifically, in
> Section 5.2.1, we just say that "[a] client may use file system location
> elements simultaneously to provide higher-performance access to the target
> file system"; most of the focus of this document makes me think that this
> statement was intended to apply only to trunking, but I also think there
> are supposed to be replication-only scenarios that provide performance
> gains.  I'm not sure if we need to clarify the distinction in that location
> as well as the high-level overview.
>
> It's also unclear to me what parts of migration flows are under the control
> of the client vs. the server.  It's clear that the server has to initiate
> migration via NFS4ERR_MOVED, but my current understanding is just that this
> prompts the client to look at fs_locations, and the client has control over
> which alternate location to move to.  But there's also a lot of discussion
> in all three documents about the servers migrating state along with
> migration, so it seems like the server should be controlling where the
> client goes.  Is this just supposed to be by limiting the fs_locations data
> to the specific migration target chosen by the server?  (If so, this would
> probably have potential for poor interaction with the implicit filesystem
> discovery described in Section 5.3.)  On the other hand, Section 5.2.6
> talks about the server putting entries "that represent addresses usable
> with the current server or a migration target before those associated with
> replicas", which seems to imply that there is some other way to know what
> the migration target is.
>
> Section 5.2.6 also tells the client to rely on that ordering:
>
>                                    To keep this process as short as
>    possible, Servers are REQUIRED to place file system location entries
>    that represent addresses usable with the current server or a
>    migration target before those associated with replicas.  A client can
>    then cease scanning for trunkable file system location entries once
>    it encounters a file system location element whose fs_name differs
>
> but I don't think a client actually can do so, since the client has no way
> to know that the server implements this document as opposed to stock
> 7530+7931 (at least, no way that I saw).
>
> Finally, removing the last paragraph of Section 8.5 of RFC 7530 could have
> negative operational impact if updated clients interact with non-updated
> servers/environments that are misconfigured in the described fashion.  It's
> probably worth stating in the top-level Section 5 that such misconfigured
> servers are believed to no longer exist (if that's in fact true, of
> course; if not, we'd need to reconsider the change).
>
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> Section 1
>
>    As part of addressing this need, [RFC7931] introduces trunking into
>    NFS version 4.0 along with a trunking detection mechanism.  This
>    enables a client to determine whether two distinct network addresses
>    are connected to the same NFS version 4.0 server instance.
>    Nevertheless, the use of the concept of server-trunkability is the
>    same in both protocol versions.
>
> Er, what are the two protocol versions in question?  (I assume 4.0 and 4.1,
> but you don't say 4.1 anywhere.)
>
>    o  To provide NFS version 4.0 with a means of trunking discovery,
>       compatible with the means of trunking detection introduced by
>       [RFC7931].
>
> We haven't yet mentioned that the distinction between "detection" and
> "discovery" is important, so it's probably worth a forward reference to the
> text below.
>
> Section 5.1
>
>    The fs_locations attribute (described as "RECOMMENDED" in [RFC7530])
>
> If you're going to describe this section as "replacing Section 8.1 of
> [RFC7530]", then it needs to stand on its own without reference to the
> current Section 8.1 of RFC 7530.  That is, if the "RECOMMENDED" nature is
> to remain, then it should be described as such de novo in this text.
>
>       Clients use the existing means for NFSv4.0 trunking detection,
>       defined in [RFC7931], to confirm that such addresses are connected
>       to the same server.  The client can ignore addresses found not to
>       be so connected.
>
> nit: I would suggest phrasing this as "use the NFSv4.0 trunking detection
> mechanism [RFC7931] to confirm [...]", as temporal refernces like
> "existing" may not age well.
> not-nit: "ignore" is pretty strong; does this imply that a client is free
> to ignore things like migration, replication, and referrals?
>
>       location entries.  If a file system location entry specifies a
>       network address, there is only a single corresponding location
>       element.  When a file system location entry contains a host name,
>       the client resolves the hostname, producing one file system
>       location element for each of the resulting network addresses.
>       Issues regarding the trustworthiness of hostname resolutions are
>       further discussed in Section 7.
>
> nit(?) this is confusing if we read "Section 7" as being "Section 7 of RFC
> 7530", which is a tempting reading since this text is supposed to replace
> text in that document.  Perhaps "Section 7 of [[this document]]" would make
> more sense (but I also forget the RFC Editor's policy on such
> self-references).
>
> Section 5.2.1
>
>                                                                  The
>    client utilizes trunking detection and/or discovery, further
>    described in Section 5.2.2 of the current document, to determine a
>
> nit(?) perhaps s/the current document/[[this document]]/ as above (for
> update by the RFC Editor).  I'll stop commenting this construction, though
> of course if such changes are made they should be done globally.
>
> Section 5.2.3
>
>    Because of the need to support multiple connections, clients face the
>
> What need?  Where is this need articulated?
>
>                     As a result, clients supporting multiple connection
>    types need to attempt to establish a connection on various connection
>    types allowing it to determine which connection types are supported.
>
> nit: maybe describe this as a "trial and error" approach to connection type
> support determination?
>
>    To avoid waiting when there is at least one viable network path
>    available, simultaneous attempts to establish multiple connection
>    types are possible.  Once a viable connection is established, the
>    client discards less-preferred connections.
>
> It's probably worth referencing the "happy eyeballs" technique used
> elsewhere (e.g., RFC 8305) as being analogous.
>
> Section 5.2.5
>
>    Such migration can help provide load balancing or general resource
>    reallocation.  [...]
>
> side note: is this load balancing generally going to be just of a "move a
> filesystem or ten to a different server when load gets too high" or are
> people also doing "send different clients to different replicas for the
> same filesystem" live load-balancing?
>
> Section 5.2.6
>
>    When the set of network addresses designated by a file system
>    location attribute changes, NFS4ERR_MOVED might or might not result.
>    In some of the cases in which NFS4ERR_MOVED is returned migration has
>    occurred, while in others there is a shift in the network addresses
>    used to access a particular file system with no migration.
>
> I got pretty confused when I first read this, thinking there was some
> implication that a server could introduce a fleeting NFS4ERR_MOVED as a
> notification that addresses changed, even if the server could otherwise
> continue handling the client's requests.  Perhaps:
>
> % When the set of network addresses on a server change in a way that would
> % affect a file system location attribute, there are several possible
> % outcomes for clients currently accessing that file system.  NFS4ERR_MOVED
> % is returned only when the server cannot satisfy a request from the
> client,
> % whether because the file system has been migrated to a different server,
> is
> % only accessible at a different trunked address on the same server, or
> some
> % other reason.
>
> Similarly, we may want to clarify that (e.g.) case (1) is not going to
> result in an NFS4ERR_MOVED.
>
>    2.  When the list of network addresses is a subset of that previously
>        in effect, immediate action is not needed if an address missing
>        in the replacement list is not currently in use by the client.
>        The client should avoid using that address in the future, whether
>        the address is for a replica or an additional path to the server
>        being used.
>
> "avoid using that address in the future" needs to be scoped to this
> filesystem; it's not going to work if clients treat it as a global
> blacklisting.
>
>    Although significant harm cannot arise from this misapprehension, it
>    can give rise to disconcerting situations.  For example, if a lock
>    has been revoked during the address shift, it will appear to the
>    client as if the lock has been lost during migration, normally
>    calling for it to be recoverable via an fs-specific grace period
>    associated with the migration event.
>
> I think this example needs to be clarified more or rewritten to describe
> what behavior fo which participant that normally happens does not happen
> (specifically, the "normally ..." clause).
>
>    from the current fs_name, or whose address is not server-trunkable
>    with the one it is currently using.
>
> nit: does it make more sense to put the address clause first, since fs_name
> is only valid within the scope of a given address/server?
>
> Section 5.3
>
>    As mentioned above, a single file system location entry may have a
>    server address target in the form of a DNS host name that resolves to
>    multiple network addresses, while multiple file system location
>    entries may have their own server address targets that reference the
>    same server.
>
> nit: I'm not sure that "while" is the right word here.  Perhaps "and
> conversely"?
>
>    When server-trunkable addresses for a server exist, the client may
>    assume that for each file system in the namespace of a given server
>    network address, there exist file systems at corresponding namespace
>    locations for each of the other server network addresses.  It may do
>
> Pretty sure you need to say "trunkable" here, too.
>
>    this even in the absence of explicit listing in fs_locations.  Such
>
> I may be confused, but we're talking about different file systems within a
> single server's single-server namespace, right?  So there is not even a way
> for them to be listed in the fs_locations for queries on FHs in the current
> filesystem (unless the server exports the same filesystem under different
> paths in its namespace for some reason).  So, we should probably be saying
> more about how these are fs_locations results returned for queries against
> different filesystems hosted on the same server...
>
>    corresponding file system locations can be used as alternative
>    locations, just as those explicitly specified via the fs_locations
>    attribute.
>
> ... (and possibly some related tweaks in this part too).
>
> Section 7
>
> We probably need to reiterate the privacy considerations inherent in the
> UCS approach, mentioned at the end of Section 5.6 of RFC 7931.
>
>       o  When DNS is used to convert NFS server host names to network
>          addresses and DNSSEC [RFC4033] is not available, the validity
>          of the network addresses returned cannot be relied upon.
>          However, when the client uses RPCSEC_GSS [RFC7861] to access
>          NFS servers, it is possible for mutual authentication to detect
>          invalid server addresses.  Other forms of transport layer
>
> nit: It seems to only sort-of be the case that the mutual authentication
> detects invalid addresses.  I tend to think of the property involved as
> ensuring that I am talking to who I think I am, which encompasses both the
> intended network address and the stuff on the other end.  On the other
> hand, one could imagine some bizzare deployments that share kerberos keys
> across servers where GSS could succeed (if the acceptor didn't have strict
> host name checking in place) but the address would still be unintended.
> If I had to rephrase this (unclear that it's really necessary), I might go
> with something like "to increase confidence in the correctness of server
> addresses", but there are lots of valid things to say here and it's not a
> big deal.
>
>       o  Fetching file system location information SHOULD be performed
>          using RPCSEC_GSS with integrity protection, as previously
>
> I forget if we have to say "integrity protection or better" or if this
> phrasing also includes the confidentiality protection case.
>
>       When a file system location attribute is fetched upon connecting
>       with an NFSv4 server, it SHOULD, as stated above, be done using
>       RPCSEC_GSS with integrity protection.
>
> It looks like this is now three places where this normative requirement is
> stated (7530's security considerations, and earlier in this section).
> Usually we try to stick to just one, to avoid risk of conflicting
> interpretations, and restate requirements non-normatively when needed.
> (It's not even clear that this duplication is needed, though.)
>
>                                                                For
>       example, if a range of network addresses can be determined that
>       assure that the servers and clients using AUTH_SYS are subject to
>       appropriate constraints (such as physical network isolation and
>       the use of administrative controls within the operating systems),
>       then network adresses in this range can be used with others
>       discarded or restricted in their use of AUTH_SYS.
>
> I'd strongly suggest adding a comma or something here to avoid the
> misparsing of "used with others".
>
>       To summarize considerations regarding the use of RPCSEC_GSS in
>       fetching file system location information, consider the following
>       possibilities for requests to interrogate location information,
>       with interrogation approaches on the referring and destination
>       servers arrived at separately:
>
> I don't understand what this is trying to say, especially in light of the
> following bullet points being essentially recommendations for behavior (in
> one case, limited to a specific situation where disrecommended behavior is
> unavoidable).
>
> I do appreciate the good discussions about the provenance and reliability
> of location data -- it seems to be pretty complete, so thank you!
>
>
>