Re: [nfsv4] Benjamin Kaduk's Discuss on draft-ietf-nfsv4-mv0-trunking-update-03: (with DISCUSS and COMMENT)

[Benjamin Kaduk <[e-mail]>]

On Thu, Jan 24, 2019 at 04:22:28PM -0500, David Noveck wrote:
> Following are the responses to your non-DISCUSS comments.
> 
> > Section 1
> >
> >   As part of addressing this need, [RFC7931] introduces trunking into
> >   NFS version 4.0 along with a trunking detection mechanism.  This
> >   enables a client to determine whether two distinct network addresses
> >   are connected to the same NFS version 4.0 server instance.
> >   Nevertheless, the use of the concept of server-trunkability is the
> >   same in both protocol versions.
> 
> > Er, what are the two protocol versions in question?  (I assume 4.0 and
> 4.1,
> > but you don't say 4.1 anywhere.)
> 
> 
> 
> This was addressed by the introductory paragraphs about locking prompted by
> your DISCUSS.
> 
> The paragraphs were:
> 
> 
> 
> As part of addressing this need, [RFC7931] introduces trunking into NFS
> version 4.0 along with a trunking detection mechanism.  A trunking
> detection mechanism enables a client to determine whether two distinct
> network addresses are connected to the same NFS version 4.0 server
> instance.  This knowledge is necessary since, without it, a client unaware
> of a trunking relationship between paths it is using simultaneous is likely
> to become confused in ways described in [RF7530].
> 
> 
> 
> NFSv4.1 was defined with an integral means of trunking detection described
> in [RFC5661], while NFSv4.0 initially did not have one, with it being added
> by [RFC7931].  Nevertheless, the use of the concept of server-trunkability
> is the same in both protocol versions
> 
> >   o  To provide NFS version 4.0 with a means of trunking discovery,
> >      compatible with the means of trunking detection introduced by
> >      [RFC7931].
> >
> > We haven't yet mentioned that the distinction between "detection" and
> > "discovery" is important, so it's probably worth a forward reference to the
> > text below.
> 
> 
> 
> I think we can revise this bullet to read as follows in -04:
> 
> 
> 
>    - To provide NFS version 4.0 with a means of finding addresses trunkable
>    with a given address. i.e., trunking discovery, compatible with the means
>    of trunking detection introduced by [RFC7931].  For an explanation of
>    trunking detection and trunking discovery see Section 3.

Thanks.

> 
> 
> >Section 5.1
> 
> >   The fs_locations attribute (described as "RECOMMENDED" in [RFC7530])
> >
> > If you're going to describe this section as "replacing Section 8.1 of
> > [RFC7530]", then it needs to stand on its own without reference to the
> > current Section 8.1 of RFC 7530.  That is, if the "RECOMMENDED" nature is
> > to remain, then it should be described as such de novo in this text.
> 
> The problem we have to deal with is the fact that the term “RECOMMENDED” is
> not in accord with RFC2119.  .  RFC7530 deals with this by making an
> exception in its Section 1.1.   To avoid us having to do the same thing, we

Ah, I had completely missed that disclaimer in RFC 7530, whoops.

> could simply delete the parenthetical material and render the sentence as
> follows:
> 
> 
> 
> The fs_locations attribute allows specification of file system locations
> where the data corresponding to a given file system may be accessed.

We might also be able to point to Section 5.2 of 7530 for the RECOMMENDED
definition.  But since Section 2.2.7 also refers to it as "the fs_locations
RECOMMENDED attribute" as well as the main treatement in the (now-replaced)
Section 8.1, it seems we can probably get away with the minimalistic
version.

> 
> >      Clients use the existing means for NFSv4.0 trunking detection,
> >      defined in [RFC7931], to confirm that such addresses are connected
> >      to the same server.  The client can ignore addresses found not to
> >      be so connected.
> 
> > nit: I would suggest phrasing this as "use the NFSv4.0 trunking detection
> > mechanism [RFC7931] to confirm [...]", as temporal refernces like
> > "existing" may not age well
> 
> 
> 
> OK.  Will fix in -04.
> 
> .
> > not-nit: "ignore" is pretty strong; does this imply that a client is free
> > to ignore things like migration, replication, and referrals?
> 
> That needs to be rephrased.   The intention is that they can ignore the
> non-confirmed
> trunking relationship.
> 
> 
> 
> I think the following should appear in -04:
> 
> 
> 
> Clients use the means for NFSv4.0 trunking detection, defined in
> [RFC7931],  to confirm that such addresses are connected to the same
> server.  The client can ignore non-confirmed trunking relationships and
> treat the corresponding addresses as connected to different servers.

SGTM

> 
> 
> >      location entries.  If a file system location entry specifies a
> >      network address, there is only a single corresponding location
> >      element.  When a file system location entry contains a host name,
> >      the client resolves the hostname, producing one file system
> >      location element for each of the resulting network addresses.
> >      Issues regarding the trustworthiness of hostname resolutions are
> >     further discussed in Section 7.
> 
> >nit(?) this is confusing if we read "Section 7" as being "Section 7 of RFC
> > 7530", which is a tempting reading since this text is supposed to replace
> > text in that document.  Perhaps "Section 7 of [[this document]]" would make
> > more sense (but I also forget the RFC Editor's policy on such
> > self-references).
> 
> 
> 
> I don't think I ever knew that policy.
> 
> 
> 
> -04 will say "Section 7 of the current document" and if that is not OK,
> with the RFC editor, we'll find out about it during the RFC editing process.

I'd consider adding an RFC Editor note calling out the question, since this
"current document" construction appears in several places and we'd want to
be consistent about it.

> > Section 5.2.1
> 
> >                                                                 The
> >   client utilizes trunking detection and/or discovery, further
> >   described in Section 5.2.2 of the current document, to determine a
> 
> > nit(?) perhaps s/the current document/[[this document]]/ as above (for
> > update by the RFC Editor).  I'll stop commenting this construction, though
> > of course if such changes are made they should be done globally.
> 
> 
> 
> We'll kick the can down the road (gently) toward the RFC Editor.
> 
> >Section 5.2.3
> 
> >    Because of the need to support multiple connections, clients face the
> 
> What need?  Where is this need articulated?
> 
> 
> 
> Part of the problem here is the need to replace "connections'" by
> "connection types".

Ah, yes, that would help.

> 
> 
> I believe we should add a short introductory paragraph at the start of
> Section 5.2.3,
> 
> reading as follows in-04:
> 
> 
> 
> NFS Version 4 may be implemented using a number of different types of
> connections:
> 
>    - Stream connections may be used to provide RPC service as described in
>    [RFC5531].
>    - RDMA-capable connections may be used to provide RPC service using
>    RPC-over-RDMA, as described in [RFC8166].

As would this (though I think this part didn't bother me, since I lurk on
the nfsv4 list and remembered the different transports being used).

> > As a result, clients supporting multiple connection
> >    types need to attempt to establish a connection on various connection
> >    types allowing it to determine which connection types are supported.
> 
> > nit: maybe describe this as a "trial and error" approach to connection type
> > support determination?
> 
> 
> 
> We could say:
> 
> 
> 
> As a result, clients supporting multiple connection types need to attempt
> to establish a connection on various connection types allowing it to
> determine, via a trial-and-error approach,  which connection types are
> supported.

Sure.

> 
> 
> > To avoid waiting when there is at least one viable network path
>    available, simultaneous attempts to establish multiple connection
>    types are possible.  Once a viable connection is established, the
>    client discards less-preferred connections.
> 
> > It's probably worth referencing the "happy eyeballs" technique used
> > elsewhere (e.g., RFC 8305) as being analogous.
> 
> 
> 
> Possibly, but I’m not familiar enough with this RFC to reference it is this
> context.

Okay; it's not going to be a critical omission if it doesn't happen.

> > Section 5.2.5
> >
> >   Such migration can help provide load balancing or general resource
> >   reallocation.  [...]
> >
> > side note: is this load balancing generally going to be just of a "move a
> > filesystem or ten to a different server when load gets too high"
> 
> 
> 
> That’s one thing that can be done.
> 
> 
> 
> > or are
> > people also doing "send different clients to different replicas for the
> >same filesystem" live load-balancing?
> 
> I don’t know of a current implementation but believe this can be done.
> 
> 
> >Section 5.2.6
> >
> >   When the set of network addresses designated by a file system
> >   location attribute changes, NFS4ERR_MOVED might or might not result.
> >  occurred, while in others there is a shift in the network addresses
> >   used to access a particular file system with no migration.
> >
> > I got pretty confused when I first read this, thinking there was some
> > implication that a server could introduce a fleeting NFS4ERR_MOVED as a
> > notification that addresses changed, even if the server could otherwise
> > continue handling the client's requests.  Perhaps:
> 
> > When the set of network addresses on a server change in a way that would
> > affect a file system location attribute, there are several possible
> > outcomes for clients currently accessing that file system.  NFS4ERR_MOVED
> > is returned only when the server cannot satisfy a request from the client,
> > whether because the file system has been migrated to a different server, is
> > only accessible at a different trunked address on the same server, or some
> > other reason.
> 
> 
> 
> Your version is clearer.  We can pick this up in -04.
> 
> > Similarly, we may want to clarify that (e.g.) case (1) is not going to
> > result in an NFS4ERR_MOVED.
> 
> 
> 
> I think we can do this in the introductory paragraph by adding the
> following sentence at the end:
> 
> 
> 
> In the cases 1 and 2 below, NFS4ERR_REMOVED not returned.

Okay.

> >   2.  When the list of network addresses is a subset of that previously
> >      in effect, immediate action is not needed if an address missing
> >       in the replacement list is not currently in use by the client.
> >       The client should avoid using that address in the future, whether
> >       the address is for a replica or an additional path to the server
> >       being used.
> 
> > "avoid using that address in the future" needs to be scoped to this
> > filesystem; it's not going to work if clients treat it as a global
> > blacklisting.
> 
> 
> 
> Ok.  Will replace “avoid using that address in the future” by the following;
> 
> 
> 
> avoid using that address to access that filesystem in the future

Thanks.

> 
> >   Although significant harm cannot arise from this misapprehension, it
> >   can give rise to disconcerting situations.  For example, if a lock
> >   has been revoked during the address shift, it will appear to the
> >   client as if the lock has been lost during migration, normally
> >   calling for it to be recoverable via an fs-specific grace period
> >   associated with the migration event.
> 
> >I think this example needs to be clarified more or rewritten to describe
> > what behavior fo which participant that normally happens does not happen
> > (specifically, the "normally ..." clause).
> 
> 
> 
> We could drop the “normally ..” clause and in its place add the following
> sentence:
> 
> 
> 
> When such a lock is lost, it is responsibility of the destination server to
> provide for its recovery
> 
> through use of an fs-specific grace period.

Thanks.

> >   from the current fs_name, or whose address is not server-trunkable
> >   with the one it is currently using.
> 
> > nit: does it make more sense to put the address clause first, since fs_name
> > is only valid within the scope of a given address/server?
> 
> 
> 
> I don’t think so.

Okay.

> 
> > Section 5.3
> 
> >   As mentioned above, a single file system location entry may have a
> >   server address target in the form of a DNS host name that resolves to
> >   multiple network addresses, while multiple file system location
> >   entries may have their own server address targets that reference the
> >   same server.
> 
> > nit: I'm not sure that "while" is the right word here.  Perhaps "and
> > conversely"?
> 
> 
> 
> Don’t think “and conversely” is right.  Open to other suggestions.

Rereading, I agree that "conversely" is not right.
All I can come up with right now is to split with a semicolon: "addresses;
it is also possible for multiple file system location entries to have"

> >   When server-trunkable addresses for a server exist, the client may
> >   assume that for each file system in the namespace of a given server
> >   network address, there exist file systems at corresponding namespace
> >   locations for each of the other server network addresses.  It may do
> 
> > Pretty sure you need to say "trunkable" here, too.
> 
> 
> 
> Ok.  Will change “other server network addresses” to “other
> server-trunkable  network addresses”.
> 
> >   this even in the absence of explicit listing in fs_locations.  Such
> 
> > I may be confused, but we're talking about different file systems within a
> > single server's single-server namespace, right?
> 
> 
> 
> Yes, these file systems exist within the containing server single-server
> namespace.
> 
> 
> 
> > So there is not even a way
> > for them to be listed in the fs_locations for queries on FHs in the current
> > filesystem (unless the server exports the same filesystem under different
> > paths in its namespace for some reason).
> 
> 
> 
> But they could appear in fs-locations entries for other file systems.  The
> text in question is
> 
> basically saying that they don’t have to.

Okay.

> 
> 
> > So, we should probably be saying
> > more about how these are fs_locations results returned for queries against
> > different filesystems hosted on the same server...
> 
> 
> >   corresponding file system locations can be used as alternative
> >   locations, just as those explicitly specified via the fs_locations
> >   attribute.
> >
> > ... (and possibly some related tweaks in this part too).
> 
> 
> 
> Not sure what would be needed.  Open to concrete suggestions.

It's probably okay to leave as-is; no concrete suggestions are coming to
me.

> > Section 7
> 
> > We probably need to reiterate the privacy considerations inherent in the
> > UCS approach, mentioned at the end of Section 5.6 of RFC 7931.
> 
> 
> Given the length of this portion of Section 5.6 of RFC7931,
> reiterating this material as part of
> 
> the security considerations of this document could result in undue
> attention to something that,
> 
> while worthy of note, will be, for most implementations, not a major
> concern.  After all, Section 5.6
> 
> of RFC7931 closes by stating “How to balance these considerations
> depends on implementation
> 
> goals” and this material does not appear in the security
> considerations section of RFC7931.   I think
> 
> we could reference this material without duplicating it in this document.

So maybe something like "the privacy considerations relating to uniform vs.
non-uniform client strings, discussed in Section 5.6 of RFC 7931, are also
applicable to their usage for trunking detection in NFS v4.0"?

> 
> 
> >      o  When DNS is used to convert NFS server host names to network
> >         addresses and DNSSEC [RFC4033] is not available, the validity
> >         of the network addresses returned cannot be relied upon.
> >         However, when the client uses RPCSEC_GSS [RFC7861] to access
> >         NFS servers, it is possible for mutual authentication to detect
> >         invalid server addresses.  Other forms of transport layer
> 
> > nit: It seems to only sort-of be the case that the mutual authentication
> > detects invalid addresses.  I tend to think of the property involved as
> > ensuring that I am talking to who I think I am,
> 
> 
> 
> I do too.
> 
> 
> 
> > which encompasses both the
> > intended network address and the stuff on the other end.  On the other
> > hand, one could imagine some bizzare deployments that share kerberos keys
> > across servers where GSS could succeed (if the acceptor didn't have strict
> >host name checking in place) but the address would still be unintended.
> 
> 
> 
> It  seems to me that what you are describing as a “bizzare deployment” is
> in fact a broken one,
> 
> in that the putative authentication of the server would be useless.
> 
> 
> > If I had to rephrase this (unclear that it's really necessary), I might go
> > with something like "to increase confidence in the correctness of server
> > addresses", but there are lots of valid things to say here and it's not a
> > big deal.
> 
> 
> 
> I think we should avoid entering that swamp.   I might be imagining things,
> but I worry that
> 
> alligators might be present.

Okay.  "It's not a big deal."

> >      o  Fetching file system location information SHOULD be performed
> >         using RPCSEC_GSS with integrity protection, as previously
> 
> > I forget if we have to say "integrity protection or better" or if this
> > phrasing also includes the confidentiality protection case.
> 
> 
> 
> I don’t know either but I believe this formulation follows the current
> practice for NFSv4 documents.

that works for me

> >      When a file system location attribute is fetched upon connecting
> >      with an NFSv4 server, it SHOULD, as stated above, be done using
> >      RPCSEC_GSS with integrity protection.
> 
> > It looks like this is now three places where this normative requirement is
> > stated (7530's security considerations, and earlier in this section).
> > Usually we try to stick to just one, to avoid risk of conflicting
> > interpretations, and restate requirements non-normatively when needed.
> > (It's not even clear that this duplication is needed, though.)
> 
> 
> 
> I believe that restating this non-normatively would contribute to confusion
> as it would seem inconsistent
> 
> I don’t see any danger of conflicting interpretations given that there is a
> clear reference to the original requirement, which is treated as primary.

It's your call; this is a non-blocking comment.

> 
>    >                                                            For
>    >   example, if a range of network addresses can be determined that
>    >   assure that the servers and clients using AUTH_SYS are subject to
>    >   appropriate constraints (such as physical network isolation and
>    >   the use of administrative controls within the operating systems),
>    >   then network adresses in this range can be used with others
>    >   discarded or restricted in their use of AUTH_SYS.
> 
> >I'd strongly suggest adding a comma or something here to avoid the
> > misparsing of "used with others".
> 
> 
> 
> Not sure where you want a comma or what other somethings might be added.
> Please make a concrete suggestion.

I'm looking for "then network addresses in this range can be used, with
others discarded".

> 
> >      To summarize considerations regarding the use of RPCSEC_GSS in
> >      fetching file system location information, consider the following
> >      possibilities for requests to interrogate location information,
> >      with interrogation approaches on the referring and destination
> >      servers arrived at separately:
> 
> > I don't understand what this is trying to say, especially in light of the
> >following bullet points being essentially recommendations for behavior
> 
> 
> 
> These are, as you suppose, recommendations for behavior.  I’m not clear
> exactly how that fact
> 
> make these security-oriented recommendations difficult to understand

I was trying to contrast "possibilities" with "recommendations"; even just
replacing the one word with the other might help me out.

-Benjamin

> 
> 
> > (in
> > one case, limited to a specific situation where disrecommended behavior is
> > unavoidable).
> 
> 
> 
> This is a consequence of the fact that use of AUTH_SYS, although not a very
> good idea, is currently
> 
> allowed, and realistically will never be disallowed.  As a result, we can’t
> prevent its use and have to
> 
> give the best guidance we can.
> 
> 
> 
> 
>