Re: [nfsv4] Benjamin Kaduk's Discuss on draft-ietf-nfsv4-rfc5661sesqui-msns-03: (with DISCUSS and COMMENT)
Benjamin Kaduk <kaduk@mit.edu> Mon, 23 December 2019 02:14 UTC
Return-Path: <kaduk@mit.edu>
X-Original-To: nfsv4@ietfa.amsl.com
Delivered-To: nfsv4@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B362712006F; Sun, 22 Dec 2019 18:14:50 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.2
X-Spam-Level:
X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vl6rLDRXvR7q; Sun, 22 Dec 2019 18:14:46 -0800 (PST)
Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A3E40120025; Sun, 22 Dec 2019 18:14:44 -0800 (PST)
Received: from kduck.mit.edu ([24.16.140.251]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id xBN2EZTL026978 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 22 Dec 2019 21:14:37 -0500
Date: Sun, 22 Dec 2019 18:14:35 -0800
From: Benjamin Kaduk <kaduk@mit.edu>
To: David Noveck <davenoveck@gmail.com>
Cc: The IESG <iesg@ietf.org>, draft-ietf-nfsv4-rfc5661sesqui-msns@ietf.org, nfsv4-chairs@ietf.org, Magnus Westerlund <magnus.westerlund@ericsson.com>, NFSv4 <nfsv4@ietf.org>
Message-ID: <20191223021435.GW35479@kduck.mit.edu>
References: <157665795217.30033.16985899397047966102.idtracker@ietfa.amsl.com> <CADaq8jegizL79V4yJf8=itMVUYDuHf=-pZgZEh-yqdT30ZdJ5w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CADaq8jegizL79V4yJf8=itMVUYDuHf=-pZgZEh-yqdT30ZdJ5w@mail.gmail.com>
User-Agent: Mutt/1.12.1 (2019-06-15)
Archived-At: <https://mailarchive.ietf.org/arch/msg/nfsv4/xHhOaLl31qEP6LfbsIUXn79NLlk>
Subject: Re: [nfsv4] Benjamin Kaduk's Discuss on draft-ietf-nfsv4-rfc5661sesqui-msns-03: (with DISCUSS and COMMENT)
X-BeenThere: nfsv4@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NFSv4 Working Group <nfsv4.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/nfsv4/>
List-Post: <mailto:nfsv4@ietf.org>
List-Help: <mailto:nfsv4-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 23 Dec 2019 02:14:51 -0000
On Fri, Dec 20, 2019 at 09:46:22AM -0500, David Noveck wrote: > > ---------------------------------------------------------------------- > > DISCUSS: > > ---------------------------------------------------------------------- > > > > it's important to get > > these points clarified, and sooner rather than later. > > Agree. > > > I expect that the > > following few issues should be quickly resolvable. > > I don't doubt it. The DISCUSS part is quite small and I > will respond to it first. The other COMMENTs will be addressed > in a mail next week. > > > Section 11.10.1 includes a reference to "Section 11.7.2.1 of RFC5661", > > but this document is obsoleting that document. > > As a general matter, you cannot avoid referring (informatively) to the > the document you are obsoleting, if only to say, "This document obsoletes > RFC xxxx". Similarly, I think you can say "This document says A about XYZ > while > RFC xxxx (incorrectly) said B". > > > It seems internally > > inconsistent to both obsolete and depend on the same source -- if we rely > > on that content, it should be included in this document. > > The question is whether we are relying on the content or merely referring > to it > as a best-eventually-forgotten piece of history. After looking at this in > detail, I have > concluded that the reference to 11.7.2.1 is, as you put it, "relying" on > the material > in RFC5661 and that deleting it was a mistake. Nice catch. Thanks for reading the right meaning into my poorly-written remark -- I did mean to say that we seem to be "relying" on this material. > I think I slipped into this because the material in 11.7.2.1 is way outside > the current > approach to these matters. I thought it could be deleted but have. now > realized that > it can't be but needs to be redescribed in more contemporary terms. > > So I have devioded to recast this mateial in two small additional sections > dealing > trunking in place of replication and trnking of file sytem without explicit > file system > location entries. Each section refer (informatively) to RFC5661 to > indicate to the reader > that the underlying reality has not changed, even though the form of > description has. > > The anticipated new sections follow; > > 11.5.4.1. File System Trunking Presented as Replication > > In some situations, a file system location entry may indicate a file > system access path to be used as an alternate location, where > trunking, rather than replication is to be used. The situtions in nit: I think we need a comma after "replication" (or to put all of "rather than replication" in parentheses). nit: s/situtions/situations/ > which this is appropriate are limited to those in which both of the > following are true. > > o The two file system locations (i.e the one on which the location nit: comma after "i.e." > attribute is obtained and the one specified in the file system > location entry) designate the same locations within their > respective single-server namespaces. > > o The two server network address(i.e. the one being used to obtain ditto, and space before open paren. > the location attribute and the one specified in the file system > location entry) designate the same server (as indicated by the > same value of the so_major_id field of the eir_server_owner field > returned in response to EXCHANGE_ID). > > When these conditions hold, access to using both access paths is nit: s/to // (though the double "access" is still perhaps unfortunate) > genrally trunked, although, when the attribute fs_locations_info is > used, trunking msy be disallowed: nit: s/msy/may/ > o When the fs_locations_info attribute shows the two entries as not > having the same simultaneous-use class, trunking is inhibited and > the two access paths cannot be used totgether. > In this case the two paths can be used serially with no transition > activity required on the part of the client. In this case, any > transition between access paths is transparent, and the client in > transferring access from one to the other is acting as it would in > the event that communication is interrupted, with a new connection > and possibly a new session being established to continue access to > the same file system. I think I've lost the context for how "no transition activity" meshes with "new connection and possibly a new session" -- it it mostly just "no need for recovery" and "use the same client ID"? > o Note that for two such location entries, any information within > the fs_locations_info attribute that indicates the need for > special transition activity, i.e., the appearance of the two file > system location entries with different handle, fileid, write- > verifier, change, and readdir classes, indicates a serious > problem. The client, if it allows transition to the file system > instance at all, must not treat any transition as a transparent > one. The server SHOULD NOT indicate that these entries belong to > different handle, fileid, write-verifier, change, and readdir Pedagogically, we might want to be more explicit about "these entries", perhaps along the lines of "multiple entries for the same file system on the same server" (if I understand correctly). > classes, whether or not the two entries are shown belonging to the > same simultaneous-use class. > > This situation was recognized by [62], even though that document made > no explicit mention of trunking. > > o It treated the situation that we describe as trunking as one of > simultaneous use of two distinct file system instances, even > though, in the explanatory framework now used to describe the > situstion, the case is one in which a single file system is > accessed by two different trunked addresses. > > o It treated the situation in which the two paths are to be used > serially as a special sort of "transparent transition", while in > the descriptve framework now used to categorize transition > situations, this is a case of a "network endpoint transition" (see > Section 11.9. > > 11.6. Trunking without File System Location Information > > In situation in which a file system is accessed using two server- nit: missing article ("the situation"?) > trunkableaddresses (as indicated by the same value of the so_major_id nit: s/trunkableaddresses/trunkable addresses/ > field of the eir_server_owner field returned in response to > EXCHANGE_ID), trunked access is allowed even though there might not > be any location entries specfically indicating the use of trunking > for that file system. > > This situation was recognized by [62], even though that document made > no explicit mention of trunking and treated the situation as one of > simultaneous use of two distinct file system instances, even though, > in the explanatory framework now used to describe the situstion, the > case is one in which a single file system is accessed by two > different trunked addresses. > > In addition I anticipate rewriting the first paragraph of Section 11.10.1 to > read as follows: > > The fs_locations_info attribute (described in Section 11.17) may > indicate that two replicas may be used simultaneously, although some > situations in which such simultaneous access is permitted are more > appropritaely described as instances of trunking (see nit: s/appropritaely/appropriately/ > Section 11.5.4.1). Although situations in which multiple replicas > may be accessed simultaneously are somewhat similar to those in which > a single replica is accessed by multiple network addresses, there are > important differences, since locking state is not shared among > multiple replicas. Thanks; this looks really good (nits aside). > > > This is somewhat awkward since the limited nature of the update results > > in my not having the full context of the rest of the document; with that > > limitation in my understanding in mind, I'd like to confirm that we're > > comfortable with the use of "network address" in the context of > > trunking/migration, specifically the extent to which we do not discuss > > port numbers. > > Now that you point this out, I guess I shouldn't have been as comfortable as > I had been. > > > The relevant XDR types do allow for optional port numbers > > to be included, with a default to be used when not specified, > > The xdr types you refer to are not relevant here, since we have, in the > interest of allowing hostname as well as IP addresses,specfied these > IP adresses, when IP addresses are used in the form of ASCII text. > Nevertheless, given that port numbers can be given, optionally after a > colon, the essence of the issue is as you described. > > > but in > > this document we do have a new note that different ports may be used for > > different connection types to the same logical server, and also that > > different ports "is not the essence of the distinction between the two > > endpoints". I think there might be cases where the port is relevant for > > a distinction, but the main ones I can think of are of questionable > > relevance (essentially, roughly equivalent to multiple userspace NFS > > servers on a single host but in different trust/privilege domains) -- > > I'd like another opinion or several. > > I agree that your major case is of limited use, but that doesn't mean that > it should be ignored. > > What I'm proposing doing is adding the following as a new paragraph > associated with the second bullet in the second set of bullets in Section > 11.1.2 (i.e. after <vspace> in xml2rfc v2), to read as follows: > > The network addresses used in file system location entries > typically appear without port number indications and are used to > designate a server at one of the standard ports for NFS access, > i.e. 2049 or 20049 for use with RPC-over-RDMA. Port numbers may > be used in file system location entries to designate servers > (typically user-level ones) accesed using other port numbers. In > the case if network addresses indicating trunking relationships, nit: should this be s/if/of/? Though, I might go with "the case where network addresses indicate", myself. > use of explcit port number is inappropriate since trunking is a nit: s/explcit/explicit/ > relationsip btetween network addresses. See Section 11.5.2 for nit: s/btetween/between/ > details. > > I also anticipate revising the second bullet in section 11.5.2 to read as > foollows: > > The client may fetch the file system location attribute for the > file system. This will provide either the name of the server > (which can be turned into a set of network addresses using DNS), > or a set of server-trunkable location entries. Using the latter > alternative, the server can provide addresses it regards as > desirable to use to access the file system in question. Although > these enties can contain port numbers, these port numbers are not > used in determining trunking relationships. Once the cadidate > addresses have been determined and EXCHANGE_ID done to the proper > server, only the value of the so_major field returned by the > serrvers in question determines whether a trunking relationship nit: s/serrvers/servers/ > actually exists. Thanks; that seems a pretty elegant way to address the topic without causing churn throughout large swathes of text. Though now I do find myself wondering whether there's anything to keep a rogue (i.e., attacker-controlled) server from falsely claiming the so_major field that matches an honest server and tricking a client into thinking that the attacker's server is trunkable with the honest one. The GSS identity check would, or course, work if available, but AFAICT the non-GSS case remains about as bad as, well, the non-GSS case in general. So I don't feel a need to hold up this document while we figure out more to say than we already do in (e.g.) Appendix C. > > > In a similar "discuss discuss" vein, Section 11.10.8 describes a > > scenario that does not give much clarity, at a protocol level, into what > > degree of replication synchronization a client can expect from a given > > file system that advertises multiple replicas. I recognize that this is > > de facto just stating the deployed reality, but it's also hard to feel > > good about having this level of ambiguity in a propsed standard, > > It gets easier over time. Sigh! This would be the fourth Proposed > Standard with this issue :-( > > Still I think we can do a bit better by relyimg on the three special cases > listed at > the end of that section and adding something like the following after the > list: > > When none of these special situations apply, there is no basis, > within the protool, for the client making assumptions about the > contents of a replica file ststem and its relationship to previous > file sytem instances. This may mean that switching between file > system that are not read-only is not available, where either the Hmm, do we want to add a descriptor like "nominally identical" to "file systems" (note: plural) > client does not use or the server does not support the > fs_locations_info atribute. That helps some. We might consider a note in the first paragraph (near "The namespace will typically be constructed") about the details of that being configured in an out-of-band manner. To be clear, I'll drop the discuss point regardless; I'm happy with the exchange we've had here and the direction that your new paragraph is going. > > and the (unchanged) text in Section 11.5.5 seems to impose a stricter > > consistency requirement, at least on potential migration targets. (A > > bit more detail in the COMMENT section.) > > That is a different and has been since the multi-server > namespace features were first added to NFSv4 decades ago. > It is inherently easier to synchronize A and B when you know that A > is over befoe B begins. Okay. (I hope you can understand that trying to review just the diff left some of the other portions a little fuzzy to me.) > > Section 11.13.2 mentions that "[i]ssues connected with a client > > impersonating another by presenting another client's id string are > > discussed in Section 21", but I failed to find this discussion in > > Section 21. > > It is there but it is not that easy to find. This is referring to the > 4.1 state protection features. > > I suggest replacing the sentence in question by "Issues connected with a > client impersonating another > by presenting another client's client id string can be adressed using > NFSv4.1 state protection features, as described in Section 21." Ah, now I see the connection; it was indeed not easy to find before this change :) > > (The discuss-level issue is just the internal > > inconsistency; there's a decent argument that this is covered by > > Appendix C's "not written in accord with RFC3552". > > True. Note that this is within a list of "Security Issues that need to be > Addressed." > > > Though if the text > > was already written for draft-ietf-nfsv4-mv1-msns-update, not including > > it here seems a little silly.) > > There is no text but there has been some speculation that the client > authentication features of rpc-tls might be helpful here. Indeed, there's much to watch in that space. Thanks for the updates; I think we're in good shape for the Discuss part, and will look for the follow-up on the Comment as time permits. (Which is to say, please enjoy your holidays and don't feel rushed on my behalf.) -Ben > On Wed, Dec 18, 2019 at 3:32 AM Benjamin Kaduk via Datatracker < > noreply@ietf.org> wrote: > > > Benjamin Kaduk has entered the following ballot position for > > draft-ietf-nfsv4-rfc5661sesqui-msns-03: Discuss > > > > When responding, please keep the subject line intact and reply to all > > email addresses included in the To and CC lines. (Feel free to cut this > > introductory paragraph, however.) > > > > > > Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html > > for more information about IESG DISCUSS and COMMENT positions. > > > > > > The document, along with other ballot positions, can be found here: > > https://datatracker.ietf.org/doc/draft-ietf-nfsv4-rfc5661sesqui-msns/ > > > > > > > > ---------------------------------------------------------------------- > > DISCUSS: > > ---------------------------------------------------------------------- > > > > Thank you for this document (and its predecessor); it's important to get > > these points clarified, and sooner rather than later. I expect that the > > following few issues should be quickly resolvable. > > > > Section 11.10.1 includes a reference to "Section 11.7.2.1 of RFC5661", > > but this document is obsoleting that document. It seems internally > > inconsistent to both obsolete and depend on the same source -- if we rely > > on that content, it should be included in this document. > > > > This is somewhat awkward since the limited nature of the update results > > in my not having the full context of the rest of the document; with that > > limitation in my understanding in mind, I'd like to confirm that we're > > comfortable with the use of "network address" in the context of > > trunking/migration, specifically the extent to which we do not discuss > > port numbers. The relevant XDR types do allow for optional port numbers > > to be included, with a default to be used when not specified, but in > > this document we do have a new note that different ports may be used for > > different connection types to the same logical server, and also that > > different ports "is not the essence of the distinction between the two > > endpoints". I think there might be cases where the port is relevant for > > a distinction, but the main ones I can think of are of questionable > > relevance (essentially, roughly equivalent to multiple userspace NFS > > servers on a single host but in different trust/privilege domains) -- > > I'd like another opinion or several. > > > > In a similar "discuss discuss" vein, Section 11.10.8 describes a > > scenario that does not give much clarity, at a protocol level, into what > > degree of replication synchronization a client can expect from a given > > file system that advertises multiple replicas. I recognize that this is > > de facto just stating the deployed reality, but it's also hard to feel > > good about having this level of ambiguity in a propsed standard, and the > > (unchanged) text in Section 11.5.5 seems to impose a stricter > > consistency requirement, at least on potential migration targets. (A > > bit more detail in the COMMENT section.) > > > > Section 11.13.2 mentions that "[i]ssues connected with a client > > impersonating another by presenting another client's id string are > > discussed in Section 21", but I failed to find this discussion in > > Section 21. (The discuss-level issue is just the internal > > inconsistency; there's a decent argument that this is covered by > > Appendix C's "not written in accord with RFC3552". Though if the text > > was already written for draft-ietf-nfsv4-mv1-msns-update, not including > > it here seems a little silly.) > > > > > > ---------------------------------------------------------------------- > > COMMENT: > > ---------------------------------------------------------------------- > > > > I think I may have mistakenly commented on some sections that are > > actually just moved text, since my lookahead window in the diff was too > > small. I expect it's most appropriate to defer those to for the full > > -bis, so sorry to have them lumped in here. > > > > Thank you for all the effort put in to get the diff against RFC 5661 to > > be minimal! I know that the current default output formatting is rather > > different than what is done in RFC 5661, but this diff is pretty easy to > > read. > > > > Thank you also for the detailed discussion in Appendix C; I do not think > > I could add anything more! While the security posture of the current > > deployed state of NFSv4 is not great (though, arguably, somewhat > > understandable given the path we took to get there), this is the right > > start to making any sort of improvement. > > > > Since the "Updates:" header is part of the immutable RFC text (though > > "Updated by:" is mutable), we should probably explicitly state that "the > > updates that RFCs 8178 and 8434 made to RFC 5661 apply equally to this > > document". > > > > I note inline (in what is probably too many places; please don't reply > > at all of them!) some question about how clear the text is that a file > > system migration is something done at a per-file-system granularity, and > > that migrating a client at a time is not possible. As was the case for > > my Discuss point about addresses/port-numbers, I'm missing the context > > of the rest of the document, so perhaps this is a non-issue, but the > > consequences of getting it wrong seem severe enough that I wanted to > > check. > > > > Does a client have any way to know in advance that two addresses will be > > session-trunkable other than the one listed in Section 11.1.1 that "when > > two connections of different connection types are made to the same > > network address and are based on a single file system location entry > > they are always session-trunkable"? It seems like mostly we're defining > > the property by saying that the client has to try it and see if it > > works; I'd love to be wrong about that. > > > > Section 1.1 > > > > The revised description of the NFS version 4 minor version 1 > > (NFSv4.1) protocol presented in this update is necessary to enable > > full use of trunking in connection with multi-server namespace > > features and to enable the use of transparent state migration in > > connection with NFSv4.1. [...] > > > > nit: do we expect all readers to know what is meant by "trunking" with > > no other lead-in? > > > > This limited scope update is applied to the main NFSv4.1 RFC with the > > > > nit: hyphenate "limited-scope" > > > > scope as could expected by a full update of the protocol. Below are > > some areas which are known to need addressing in a future update of > > the protocol. > > [...] > > > > side note: I'd be interested in better understanding the preference for > > the subjunctive verb tense for most of these points ("work would have to > > be done"); my naive expectation would be that since there are plans to > > undertake the work, just "work needs to be done" or "work will be done" > > might suffice. > > > > o Work would have to be done with regard to RFC8178 [63] which > > establishes NFSv4-wide versioning rules. As RFC5661 is curretly > > inconsistent with this document, changes are needed in order to > > arrive at a situation in which there would be no need for RFC8178 > > to update the NFSv4.1 specfication. > > > > nit: s/this document/that document/ -- "this document" is > > draft-ietf-nfsv4-rfc5661sesqui-msns. > > > > o Work would have to be done with regard to RFC8434 [66], which > > establishes the requirements for pNFS layout types, which are not > > clearly defined in RFC5661. When that work is done and the > > resulting documents approved, the new NFSv4.1 specfication > > document will provide a clear set of requirements for layout types > > and a description of the file layout type that conforms to those > > requirements. Other layout types will have their own specfication > > documents that conforms to those requirements as well. > > > > It's not entirely clear to me that the other layout types need to get > > mentioned in this document; how do they relate to the formal status of > > the "current NFSv4.1 core protocol specification document"? > > > > o Work would have to be done to address many erratas relevant to RFC > > 5661, other than errata 2006 [60], which is addressed in this > > document. That errata was not deferrable because of the > > interaction of the changes suggested in that errata and handling > > of state and session migration. The erratas that have been > > deferred include changes originally suggested by a particular > > errata, which change consensus decisions made in RFC 5661, which > > need to be changed to ensure compatibility with existing > > implementations that do not follow the handling delineated in RFC > > 5661. Note that it is expected that such erratas will remain > > > > This sentence is pretty long and hard to follow; maybe it could be split > > after "change consensus decisions made in RFC 5661" and the second half > > start with a more declarative statement about existing implementations? > > (E.g., "Existing implementations did not perform handling as delineated in > > RFC > > 5661 since the procedures therein were not workable, and in order to > > have the specification accurately reflect the existing deployment base, > > changes are needed [...]") > > > > relevant to implementers and the authors of an eventual > > rfc5661bis, despite the fact that this document, when approved, > > will obsolete RFC 5661. > > > > (I assume the RFC Editor can tweak this line to reflect what actually > > happens; my understanding is that the errata reports will get cloned to > > this-RFC.) > > [rant about "errata" vs. "erratum" elided] > > > > Section 2.10.4 > > > > Servers each specify a server scope value in the form of an opaque > > string eir_server_scope returned as part of the results of an > > EXCHANGE_ID operation. The purpose of the server scope is to allow a > > group of servers to indicate to clients that a set of servers sharing > > the same server scope value has arranged to use compatible values of > > otherwise opaque identifiers. Thus, the identifiers generated by two > > servers within that set can be assumed compatible so that, in some > > cases, identifiers generated by one server in that set may be > > presented to another server of the same scope. > > > > Is there more that we can say than "in some cases"? The previous text > > implies a higher level of reliability than just "some cases", to me. > > > > Section 2.10.4 > > > > I see the list of identifier types for which same-scope compatibility > > applies got reduced from RFC 5661 to this document, by removing session > > ID, client ID, and state ID values. For at least one of those I can see > > this making sense as only being workable when the server really is "the > > same server", inline with the improved discussion of migration vs. > > trunking that is a main focus of this document. Does that > > justification apply to all of them, or are there more reasons involved? > > > > We also remove the text about a client needing to compare server scope > > values during a potential migration event, to determine whether the > > migration preserved state or a reclaim is needed. I thought this > > scenario would still be possible (and thus still need to be listed), > > though perhaps we are claiming that it is so under-specified so as to be > > never workable in practice? > > > > Section 2.10.5 > > > > o When eir_server_scope changes, the client has no assurance that > > any id's it obtained previously (e.g. file handles, state ids, > > client ids) can be validly used on the new server, and, even if > > > > It's interesting to see file handles, state ids, and client ids listed > > together here (nit: also with lowercase "id"), when in the previous > > section we have removed state IDs and client IDs from a list that > > includes all three in RFC 5661. > > > > o When eir_server_scope remains the same and > > eir_server_owner.so_major_id changes, the client can use the > > filehandles it has, consider its locking state lost, and attempt > > to reclaim or otherwise re-obtain its locks. It may find that its > > file handle IS now stale but if NFS4ERR_STALE is not received, it > > can proceed to reclaim or otherwise re-obtain its open locking > > state. > > > > nit(?): this bit about "It may find that its file handle IS now stale > > but if NFS4ERR_STALE is not received" seems to assume some familiarity > > by the reader as to what actions would be performed that would get > > NFS4ERR_STALE back. > > > > Section 2.10.5.1 > > > > When the server responds using two different connections claim > > matching or partially matching eir_server_owner, eir_server_scope, > > > > nit: The grammar got wonky here; maybe s/claim/claiming/? > > > > Section 11.1.1 > > > > In the case of NFS version 4.1 and later minor versions, the means > > of trunking detection are as described in this document and are > > available to every client. Two network addresses connected to the > > same server are always server-trunkable but cannot necessarily be > > used together to access a single session. > > > > nit: we haven't defined "server-trunkable" yet, so it may be worth a > > hint that the definition is coming soon. > > > > The combination of a server network address and a particular > > connection type to be used by a connection is referred to as a > > "server endpoint". Although using different connection types may > > result in different ports being used, the use of different ports by > > multiple connections to the same network address is not the essence > > of the distinction between the two endpoints used. > > > > There's perhaps a fine line to walk here, as the port can still have > > significant relevance, in general, and we are frequently in the IETF > > told to make no assumption about what is behind specific port values at > > a given network address. (Consider, for example, a hypothetical virtual > > hosting service that provides "DS-as-a-service" where customers run > > their own MDS that point to configured DSes for actual storage. > > Different ports on that cloud provider would represent entirely > > different customers/servers!) [This became a discuss point but it > > didn't end up including all the discussion here, so I left it as an > > informational thing; discussion should happen in the Discuss section] > > > > Section 11.1.2 > > > > o In some cases, a server will have a namespace more extensive than > > its local namespace by using features associated with attributes > > that provide file system location information. These features, > > which allow construction of a multi-server namespace are all > > > > nit: comma after "multi-server namespace". > > > > o A file system present in a server's pseudo-fs may have multiple > > file system instances on different servers associated with it. > > All such instances are considered replicas of one another. > > > > [Some readers might take this as requiring live read/write replication > > such that all writes to any instance are immediately visible on all > > other instances. The rest of the document ought to disabuse them of > > that notion, and yet...] > > > > o File system location entries provide the individual file system > > locations within the file system location attributes. Each such > > entry specifies a server, in the form of a host name or IP an > > address, and an fs name, which designates the location of the file > > > > nit: s/IP an/an IP/. > > > > client may establish connections. There may be multiple endpoints > > because a host name may map to multiple network addresses and > > because multiple connection types may be used to communicate with > > a single network address. However, all such endpoints MUST > > provide a way of connecting to a single server. The exact form of > > > > nit: "MUST provide" feels strange here, since it implies in some sense > > an extra layer of indirection ("A lists X, and X among other things > > provides Y"); would a different word like "indicate" work? > > > > element derives from a corresponding location entry. When a > > location entry specifies an IP address there is only a single > > corresponding location element. File system location entries that > > contain a host name are resolved using DNS, and may result in one > > or more location elements. All location elements consist of a > > location address which is the IP address of an interface to a > > server and an fs name which is the location of the file system > > within the server's local namespace. The fs name can be empty if > > > > I can't decide whether both instances of "IP address" are pedantically > > correct, in the presence of the potential for port information to be > > included/available. The former is probably okay, but the latter might > > need some clarification. > > > > Section 11.2 > > > > The fs_locations attribute defined in NFSv4.0 is also a part of > > NFSv4.1. This attribute only allows specification of the file system > > locations where the data corresponding to a given file system may be > > found. Servers should make this attribute available whenever > > fs_locations_info is supported, but client use of fs_locations_info > > is preferable, as it provides more information. > > > > I think this was probably okay as "SHOULD make this attribute available" > > (as it was in 5661), but don't object to the lowercase version either. > > > > Section 11.5 > > > > Where a file system had been absent, specification of file system > > > > I guess I'm probably in the rough on this one (since 5661 had my > > more-preferred language), but it still feels like "had been absent" > > implies that it is no longer absent, i.e., that it is now present or has > > otherwise changed. What's going on here with referrals is more like a > > "was never present" case, though using "never" is of course problematic > > as it's more absolute than is appropriate. > > > > > > If we're going to talk about "pure referral"s, do we want to make > > mention of or otherwise differentiate/characterize "non-pure" > > ("impure"?) referrals? > > > > Section 11.5.1 > > > > In order to simplify client handling and allow the best choice of > > replicas to access, the server should adhere to the following > > guidelines. > > > > Just to check: these are just informal "guidelines" and not something > > that a server SHOULD or even MUST adhere to? > > > > Section 11.5.2 > > > > Locations entries used to discover candidate addresses for use in > > > > nit(?): is this supposed to just be "Location" singular? > > > > Section 11.5.3 > > > > Irrespective of the particular attribute used, when there is no > > indication that a step-up operation can be performed, a client > > supporting RDMA operation can establish a new RDMA connection and it > > can be bound to the session already established by the TCP > > connection, allowing the TCP connection to be dropped and the session > > converted to further use in RDMA node. > > > > Should we say something to make this contingent on the server also > > supporting RDMA? > > > > Section 11.5.5 > > > > will typically use the first one provided. If that is inaccessible > > for some reason, later ones can be used. In such cases the client > > might consider that the transition to the new replica as a migration > > event, even though some of the servers involved might not be aware of > > the use of the server which was inaccessible. In such a case, a > > > > nit: the grammar here got wonky; maybe s/as a/is a/? > > > > Section ?? > > > > The old (RFC 5661) Section 11.5 mentioned several things, and I'd like > > to check that we have either covered or disavowed all of them. > > My current understanding is that: > > > > The first paragraph basically talked about trunking detection, and is > > covered elsewhere. > > > > The second paragraph talks about something that I would call "implicit > > replication" with the 5661 definition of "replica", but in the new model > > is essentially definitionally true, since we consider all addresses for > > the same server to be ... part of the same server, so of course that > > server's namespaces match up. Though perhaps the discussion about not > > all of the cartesian product of (addresses-for-server, local path) being > > listed is still worth having? > > > > The third paragraph basically talks about the need for trunking > > detection, and includes some guidance to clients about assuming server > > misconfiguration that seems of questionable merit. > > > > Section 11.5.7 > > > > o Deletions from the list of network addresses for the current file > > system instance need not be acted on immediately, although the > > client might need to be prepared for a shift in access whenever > > the server indicates that a network access path is not usable to > > access the current file system, by returning NFS4ERR_MOVED. > > > > I think this should be wordsmithed a bit more, as (IIUC) the idea here > > is that if a client notices in a location response that the address the > > client is currently using for a filesystem has disappeared from the > > list, the client should be prepared for imminent changes in server > > behavior relating to the presumed-move. Those imminent changes would > > most likely be reflected in the form of the server returning > > NFS4ERR_MOVED, but there is no NFS4ERR_MOVED involved in the actual > > deletion from the list of network instances of the current system > > instance. > > > > Section 11.6 > > > > corresponding attribute is interrogated subsequently. In the case of > > a multi-server namespace, that same promise applies even if server > > boundaries have been crossed. Similarly, when the owner attribute of > > a file is derived from the securiy principal which created the file, > > that attribute should have the same value even if the interrogation > > occurs on a different server from the file creation. > > > > I can see how the interrogation would be on a different server from file > > creation for "simple" replication scenarios, but I'm not sure I'm seeing > > how non-replication cases would arise, paritulcarly that cross server > > boundaries in a multi-server (hierarchical?) namespace. Am I missing > > something obvious? > > nit: s/securiy/security/ > > > > o All servers support a common set of domains which includes all of > > the domains clients use and expect to see returned as the domain > > portion of an owner or group in the form "id@domain". Note that > > although this set most ofen consists of a single domain, it is > > possible for mutiple domains to be supported. > > > > I a little bit wonder if the "most often" still holds when client > > principals come from an AD forest. > > > > o All servers recognize the same set of security principals, and > > each principal, the same credential are required, independent of > > the server being accessed. In addition, the group membership for > > > > nit: I think there's a missing word here, maybe "and for each > > principal"? > > > > Note that there is no requirment that the users corresponding to > > > > nit: "requirement" > > > > o The "local" representation of all owners and groups must be the > > same on all servers. The word "local" is used here since that is > > the way that numeric user and group ids are described in > > Section 5.9. However, when AUTH_SYS or stringified owners or > > group are used, these identifiers are not truly local, since they > > are known tothe clients as well as the server. > > > > I am trying to find a way to note that the AUTH_SYS case mentioned here > > is precisely because of the requirement being imposed by this bullet > > point, while acknowledging that the "stringified owners or group" case > > is separate, but not having much luck. > > Also, nit: "to the" > > > > Section 11.9 > > > > o When use of a particular address is to cease and there is also one > > currently in use which is server-trunkable with it, requests that > > would have been issued on the address whose use is to be > > discontinued can be issued on the remaining address(es). When an > > address is not a session-trunkable one, the request might need to > > be modified to reflect the fact that a different session will be > > used. > > > > I suggest writing this as "when an address is server-trunkable but not > > session-trunkable,". > > > > o When use of a particular connection is to cease, as indicated by > > receiving NFS4ERR_MOVED when using that connection but that > > address is still indicated as accessible according to the > > appropriate file system location entries, it is likely that > > requests can be issued on a new connection of a different > > connection type, once that connection is established. Since any > > two server endpoints that share a network address are inherently > > session-trunkable, the client can use BIND_CONN_TO_SESSION to > > access the existing session using the new connection and proceed > > to access the file system using the new connection. > > > > I'm not entirely sure how "inherent" this is (in the vein of my Discuss > > point, and what we mean by "network address"). > > > > o When there are no potential replacement addresses in use but there > > > > What is a "replacement address"? > > > > are valid addresses session-trunkable with the one whose use is to > > be discontinued, the client can use BIND_CONN_TO_SESSION to access > > the existing session using the new address. Although the target > > session will generally be accessible, there may be cases in which > > that session is no longer accessible. In this case, the client > > can create a new session to enable continued access to the > > existing instance and provide for use of existing filehandles, > > stateids, and client ids while providing continuity of locking > > state. > > > > I'm not sure I understand this last sentence. On its own, the "new > > session to enable continued access to the existing instance" sounds like > > the continued access would be on the address whose use is to cease, and > > thus the new session would be there. But why make a new session when > > the old one is still good, especially when we just said in the previous > > sentence that the old session can't be moved to the new > > connection/address? > > Perhaps a forward reference down to Section 11.12.{4,5} for this and the > > next bullet point would help as well as rewording? > > > > Section 11.10.6 > > > > In a file system transition, the two file systems might be clustered > > in the handling of unstably written data. When this is the case, and > > > > What does "clustered in the handling of unstably written data" mean? > > > > the two file systems belong to the same write-verifier class, write > > > > How is the client supposed to determine "when this is the case"? > > > > Section 11.10.7 > > > > In a file system transition, the two file systems might be consistent > > in their handling of READDIR cookies and verifiers. When this is the > > case, and the two file systems belong to the same readdir class, > > > > As above, how is the client supposed to determine "when this is the > > case"? > > > > READDIR cookies and verifiers from one system may be recognized by > > the other and READDIR operations started on one server may be validly > > continued on the other, simply by presenting the cookie and verifier > > returned by a READDIR operation done on the first file system to the > > second. > > > > Are these "may be"s supposed to admit the possibility that the > > destination server can just decide to not honor them arbitrarily? > > > > Section 11.10.8 > > > > the degree indicated by the fs_locations_info attribute). However, > > when multiple file systems are presented as replicas of one another, > > the precise relationship between the data of one and the data of > > another is not, as a general matter, specified by the NFSv4.1 > > protocol. It is quite possible to present as replicas file systems > > where the data of those file systems is sufficiently different that > > some applications have problems dealing with the transition between > > replicas. The namespace will typically be constructed so that > > applications can choose an appropriate level of support, so that in > > one position in the namespace a varied set of replicas will be > > listed, while in another only those that are up-to-date may be > > considered replicas. [...] > > > > This seems quite wishy-washy for a standards-track protocol! We give no > > hard bounds on how "different" replicas may be, no protocol element to > > convey even a qualitative sense of where on the spectrum of replication > > fidelity a replica may lie, and no indication as to how the namespace > > might be constructed to indicate a level of support. > > > > The protocol does define three special cases of > > the relationship among replicas to be specified by the server and > > relied upon by clients: > > > > I'd like to hear from the rest of the IESG, but we may need to consider > > limiting "replication" to just these special cases until we can be more > > precise about the other cases. > > > > o When multiple replicas exist and are used simultaneously by a > > client (see the FSLIB4_CLSIMUL definition within > > fs_locations_info), they must designate the same data. Where file > > systems are writable, a change made on one instance must be > > visible on all instances, immediately upon the earlier of the > > return of the modifying requester or the visibility of that change > > on any of the associated replicas. This allows a client to use > > > > Hmm, how would this "earlier of [...]" work when there are three > > nominally equivalent machines? Assume the RPC is made to A, and the > > other two are B and C. If the update first goes visible on B, it must > > also be visible on C, instilling what is apparently a hard requirement > > for exact synchronization between B an C, perhaps by some sort of > > negotiated "make visible at timestamp X" mechanism. But if the RPC > > returns from A first, then the change still has to be visible on B and C > > at the same time. Does this phrasing give any weaker a requirement than > > "must be visible on all machines at the same time", in practice? (There > > are, of course, various distributed-consensus protocols that can do > > this, as could a scenario where all NFS servers are connected to a > > common file store backend.) > > > > Section 11.10.9 > > > > When access is transferred between replicas, clients need to be > > assured that the actions disallowed by holding these locks cannot > > > > To check my understanding: this "access is transferred" means *all* > > clients' access (not just one particular client)? Otherwise I'm not > > sure how the destination would know to enforce the grace period. > > > > Section 11.11.1 > > > > I think the last two paragraphs might be duplicating some things > > mentioned earlier in the section, but the repetition is probably not > > harmful. > > > > Section 11.12.1 > > > > Because of the absence of NFSV4ERR_LEASE_MOVED, it is possible for > > file systems whose access path has not changed to be successfully > > > > It might be worth phrasing this as "SEQ4_STATUS_LEASE_MOVED is not an > > error condition". > > > > Section 11.12.2 > > > > o No action needs to be taken for such indications received by the > > those performing migration discovery, since continuation of that > > work will address the issue. > > > > nit: "by the those" is not right, but the proper fix eludes me, as this > > bullet point needs to be more specific somehow than the next one. > > > > o If the fs_status attribute indicates that the file system is a > > migrated one (i.e. fss_absent is true and fss_type != > > STATUS4_REFERRAL) and thus that it is likely that the fetch of the > > file system location attribute has cleared one the file systems > > contributing to the lease-migrated indication. > > > > This looks like a sentence fragment -- it's of the form "If X, and thus > > Y." with no concluding clause. > > > > Section 11.12.4 > > > > Once the client has determined the initial migration status, and > > determined that there was a shift to a new server, it needs to re- > > establish its locking state, if possible. To enable this to happen > > without loss of the guarantees normally provided by locking, the > > destination server needs to implement a per-fs grace period in all > > cases in which lock state was lost, including those in which > > Transparent State Migration was not implemented. > > > > Similarly to above, does this imply that the migration has to happen for > > all clients concurrently, as opposed to clients getting migrated in > > sequence? > > > > Section 11.3.1 > > > > In this case, destination server need have no knowledge of the locks > > > > nit: singular/plural mismatch "destination server"/"need" > > > > Section 11.13.3 > > > > o Not responding with NFS4ERR_SEQ_MISORDERED for the initial request > > on a slot within a transferred session, since the destination > > > > Does this then translate to "process as usual in the absence of > > migration"? "Don't return error X" tells me what not to do, but doesn't > > really tell me what to do instead. > > > > Section 11.16.1 > > > > With the exception of the transport-flag field (at offset > > FSLI4BX_TFLAGS with the fls_info array), all of this data applies to > > the replica specified by the entry, rather that the specific network > > path used to access it. > > > > Is it clear that this applies only to the fields defined by this > > specification (since, as mentioned later, future extensions must specify > > whether they apply to the replica or the entry)? > > > > Section 15.1.1.3 > > > > o When NFS4ERR_DELAY is returned on an operation other than the > > first within a request and there has been a non-idempotent > > operation processed before the NFS4ERR_DELAY was returned, the > > reissued request should avoid the non-idempotent operation. The > > request still must use a SEQUENCE operation with either a > > different slot id or sequence value from the SEQUENCE in the > > original request. Because this is done, there is no way the > > replier could avoid spuriously re-executing the non-idempotent > > operation since the different SEQUENCE parameters prevent the > > requester from recognizing that the non-idempotent operation is > > being retried. > > > > I don't think that this is very clear about the counterfactual scenario > > in which the replier is trying to avoid spuriously re-executing the > > non-idempotent operation. Is it supposed to be explaining why the > > client has to use a different slot or sequence value, because the > > replier would reexecute the non-idempotent operation otherwise? > > > > Section 18.35.3 > > > > I a little bit wonder if we want to reaffirm that co_verifier remains > > fixed when the client is establishing multiple connections for trunking > > usage -- the "incarnation of the client" language here could make a > > reader wonder, though I think the discussion of its use elsewhere as > > relating to "client restart" is sufficiently clear. > > > > The eia_clientowner field is composed of a co_verifier field and a > > co_ownerid string. As noted in s Section 2.4, the co_ownerid > > > > s/s // > > > > Section 18.51.4 > > > > o When a server might become the destination for a file system being > > migrated, inappropriate use of per-fs RECLAIM_COMPLETE is more > > concerning. In the case in which the file system designated is > > not within a per-fs grace period, the per-fs RECLAIM_COMPLETE > > SHOULD be ignored, with the negative consequences of accepting it > > being limited, as in the case in which migration is not supported. > > However, if the server encounters a file system undergoing > > migration, the operation cannot be accepted as if it were a global > > RECLAIM_COMPLETE without invalidating its intended use. > > > > This seems to be the only place where we acknowledge that the "misuse" > > in question was to "treat rca_one_fs of TRUE as if it was FALSE", which > > is probably not so great for clarity. > > > > Section 21 > > > > Some other topics at least somewhat related to trunking and migration > > that we could potentially justify including in the current, > > limited-scope, update (as opposed to deferring for a full -bis) include: > > > > - clients that lie about reclaimed locks during a post-migration grace > > period > > - how attacker capabilities compare by using a compromised server to > > give bogus referrals/etc. as opposed to just giving bogus data/etc. > > - an attacker in the network trying to shift client traffic (in terms of > > what endpoints/connections they use) to overload a server > > - how asynchronous replication can cause clients to repeat > > non-idempotent actions > > - the potential for state skew and/or data loss if migration events > > happen in close succession and the client "misses a notification" > > - cases where a filesystem moves and there's no longer anything running > > at the old network endpoint to return NFS4ERR_MOVED > > - what can happen when non-idempotent requests are in a COMPOUND before > > a request that gets NFS4ERR_MOVED > > - how bad it is if the client messes up at Transparent State Migration > > discovery, most notably in the case when some lock state is lost > > - the interactions between cached replies and migration(-like) events, > > though a lot of this is discussed in section 11.13.X and 15.1.1.3 > > already > > > > but I defer to the WG as to what to cover now vs. later. > > > > In light of the ongoing work on draft-ietf-nfsv4-rpc-tls, it might be > > reasonable to just talk about "integrity protection" as an abstract > > thing without the specific focus on RPCSEC_GSS's integrity protection > > (or authentication) > > > > being returned. These include cases in which the client is > > directed a server under the control of an attacker, who might get > > > > nit: "directed to" > > > > o Despite the fact that it is a requirement that "implementations" > > provide "support" for use of RPCSEC_GSS, it cannot be assumed that > > use of RPCSEC_GSS is always available between any particular > > client-server pair. > > > > side note: scare-quotes around "support" makes sense to me, but not > > around "implementations". > > > > the destination. Even when RPCSEC_GSS authentication is available on > > the destination, the server might validly represent itself as the > > server to which the client was erroneously directed. Without a way > > > > Something about the wording here tickles me funny; at first I thought it > > was the "validly", but now I think it's "represent itself", perhaps > > because that phrasing can have connotations of "falsely represent". > > ("Valid" is fine -- the attack here is the misdirection, and the target > > of the misdirection doesn't have to misbehave at all for it to be a > > damaging attack.) The best remedy I can come up with is a somewhat > > drastic change, and thus questionable: "Even when [...], the server > > might still properly authenticate as the server to which the client was > > erroneously directed." > > > > > > I'd also consider adding a third bullet point to the final list ("to > > summarize considerations regarding the use of RPCSEC_GSS"): > > > > % o The integrity protection afforded to results by RPCSEC_GSS protects > > % only a given request/response transaction; RPCSEC_GSS does not > > % protect the binding from one server to another as part of a referral > > % or migration event. The source server must be trusted to provide > > % the correct information, based on whatever factors are available to > > % the client. > > > > Section 22.1 > > > > Thank you for thinking about how the IANA considerations should be > > presented in the post-update document. (I think I've had to place at > > least two Discuss positions on bis documents that did not...) > > > > Section 23.2 > > > > I'm not sure that all of the moves from Normative to Informative should > > stick; e.g., HMAC (which went from [11] to [59]) is needed for SSV > > calculation. Hmm, actually, maybe that's the only one. > > > > Appendix B > > > > I have mixed feelings about whether to keep this content for the final > > RFC. (Appendix A seems clearly useful; the specific details of the > > reorganization are less clear, as to some extent they can be deduced > > from the changes themselves. But only to some extent...) > > > > Appendix B.1.2 > > > > o The new Sections 11.8 and 11.9 have resulted in existing sections > > wit these numbers to be renumbered. > > > > s/wit/with/ > > > > Section B.2.1 > > > > The new treatment can be found in Section 18.35 below. It is > > > > s/below/above/ > > > > intended to supersede the treatment in Section 18.35 of RFC5661 [62]. > > Publishing a complete replacement for Section 18.35 allows the > > corrected definition to be read as a whole, in place of the one in > > RFC5661 [62]. > > > > This seems like it was more appropriate in the scope of > > draft-ietf-nfsv4-mv1-msns-update but could be obsolete here. > > > > Section B.4 > > > > o The discussion of trunking which appeared in Section 2.10.5 of > > RFC5661 [62] needed to be revised, to more clearly explain the > > multiple types of trunking supporting and how the client can be > > made aware of the existing trunking configuration. In addition > > the last paragraph (exclusive of sub-sections) of that section, > > dealing with server_owner changes, is literally true, it has been > > a source of confusion. [...] > > > > nit: the grammar here is weird; I think there's a missing "while" or > > similar. > > > > > >
- [nfsv4] Benjamin Kaduk's Discuss on draft-ietf-nf… Benjamin Kaduk via Datatracker
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… David Noveck
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… Benjamin Kaduk
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… David Noveck
- [nfsv4] Benjamin Kaduk's Discuss on draft-ietf-nf… David Noveck
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… Benjamin Kaduk
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… Benjamin Kaduk
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… Benjamin Kaduk
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… David Noveck
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… David Noveck
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… Benjamin Kaduk
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… Benjamin Kaduk
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… David Noveck
- Re: [nfsv4] Benjamin Kaduk's Discuss on draft-iet… Magnus Westerlund