Re: [nfsv4] AD Evaluation of draft-ietf-nfsv4-mv1-msns-update-03

[Spencer Dawkins at IETF <[e-mail]>]

Hi, David,

On Sat, Feb 2, 2019 at 8:15 AM David Noveck <[e-mail]> wrote:

> > since I'm doing AD Evaluation on a 100-page document that updates a
> 600-page document, we can't be surprised that
> > I had a few questions
>
> Actually, I'm only surprised that it was done so quickly.   Thanks.
>
> > please let me know how you'd like to proceed.
> > I'll mark this as Revised ID Required for now.
>
> I'll provide initial responses below.   If there turns out to be anything
> controversial or difficult to resolve, I'll just note the issue now and
> discuss it with my co-author.   Based on my initial scan of your comments,
> my expectation is that a revised I-D could be provided in one to two weeks.
>

I'm not seeing any of your responses that worry me, so please proceed in
that direction, and I'll chase down the details on "pre-RFC5378" work, and
follow up in a separate email, most likely today.

Thanks for the quick response!

Spencer (D)


> > This popped up from IDnits, and would be worth fixing before Last Call.
>
> It seems that not fixing it is not really an option, much as I'd like to
> sweep the whole thing under the rug :-(.
>
> > -- The document seems to lack a disclaimer for pre-RFC5378 work, but may
> >     have content which was first submitted before 10 November 2008.  If
> you
> >     have contacted all the original authors
>
> I'm not clear about the set of people within "all the original authors".
>  If this referring to the authors of RFC5661, then it is just contacting
> Spencer S and Mike Eisler, in addition to myself.  On the other hand, some
> material from RFC3530 may have migrated to RFC5661, which mkes the poblem
> more difficult.
>
> > and they are all willing to grant
> >     the BCP78 rights to the IETF Trust, then this is fine, and you can
> ignore
> >     this comment.
>
> I looked at BCP78 and it was not very clear what exactly I would be
> agreeing to or asking other authors to agree to, beyond what one would
> have agreed to before 11/10/2008.   BCP78 does  have a section entitled
> "Changes since RFC 3968" but it isn't very helpful.
>
> > If not, you may need to add the pre-RFC5378 disclaimer.
> >     (See the Legal Provisions document at
> >    https://trustee.ietf.org/license-info for more information.)
>
> I looked in that document but I couldn't find the text of the needed
> disclaimer.
>
> > (This one tends to get Discusses during IESG Evaluation)
>
> I think I can avoid that.
>
> > The description of "trunking" appears (as best I can tell) appears 7
> pages into the document. Is there a reasonable
> > reference you could provide for the first use of the term in the
> Introduction? (Even a forward reference to Section 3.1
> > would be an improvement)
>
> What I'm anticipating doing in -04 is prepending the following sentences
> to the final paragraph of Section 1:
>
> This document supplements facilities related to trunking, introduced in
> [RFC5661].  For some important terminology regarding trunking, see Section
> 3.1.
>
>
> >Does this text
> >
> > o  Some confusing text regarding changes in server_owner needs to be
> >                                                          ^^^^^^^^^^^
> >      clarified.
> >
> > mean
>
> >  o  Some confusing text regarding changes in server_owner has been
> >                                                           ^^^^^^^^
> >      clarified.
>
> >?
>
> Yes.  Will be fixed in -04.
>
> >Does this text
>
> >  This document contains sections that propose additions to and other
> >                                       ^^^^^^^
> >   modifications of [RFC5661] as well as others that explain the reasons
> >   for modifications but do not directly affect existing specifications.
>
> > mean "specify additions", or "provide additions"? If approved, this
> specification will no longer be proposing text changes > - they'll be in
> effect.
>
> Yes.  Will fix in -04.
>
> > I can usually follow passive tense English in IETF specifications, but
> >
> >  When a file system is present and becomes absent, clients can be
> >                                                            ^^^^^^
> >   given the opportunity to have continued access to their data, using a
> >
> >   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >    different replica.
> >
> > is just a bit too hand-wavish. Who/what is giving clients this
> opportunity?
>
> At one level, it is the protocol that provides the ability  to communicate
> the new location.  At another level it is the source and destination
> servers that co-operate to effect the necessary handoff.
>
> I anticipate rewriting the sentence as follows in -04:
>
> When a file system is present and becomes absent, the NFSv4 protocol
> provides a means by which clients can be given the opportunity to have
> continued access to their data, using a different replica.
>
>
>
> >This is a pretty mundane point, but in text like this (and there are
> other similar occurrences):
> >
> > 4.5.1.  New section to be added as the first sub-section of Section 11.4
> >        of [RFC5661] to be entitled "Combining Multiple Uses in a Single
> >       Attribute"
> >
> > it is not clear to me exactly where the added text goes - I'm guessing
> that the added text becomes the new 11.4.1,
> > pushing all other subsections of 11.4 down, but I'm not sure if that's
> what was intended. I just noticed that >
> https://www.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-mv0-trunking-update-04
> shows an explicit renumbering ("this section is
> > now 11.4.1") being added to similar insertions, which is likely more
> helpful for the area review teams, reviewing ADs, the > RFC Editor, and the
> eventual readers. That happened during a recent IESG Evaluation, so it's
> especially likely to come > up for another NFSv4 draft ;-) ...
>
> I can do something anaogous for this case by adding similar material to
> Section 4.4 in -04.
>
> For the othr cases, I can do something similar but I may have to create or
> modify sections which describe reoganizations of various pieces of
> RFC5661's Section 11.
>
> > I'm lost in the "it"s in this text:
> >
> >  o  It may fetch the file system location attribute for the filesystem
> >      which will provide either the name of the server (which can be
> >      turned into a set of network addresses using DNS), or it will find
> >      a set of server-trunkable location entries which can provide the
> >      addresses specified by the server as desirable to use to access
> >     the file system in question.
> >
> > I THINK this is (indentation is significant)
>
> > o  The client may fetch the file system location attribute for the
> filesystem
> >      which will provide either
> >
> >         the name of the server (which can be
> >          turned into a set of network addresses using DNS),
> >
> >     or the client will find a set of server-trunkable location entries
> which can
> >
> >          provide the
> >          addresses specified by the server as desirable to use to access
> >         the file system in question.
> >
> > just following the text, but that's not making a lot of sense to me.
>
> :-(
>
> > Is "or it will find" garbled?
>
> I don't think so.
>
> > Or am I just lost here?
>
> I don't think so, but since you feel lost, I'l propose a rewrite and see
> if that clarifies things.
>
> How about:
>
> The client may fetch the file system location attribute for the
> filesystem.  This will provide either:
>
>    - The name of the server (which can be turned into a set of network
> addresses using DNS).
>
>    - A set of server-trunkable location entries specified by the server as
> desirable to use to access the file system in      question.
>
> >
> >     or the client will find a set of server-trunkable location entries
> which can    rewrite and see how that strikes you.
>
> How about:
>
>
> > It's a nit, but
> >
> >  Fs_locations_info provides a flag, FSLI4TF_RDMA flag.  indicating
>
> >                                                     ^ should be a comma.
>
> Yes.  Will fix in -04.
>
> >   that RPC-over-RDMA support is available using the specified location
> >   entry.
>
> > I'm not entirely following
> >
> >  Fs_locations_info provides a flag, FSLI4TF_RDMA flag.  indicating
> >   that RPC-over-RDMA support is available using the specified location
> >   entry.
>
> This is true.
>
> >This flag makes it for a convenient for a client wishing to
> > use RDMA, to establish a TCP connection and then convert to use of
> >  RDMA.  After establishing a TCP connection, the step-up facility, can
> >   be used, if available, to convert that connection to RDMA mode.
>
> This would make it convenient, if server implemented it, but it turns out
> they generally don't.   Still, they could do so.
>
> > Otherwise, if RDMA availability is indicated, a new RDMA connection
> >   can be established and it can be bound to the session already
> >   established by the TCP connection, allowing the TCP connection to be
> >   dropped and the session converted to further use in RDMA node.
>
> > because I'm trying to guess why you would establish a TCP connection and
> then convert that to RDMA mode,
> > instead of establishing an RDMA connection in the first place.
>
> Although you have been successful in guessing up to now, it is not
> surprising that you can't guess this, as it depends on some of the history
> of RDMA in the IETF.
>
> > A quick spin through Section 18.16 of [RFC5661] didn't give me a lot of
> guidance here (but I did look).
>
> It doesn't explain why this was put in, but I'm guessing that, when it was
> written, it seemrd not to require explanation.  At that time, when RDMA was
> discussed, it was assumed you were referring to iWarp.  In the contect of
> iWarp, stepping a connection from TCP to RDMA is natural, since you are
> adding an RDMA facility to an existing TCP connection and there is no point
> in tearing down the TCP connection and then establishing a new one.   Now,
> given that use of IB and roCE are more common, Inerest in this has waned,
> since non-iWarp implementations cannot use this fcility and iplementers may
> not want to  invest in an iWarp-only facility.
>
> >It's a nit, but
> >
> >  or locations of the file system can be determined by fetching the a
> >   locations attribute.  attribute.
> >
>  >                        ^^^^^^^^^^ is probably a cut and paste error.
>
> Probably.  In any case, it will be fixed in -04.
>
> > It would be helpful to readers like me, if
> >
> > Generally, multi-server namespaces are for the most part uniform, in
> >   that the same data made available to one client at a given location
> >   in the namespace is made available to all clients at that location.
>  >  However, as described above, there are facilities provided that allow
> >
> >           ^^^^^^^^^^^^^^^^^^
> >   different clients to be directed different sets of data, to enable
> >   adaptation to such client characteristics as CPU architecture.
> >
> > this cross-reference was a little more precise (is it "earlier in this
> section", or "in Section M.N.O" - something like that).
>
> Actually the reference is wrong rather than being (only) imprecise.  It
> should be section 11.10.3 of [RFC5661].  Will fix in -04.
>
> > This text,
> >
> >  o  Whenever a SEQUENCE operation is sent by a client to a server
> >      which generated state held on that client which is associated with
> >      a file system that is no longer accessible on the server at which
> >      it was previously available, a lease-migrated indication, in the
> >
>  >                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>  >     form the SEQ4_STATUS_LEASE_MOVED status bit being set, appears in
> >
>  >     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >
>  >     the response.
> >
> > wasn't easy for me to parse. Perhaps something like
> >
> >
> >  o  Whenever a SEQUENCE operation is sent by a client to a server
> >      which generated state held on that client which is associated with
>  >     a file system that is no longer accessible on the server at which
> >      it was previously available, the response will contain a
> lease-migrated
> >
>  >
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >
>  >     indication, with the SEQ4_STATUS_LEASE_MOVED status bit being set.
> >
>  >     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >
>
> Seems clearer.  Will pick up for -04
>
>
>
> >  o  In the state merger case, it is possible that the server has not
> >      attempted Transparent State Migration, in which case state may
> >      have been lost without it being reflected in the SEQ4_STATUS bits.
> >      To determine whether this has happened, the client can use
> >      TEST_STATEID to check whether the stateids created on the source
> >      server are still accessible on the destination server.  Once a
> >      single stateid is found to have been successfully transferred, the
> >      client can conclude that Transparent State Migration was begun and
> >      any failure to transport all of the stateids will be reflected in
> >      the SEQ4_STATUS bits.  Otherwise.  Transparent State Migration has
> >
> >                                     ^ should this be a comma?
>
> It should.  Will fix in -04.
>
> >      not occurred.
>
>
> > but asking now, because there might be something else happening with
> this text, if it's not a comma.
>
>
> > I couldn't parse this paragraph. I think the problem is in the first
> phrase:
>
>
>  > o  If information about the set of clients that have locking state
> >
> >    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >     for the transferred file system, the destination server will be
> >    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >     able to terminate the grace period once all such clients have
> >      reclaimed their locks, allowing normal locking activity to resume
> >      earlier than it would have otherwise.
>
> It should be "If information about the set of clients that have locking
> state for the transferred file system is made available".  Will fix in -04.
>
> >It's a nit, but
> >
> >  o  For each lock type there will by type-specific information, such
> >
> >                                   ^ be
>
> In -04, will be "for each lock type, there will be type-specfic
> information,".
>
> >      as share and deny modes for opens and type and byte ranges for
> >      byte-range locks and layouts.
>
>
> > If this matters,
>
>   > o  If both EXCHANGE_ID requests were sent with RPCSEC_GSS ([RFC2203],
>   >    [RFC5403], [RFC7861]) authentication and the server principal is
>   >    the same for both targets, the equality of server scope is
>   >    validated.  It is RECOMMENDED that two servers intending to share
>  >
>   >                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   >    the same server scope also share the same principal name.
>  >
>   >    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >
> >
> > the reader would likely benefit from a sentence or two explaining why
> this is RECOMMENDED, and/or what the
> > consequences of not sharing the same principal name are.
>
> Right now, I'm not sure.  This text was copied over from RFC5661.
>
> > It's a nit, but
> >
> >  o  The cases of change which are very disruptive (e.g. change if
> >
> >                                            "if" is likely "of" ^
>
> It will by "of" in -04.
>
> >      server scope) are not sufficiently distinguished from those that
> >      simply involve a change of trunking modes (i.e. change
> >      server_owner minor id)
>
>
> On Fri, Feb 1, 2019 at 8:55 PM Spencer Dawkins at IETF <
> [e-mail]> wrote:
>
>> Dear Author and Editor,
>>
>>
>> The NFSv4 working group tends to produce solid documents - thank you for
>> that - but since I'm doing AD Evaluation on a 100-page document that
>> updates a 600-page document, we can't be surprised that I had a few
>> questions ... please let me know how you'd like to proceed.
>>
>> I'll mark this as Revised ID Required for now.
>>
>> Thanks,
>>
>> Spencer
>>
>> This popped up from IDnits, and would be worth fixing before Last Call.
>>
>> -- The document seems to lack a disclaimer for pre-RFC5378 work, but may
>>     have content which was first submitted before 10 November 2008.  If
>> you
>>     have contacted all the original authors and they are all willing to
>> grant
>>     the BCP78 rights to the IETF Trust, then this is fine, and you can
>> ignore
>>     this comment.  If not, you may need to add the pre-RFC5378
>> disclaimer.
>>     (See the Legal Provisions document at
>>     https://trustee.ietf.org/license-info for more information.)
>>
>> (This one tends to get Discusses during IESG Evaluation)
>>
>> The description of "trunking" appears (as best I can tell) appears 7
>> pages into the document. Is there a reasonable reference you could provide
>> for the first use of the term in the Introduction? (Even a forward
>> reference to Section 3.1 would be an improvement)
>>
>> Does this text
>>
>>  o Some confusing text regarding changes in server_owner needs to be
>>
>>                                                           ^^^^^^^^^^^
>>      clarified.
>>
>> mean
>>
>>  o Some confusing text regarding changes in server_owner has been
>>
>>                                                           ^^^^^^^^
>>      clarified.
>>
>> ?
>>
>> Does this text
>>
>>  This document contains sections that propose additions to and other
>>
>>                                       ^^^^^^^
>>   modifications of [RFC5661] as well as others that explain the reasons
>>   for modifications but do not directly affect existing specifications.
>>
>> mean "specify additions", or "provide additions"? If approved, this
>> specification will no longer be proposing text changes - they'll be in
>> effect.
>>
>> I can usually follow passive tense English in IETF specifications, but
>>
>>  When a file system is present and becomes absent, clients can be
>>
>>                                                            ^^^^^^
>>   given the opportunity to have continued access to their data, using a
>>
>>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>   different replica.
>>
>> is just a bit too hand-wavish. Who/what is giving clients this
>> opportunity?
>>
>> This is a pretty mundane point, but in text like this (and there are
>> other similar occurrences):
>>
>> 4.5.1.  New section to be added as the first sub-section of Section 11.4
>>        of [RFC5661] to be entitled "Combining Multiple Uses in a Single
>>        Attribute"
>>
>> it is not clear to me exactly where the added text goes - I'm guessing
>> that the added text becomes the new 11.4.1, pushing all other subsections
>> of 11.4 down, but I'm not sure if that's what was intended. I just noticed
>> that
>> https://www.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-mv0-trunking-update-04
>> shows an explicit renumbering ("this section is now 11.4.1") being added to
>> similar insertions, which is likely more helpful for the area review teams,
>> reviewing ADs, the RFC Editor, and the eventual readers. That happened
>> during a recent IESG Evaluation, so it's especially likely to come up for
>> another NFSv4 draft ;-) ...
>>
>> I'm lost in the "it"s in this text:
>>
>>  o It may fetch the file system location attribute for the filesystem
>>      which will provide either the name of the server (which can be
>>      turned into a set of network addresses using DNS), or it will find
>>      a set of server-trunkable location entries which can provide the
>>      addresses specified by the server as desirable to use to access
>>      the file system in question.
>>
>> I THINK this is (indentation is significant)
>>
>> o  The client may fetch the file system location attribute for the
>> filesystem
>>      which will provide either
>>
>>          the name of the server (which can be
>>          turned into a set of network addresses using DNS),
>>
>> or the client will find a set of server-trunkable location entries which
>> can
>>
>>          provide the
>>          addresses specified by the server as desirable to use to access
>>          the file system in question.
>>
>> just following the text, but that's not making a lot of sense to me. Is
>> "or it will find" garbled? Or am I just lost here?
>>
>> It's a nit, but
>>
>>  Fs_locations_info provides a flag, FSLI4TF_RDMA flag.  indicating
>>
>>                                                      ^ should be a comma.
>>   that RPC-over-RDMA support is available using the specified location
>>   entry.
>>
>> I'm not entirely following
>>
>>  Fs_locations_info provides a flag, FSLI4TF_RDMA flag.  indicating
>>   that RPC-over-RDMA support is available using the specified location
>>   entry.  This flag makes it for a convenient for a client wishing to
>>   use RDMA, to establish a TCP connection and then convert to use of
>>   RDMA.  After establishing a TCP connection, the step-up facility, can
>>   be used, if available, to convert that connection to RDMA mode.
>>   Otherwise, if RDMA availability is indicated, a new RDMA connection
>>   can be established and it can be bound to the session already
>>   established by the TCP connection, allowing the TCP connection to be
>>   dropped and the session converted to further use in RDMA node.
>>
>> because I'm trying to guess why you would establish a TCP connection and
>> then convert that to RDMA mode, instead of establishing an RDMA connection
>> in the first place. A quick spin through Section 18.16 of [RFC5661] didn't
>> give me a lot of guidance here (but I did look).
>>
>> It's a nit, but
>>
>>  or locations of the file system can be determined by fetching the a
>>   locations attribute.  attribute.
>>
>>                         ^^^^^^^^^^ is probably a cut and paste error.
>>
>> It would be helpful to readers like me, if
>>
>>  Generally, multi-server namespaces are for the most part uniform, in
>>   that the same data made available to one client at a given location
>>   in the namespace is made available to all clients at that location.
>>   However, as described above, there are facilities provided that allow
>>
>>            ^^^^^^^^^^^^^^^^^^
>>   different clients to be directed different sets of data, to enable
>>   adaptation to such client characteristics as CPU architecture.
>>
>> this cross-reference was a little more precise (is it "earlier in this
>> section", or "in Section M.N.O" - something like that).
>>
>> This text,
>>
>>  o Whenever a SEQUENCE operation is sent by a client to a server
>>      which generated state held on that client which is associated with
>>      a file system that is no longer accessible on the server at which
>>      it was previously available, a lease-migrated indication, in the
>>
>>                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>      form the SEQ4_STATUS_LEASE_MOVED status bit being set, appears in
>>
>>      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>
>>      the response.
>>
>> wasn't easy for me to parse. Perhaps something like
>>
>>  o Whenever a SEQUENCE operation is sent by a client to a server
>>      which generated state held on that client which is associated with
>>      a file system that is no longer accessible on the server at which
>>      it was previously available, the response will contain a
>> lease-migrated
>>
>>
>>                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>
>>      indication, with the SEQ4_STATUS_LEASE_MOVED status bit being set.
>>
>>      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>
>> I'm guessing
>>
>>  o In the state merger case, it is possible that the server has not
>>      attempted Transparent State Migration, in which case state may
>>      have been lost without it being reflected in the SEQ4_STATUS bits.
>>      To determine whether this has happened, the client can use
>>      TEST_STATEID to check whether the stateids created on the source
>>      server are still accessible on the destination server.  Once a
>>      single stateid is found to have been successfully transferred, the
>>      client can conclude that Transparent State Migration was begun and
>>      any failure to transport all of the stateids will be reflected in
>>      the SEQ4_STATUS bits.  Otherwise. Transparent State Migration has
>>
>>                                      ^ should this be a comma?
>>      not occurred.
>>
>> but asking now, because there might be something else happening with this
>> text, if it's not a comma.
>>
>> I couldn't parse this paragraph. I think the problem is in the first
>> phrase:
>>
>>  o If information about the set of clients that have locking state
>>
>>     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>      for the transferred file system, the destination server will be
>>
>>      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>      able to terminate the grace period once all such clients have
>>      reclaimed their locks, allowing normal locking activity to resume
>>      earlier than it would have otherwise.
>>
>> It's a nit, but
>>
>>  o For each lock type there will by type-specific information, such
>>
>>                                    ^ be
>>      as share and deny modes for opens and type and byte ranges for
>>      byte-range locks and layouts.
>>
>> If this matters,
>>
>>  o If both EXCHANGE_ID requests were sent with RPCSEC_GSS ([RFC2203],
>>      [RFC5403], [RFC7861]) authentication and the server principal is
>>      the same for both targets, the equality of server scope is
>>      validated.  It is RECOMMENDED that two servers intending to share
>>
>>                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>      the same server scope also share the same principal name.
>>
>>      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>
>> the reader would likely benefit from a sentence or two explaining why
>> this is RECOMMENDED, and/or what the consequences of not sharing the same
>> principal name are.
>>
>> It's a nit, but
>>
>>  o The cases of change which are very disruptive (e.g. change if
>>
>>                                            "if" is likely "of" ^
>>      server scope) are not sufficiently distinguished from those that
>>      simply involve a change of trunking modes (i.e. change
>>      server_owner minor id)
>>
>