Re: [nfsv4] AD Evaluation of draft-ietf-nfsv4-mv1-msns-update-03

[David Noveck <[e-mail]>]

> since I'm doing AD Evaluation on a 100-page document that updates a
600-page document, we can't be surprised that
> I had a few questions

Actually, I'm only surprised that it was done so quickly.   Thanks.

> please let me know how you'd like to proceed.
> I'll mark this as Revised ID Required for now.

I'll provide initial responses below.   If there turns out to be anything
controversial or difficult to resolve, I'll just note the issue now and
discuss it with my co-author.   Based on my initial scan of your comments,
my expectation is that a revised I-D could be provided in one to two weeks.

> This popped up from IDnits, and would be worth fixing before Last Call.

It seems that not fixing it is not really an option, much as I'd like to
sweep the whole thing under the rug :-(.

> -- The document seems to lack a disclaimer for pre-RFC5378 work, but may
>     have content which was first submitted before 10 November 2008.  If
you
>     have contacted all the original authors

I'm not clear about the set of people within "all the original authors".
 If this referring to the authors of RFC5661, then it is just contacting
Spencer S and Mike Eisler, in addition to myself.  On the other hand, some
material from RFC3530 may have migrated to RFC5661, which mkes the poblem
more difficult.

> and they are all willing to grant
>     the BCP78 rights to the IETF Trust, then this is fine, and you can
ignore
>     this comment.

I looked at BCP78 and it was not very clear what exactly I would be
agreeing to or asking other authors to agree to, beyond what one would
have agreed to before 11/10/2008.   BCP78 does  have a section entitled
"Changes since RFC 3968" but it isn't very helpful.

> If not, you may need to add the pre-RFC5378 disclaimer.
>     (See the Legal Provisions document at
>    https://trustee.ietf.org/license-info for more information.)

I looked in that document but I couldn't find the text of the needed
disclaimer.

> (This one tends to get Discusses during IESG Evaluation)

I think I can avoid that.

> The description of "trunking" appears (as best I can tell) appears 7
pages into the document. Is there a reasonable
> reference you could provide for the first use of the term in the
Introduction? (Even a forward reference to Section 3.1
> would be an improvement)

What I'm anticipating doing in -04 is prepending the following sentences to
the final paragraph of Section 1:

This document supplements facilities related to trunking, introduced in
[RFC5661].  For some important terminology regarding trunking, see Section
3.1.


>Does this text
>
> o  Some confusing text regarding changes in server_owner needs to be
>                                                          ^^^^^^^^^^^
>      clarified.
>
> mean

>  o  Some confusing text regarding changes in server_owner has been
>                                                           ^^^^^^^^
>      clarified.

>?

Yes.  Will be fixed in -04.

>Does this text

>  This document contains sections that propose additions to and other
>                                       ^^^^^^^
>   modifications of [RFC5661] as well as others that explain the reasons
>   for modifications but do not directly affect existing specifications.

> mean "specify additions", or "provide additions"? If approved, this
specification will no longer be proposing text changes > - they'll be in
effect.

Yes.  Will fix in -04.

> I can usually follow passive tense English in IETF specifications, but
>
>  When a file system is present and becomes absent, clients can be
>                                                            ^^^^^^
>   given the opportunity to have continued access to their data, using a
>
>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>    different replica.
>
> is just a bit too hand-wavish. Who/what is giving clients this
opportunity?

At one level, it is the protocol that provides the ability  to communicate
the new location.  At another level it is the source and destination
servers that co-operate to effect the necessary handoff.

I anticipate rewriting the sentence as follows in -04:

When a file system is present and becomes absent, the NFSv4 protocol
provides a means by which clients can be given the opportunity to have
continued access to their data, using a different replica.



>This is a pretty mundane point, but in text like this (and there are other
similar occurrences):
>
> 4.5.1.  New section to be added as the first sub-section of Section 11.4
>        of [RFC5661] to be entitled "Combining Multiple Uses in a Single
>       Attribute"
>
> it is not clear to me exactly where the added text goes - I'm guessing
that the added text becomes the new 11.4.1,
> pushing all other subsections of 11.4 down, but I'm not sure if that's
what was intended. I just noticed that >
https://www.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-mv0-trunking-update-04
shows an explicit renumbering ("this section is
> now 11.4.1") being added to similar insertions, which is likely more
helpful for the area review teams, reviewing ADs, the > RFC Editor, and the
eventual readers. That happened during a recent IESG Evaluation, so it's
especially likely to come > up for another NFSv4 draft ;-) ...

I can do something anaogous for this case by adding similar material to
Section 4.4 in -04.

For the othr cases, I can do something similar but I may have to create or
modify sections which describe reoganizations of various pieces of
RFC5661's Section 11.

> I'm lost in the "it"s in this text:
>
>  o  It may fetch the file system location attribute for the filesystem
>      which will provide either the name of the server (which can be
>      turned into a set of network addresses using DNS), or it will find
>      a set of server-trunkable location entries which can provide the
>      addresses specified by the server as desirable to use to access
>     the file system in question.
>
> I THINK this is (indentation is significant)

> o  The client may fetch the file system location attribute for the
filesystem
>      which will provide either
>
>         the name of the server (which can be
>          turned into a set of network addresses using DNS),
>
>     or the client will find a set of server-trunkable location entries
which can
>
>          provide the
>          addresses specified by the server as desirable to use to access
>         the file system in question.
>
> just following the text, but that's not making a lot of sense to me.

:-(

> Is "or it will find" garbled?

I don't think so.

> Or am I just lost here?

I don't think so, but since you feel lost, I'l propose a rewrite and see if
that clarifies things.

How about:

The client may fetch the file system location attribute for the
filesystem.  This will provide either:

   - The name of the server (which can be turned into a set of network
addresses using DNS).

   - A set of server-trunkable location entries specified by the server as
desirable to use to access the file system in      question.

>
>     or the client will find a set of server-trunkable location entries
which can    rewrite and see how that strikes you.

How about:


> It's a nit, but
>
>  Fs_locations_info provides a flag, FSLI4TF_RDMA flag.  indicating

>                                                     ^ should be a comma.

Yes.  Will fix in -04.

>   that RPC-over-RDMA support is available using the specified location
>   entry.

> I'm not entirely following
>
>  Fs_locations_info provides a flag, FSLI4TF_RDMA flag.  indicating
>   that RPC-over-RDMA support is available using the specified location
>   entry.

This is true.

>This flag makes it for a convenient for a client wishing to
> use RDMA, to establish a TCP connection and then convert to use of
>  RDMA.  After establishing a TCP connection, the step-up facility, can
>   be used, if available, to convert that connection to RDMA mode.

This would make it convenient, if server implemented it, but it turns out
they generally don't.   Still, they could do so.

> Otherwise, if RDMA availability is indicated, a new RDMA connection
>   can be established and it can be bound to the session already
>   established by the TCP connection, allowing the TCP connection to be
>   dropped and the session converted to further use in RDMA node.

> because I'm trying to guess why you would establish a TCP connection and
then convert that to RDMA mode,
> instead of establishing an RDMA connection in the first place.

Although you have been successful in guessing up to now, it is not
surprising that you can't guess this, as it depends on some of the history
of RDMA in the IETF.

> A quick spin through Section 18.16 of [RFC5661] didn't give me a lot of
guidance here (but I did look).

It doesn't explain why this was put in, but I'm guessing that, when it was
written, it seemrd not to require explanation.  At that time, when RDMA was
discussed, it was assumed you were referring to iWarp.  In the contect of
iWarp, stepping a connection from TCP to RDMA is natural, since you are
adding an RDMA facility to an existing TCP connection and there is no point
in tearing down the TCP connection and then establishing a new one.   Now,
given that use of IB and roCE are more common, Inerest in this has waned,
since non-iWarp implementations cannot use this fcility and iplementers may
not want to  invest in an iWarp-only facility.

>It's a nit, but
>
>  or locations of the file system can be determined by fetching the a
>   locations attribute.  attribute.
>
 >                        ^^^^^^^^^^ is probably a cut and paste error.

Probably.  In any case, it will be fixed in -04.

> It would be helpful to readers like me, if
>
> Generally, multi-server namespaces are for the most part uniform, in
>   that the same data made available to one client at a given location
>   in the namespace is made available to all clients at that location.
 >  However, as described above, there are facilities provided that allow
>
>           ^^^^^^^^^^^^^^^^^^
>   different clients to be directed different sets of data, to enable
>   adaptation to such client characteristics as CPU architecture.
>
> this cross-reference was a little more precise (is it "earlier in this
section", or "in Section M.N.O" - something like that).

Actually the reference is wrong rather than being (only) imprecise.  It
should be section 11.10.3 of [RFC5661].  Will fix in -04.

> This text,
>
>  o  Whenever a SEQUENCE operation is sent by a client to a server
>      which generated state held on that client which is associated with
>      a file system that is no longer accessible on the server at which
>      it was previously available, a lease-migrated indication, in the
>
 >                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 >     form the SEQ4_STATUS_LEASE_MOVED status bit being set, appears in
>
 >     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
 >     the response.
>
> wasn't easy for me to parse. Perhaps something like
>
>
>  o  Whenever a SEQUENCE operation is sent by a client to a server
>      which generated state held on that client which is associated with
 >     a file system that is no longer accessible on the server at which
>      it was previously available, the response will contain a
lease-migrated
>
 >
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
 >     indication, with the SEQ4_STATUS_LEASE_MOVED status bit being set.
>
 >     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>

Seems clearer.  Will pick up for -04



>  o  In the state merger case, it is possible that the server has not
>      attempted Transparent State Migration, in which case state may
>      have been lost without it being reflected in the SEQ4_STATUS bits.
>      To determine whether this has happened, the client can use
>      TEST_STATEID to check whether the stateids created on the source
>      server are still accessible on the destination server.  Once a
>      single stateid is found to have been successfully transferred, the
>      client can conclude that Transparent State Migration was begun and
>      any failure to transport all of the stateids will be reflected in
>      the SEQ4_STATUS bits.  Otherwise.  Transparent State Migration has
>
>                                     ^ should this be a comma?

It should.  Will fix in -04.

>      not occurred.


> but asking now, because there might be something else happening with this
text, if it's not a comma.


> I couldn't parse this paragraph. I think the problem is in the first
phrase:


 > o  If information about the set of clients that have locking state
>
>    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>     for the transferred file system, the destination server will be
>    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>     able to terminate the grace period once all such clients have
>      reclaimed their locks, allowing normal locking activity to resume
>      earlier than it would have otherwise.

It should be "If information about the set of clients that have locking
state for the transferred file system is made available".  Will fix in -04.

>It's a nit, but
>
>  o  For each lock type there will by type-specific information, such
>
>                                   ^ be

In -04, will be "for each lock type, there will be type-specfic
information,".

>      as share and deny modes for opens and type and byte ranges for
>      byte-range locks and layouts.


> If this matters,

  > o  If both EXCHANGE_ID requests were sent with RPCSEC_GSS ([RFC2203],
  >    [RFC5403], [RFC7861]) authentication and the server principal is
  >    the same for both targets, the equality of server scope is
  >    validated.  It is RECOMMENDED that two servers intending to share
 >
  >                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  >    the same server scope also share the same principal name.
 >
  >    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
>
> the reader would likely benefit from a sentence or two explaining why
this is RECOMMENDED, and/or what the
> consequences of not sharing the same principal name are.

Right now, I'm not sure.  This text was copied over from RFC5661.

> It's a nit, but
>
>  o  The cases of change which are very disruptive (e.g. change if
>
>                                            "if" is likely "of" ^

It will by "of" in -04.

>      server scope) are not sufficiently distinguished from those that
>      simply involve a change of trunking modes (i.e. change
>      server_owner minor id)


On Fri, Feb 1, 2019 at 8:55 PM Spencer Dawkins at IETF <
[e-mail]> wrote:

> Dear Author and Editor,
>
>
> The NFSv4 working group tends to produce solid documents - thank you for
> that - but since I'm doing AD Evaluation on a 100-page document that
> updates a 600-page document, we can't be surprised that I had a few
> questions ... please let me know how you'd like to proceed.
>
> I'll mark this as Revised ID Required for now.
>
> Thanks,
>
> Spencer
>
> This popped up from IDnits, and would be worth fixing before Last Call.
>
> -- The document seems to lack a disclaimer for pre-RFC5378 work, but may
>     have content which was first submitted before 10 November 2008.  If you
>     have contacted all the original authors and they are all willing to
> grant
>     the BCP78 rights to the IETF Trust, then this is fine, and you can
> ignore
>     this comment.  If not, you may need to add the pre-RFC5378 disclaimer.
>     (See the Legal Provisions document at
>     https://trustee.ietf.org/license-info for more information.)
>
> (This one tends to get Discusses during IESG Evaluation)
>
> The description of "trunking" appears (as best I can tell) appears 7 pages
> into the document. Is there a reasonable reference you could provide for
> the first use of the term in the Introduction? (Even a forward reference to
> Section 3.1 would be an improvement)
>
> Does this text
>
>  o Some confusing text regarding changes in server_owner needs to be
>
>                                                           ^^^^^^^^^^^
>      clarified.
>
> mean
>
>  o Some confusing text regarding changes in server_owner has been
>
>                                                           ^^^^^^^^
>      clarified.
>
> ?
>
> Does this text
>
>  This document contains sections that propose additions to and other
>
>                                       ^^^^^^^
>   modifications of [RFC5661] as well as others that explain the reasons
>   for modifications but do not directly affect existing specifications.
>
> mean "specify additions", or "provide additions"? If approved, this
> specification will no longer be proposing text changes - they'll be in
> effect.
>
> I can usually follow passive tense English in IETF specifications, but
>
>  When a file system is present and becomes absent, clients can be
>
>                                                            ^^^^^^
>   given the opportunity to have continued access to their data, using a
>
>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   different replica.
>
> is just a bit too hand-wavish. Who/what is giving clients this
> opportunity?
>
> This is a pretty mundane point, but in text like this (and there are other
> similar occurrences):
>
> 4.5.1.  New section to be added as the first sub-section of Section 11.4
>        of [RFC5661] to be entitled "Combining Multiple Uses in a Single
>        Attribute"
>
> it is not clear to me exactly where the added text goes - I'm guessing
> that the added text becomes the new 11.4.1, pushing all other subsections
> of 11.4 down, but I'm not sure if that's what was intended. I just noticed
> that
> https://www.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-mv0-trunking-update-04
> shows an explicit renumbering ("this section is now 11.4.1") being added to
> similar insertions, which is likely more helpful for the area review teams,
> reviewing ADs, the RFC Editor, and the eventual readers. That happened
> during a recent IESG Evaluation, so it's especially likely to come up for
> another NFSv4 draft ;-) ...
>
> I'm lost in the "it"s in this text:
>
>  o It may fetch the file system location attribute for the filesystem
>      which will provide either the name of the server (which can be
>      turned into a set of network addresses using DNS), or it will find
>      a set of server-trunkable location entries which can provide the
>      addresses specified by the server as desirable to use to access
>      the file system in question.
>
> I THINK this is (indentation is significant)
>
> o  The client may fetch the file system location attribute for the
> filesystem
>      which will provide either
>
>          the name of the server (which can be
>          turned into a set of network addresses using DNS),
>
> or the client will find a set of server-trunkable location entries which
> can
>
>          provide the
>          addresses specified by the server as desirable to use to access
>          the file system in question.
>
> just following the text, but that's not making a lot of sense to me. Is
> "or it will find" garbled? Or am I just lost here?
>
> It's a nit, but
>
>  Fs_locations_info provides a flag, FSLI4TF_RDMA flag.  indicating
>
>                                                      ^ should be a comma.
>   that RPC-over-RDMA support is available using the specified location
>   entry.
>
> I'm not entirely following
>
>  Fs_locations_info provides a flag, FSLI4TF_RDMA flag.  indicating
>   that RPC-over-RDMA support is available using the specified location
>   entry.  This flag makes it for a convenient for a client wishing to
>   use RDMA, to establish a TCP connection and then convert to use of
>   RDMA.  After establishing a TCP connection, the step-up facility, can
>   be used, if available, to convert that connection to RDMA mode.
>   Otherwise, if RDMA availability is indicated, a new RDMA connection
>   can be established and it can be bound to the session already
>   established by the TCP connection, allowing the TCP connection to be
>   dropped and the session converted to further use in RDMA node.
>
> because I'm trying to guess why you would establish a TCP connection and
> then convert that to RDMA mode, instead of establishing an RDMA connection
> in the first place. A quick spin through Section 18.16 of [RFC5661] didn't
> give me a lot of guidance here (but I did look).
>
> It's a nit, but
>
>  or locations of the file system can be determined by fetching the a
>   locations attribute.  attribute.
>
>                         ^^^^^^^^^^ is probably a cut and paste error.
>
> It would be helpful to readers like me, if
>
>  Generally, multi-server namespaces are for the most part uniform, in
>   that the same data made available to one client at a given location
>   in the namespace is made available to all clients at that location.
>   However, as described above, there are facilities provided that allow
>
>            ^^^^^^^^^^^^^^^^^^
>   different clients to be directed different sets of data, to enable
>   adaptation to such client characteristics as CPU architecture.
>
> this cross-reference was a little more precise (is it "earlier in this
> section", or "in Section M.N.O" - something like that).
>
> This text,
>
>  o Whenever a SEQUENCE operation is sent by a client to a server
>      which generated state held on that client which is associated with
>      a file system that is no longer accessible on the server at which
>      it was previously available, a lease-migrated indication, in the
>
>                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>      form the SEQ4_STATUS_LEASE_MOVED status bit being set, appears in
>
>      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
>      the response.
>
> wasn't easy for me to parse. Perhaps something like
>
>  o Whenever a SEQUENCE operation is sent by a client to a server
>      which generated state held on that client which is associated with
>      a file system that is no longer accessible on the server at which
>      it was previously available, the response will contain a
> lease-migrated
>
>
>                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
>      indication, with the SEQ4_STATUS_LEASE_MOVED status bit being set.
>
>      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> I'm guessing
>
>  o In the state merger case, it is possible that the server has not
>      attempted Transparent State Migration, in which case state may
>      have been lost without it being reflected in the SEQ4_STATUS bits.
>      To determine whether this has happened, the client can use
>      TEST_STATEID to check whether the stateids created on the source
>      server are still accessible on the destination server.  Once a
>      single stateid is found to have been successfully transferred, the
>      client can conclude that Transparent State Migration was begun and
>      any failure to transport all of the stateids will be reflected in
>      the SEQ4_STATUS bits.  Otherwise. Transparent State Migration has
>
>                                      ^ should this be a comma?
>      not occurred.
>
> but asking now, because there might be something else happening with this
> text, if it's not a comma.
>
> I couldn't parse this paragraph. I think the problem is in the first
> phrase:
>
>  o If information about the set of clients that have locking state
>
>     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>      for the transferred file system, the destination server will be
>
>      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>      able to terminate the grace period once all such clients have
>      reclaimed their locks, allowing normal locking activity to resume
>      earlier than it would have otherwise.
>
> It's a nit, but
>
>  o For each lock type there will by type-specific information, such
>
>                                    ^ be
>      as share and deny modes for opens and type and byte ranges for
>      byte-range locks and layouts.
>
> If this matters,
>
>  o If both EXCHANGE_ID requests were sent with RPCSEC_GSS ([RFC2203],
>      [RFC5403], [RFC7861]) authentication and the server principal is
>      the same for both targets, the equality of server scope is
>      validated.  It is RECOMMENDED that two servers intending to share
>
>                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>      the same server scope also share the same principal name.
>
>      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> the reader would likely benefit from a sentence or two explaining why this
> is RECOMMENDED, and/or what the consequences of not sharing the same
> principal name are.
>
> It's a nit, but
>
>  o The cases of change which are very disruptive (e.g. change if
>
>                                            "if" is likely "of" ^
>      server scope) are not sufficiently distinguished from those that
>      simply involve a change of trunking modes (i.e. change
>      server_owner minor id)
>