Re: [nfsv4] (belated) copy of my review of draft-ietf-nfsv4-layout-types-03

[Thomas Haynes <[e-mail]>]

Hi Dave,

I’ve evidently already applied some of these in my draft-05 working
copy. As such, I’ll either note I’ve made the change at this time or
ack that I made it at some earlier point.

I’m going to split my replies to avoid getting notified by the mailer
that mu message is too large.

Here is part 1:

On Jul 16, 2015, at 4:21 PM, David Noveck <[e-mail]<mailto:[e-mail]>> wrote:

I sent the comments below to Tom a while back.  This was the review that Tom was referring to at the Dallas meeting.  Although Tom characterized my comments as a "good review", I don't think one can assume that he necessarily agrees with everything in these comments.  Still, it would be good if the working group had access to these comments before discussing next steps for draft-ietf-nfsv4-layout-types.

_______________

Overall Comments

Overall this document has made a good start in addressing the problems it set out to solve.  I'd like to thank you for undertaking what might have seemed a thankless task.  It's now almost thankless but I hope it won't remain so.

I think the problem it deals with is more difficult than I expected it to be.  I hadn't recognized the complexity that derives from:

  *   The fact that we have three components to provide interoperability among, rather than just two.
  *   The fact that we are trying to accommodate such different protocols within the same framework.
  *   The fact that locking and permissions are similar enough that they need to be treated together but different enough that doing so is not easy.

There are a couple of issues that I think would be best addressed relatively soon::

  *   The fact that the phrase "control protocol" is  used to mean a number of different things (i.e, requirements, and also the mechanism to satisfy them) makes things hard to understand.  Phrases like "no control protocol" and "real control protocol" compound the difficulty.
  *   Re-organization of section 3.

As noted above, I think the material in section 3 needs to be re-organized.   Much of it is not really related, or related only tangentially, to the control protocol, at least as I understand the term.  One possible classification of the material is as follows:

  *   Semantic requirements that apply to the protocol as a whole.
  *   Issues that relate to the interaction of the Metadata sever and the data storage devices.  This covers the control program requirements proper, or at least that part of them that is layout-type-specific.
  *   Issues that relate to the interaction of clients and the data storage devices.  This includes the need to clearly define the data storage protocol and to specify who is responsible for making sure that IO requests conform to valid layouts.
  *   Issues that relate to the interaction between the clients and the metadata server.

For all these, it is necessary to distinguish among:

  *   REQUIREMENTS specified in this document that apply directly to pNFS implementations.
  *   requirements for the document defining a layout type to clearly explain/define certain things.
  *   guidance regarding the need for layout type definition documents to impose appropriate REQUIREMENTS on implementations to ensure interoperability.

Comments by Section

Abstract

Suggest replacing the first sentence by the following two sentences:


This document defines the requirements which individual pNFS layout types need to meet in order to work within the parallel NFS (pNFS) framework as defined in RFC5661.  In so doing, it aims to more clearly distinguish between requirements for pNFS as a whole  and those those specifically directed to the pNFS File Layout.


Ack


In the last sentence suggest adding "in this regard" after "RFC5661".

Introduction

In the second paragraph suggest the following changes:

  *   In the first sentence, replace "being strictly for" by "applying only to”.

Ack



  *   In the second sentence, replace "I.e., since" by "Because" and "is some" by "has been some”.

Either done earlier by me or replaced.

In the second sentence of the third paragraph, suggest replacing "clarifies what are" by "specifies”.


Ditto.


Definitions

I'd like to offer the following suggestions for modification/clarification of the definitions. There are also proposed additions. In the case of proposed new definitions, the term to be be defined is underlined

We currently have the issue that it is almost impossible not to use terms in the definitions before they themselves are defined. Like you, I've abused to a degree the principle of alphabetical order so as to limit the difficulties. In addition, given the centrality of the concept of "Layout Type" . I'm proposing bringing it forward in an introductory paragraphs such as the following.

The concept of layout type has a central role in the definition and implementation of parallel NFS. Clients and servers implementing different layout types behave differently in many ways while conforming to the overall parallel NFS framework defined in [RFC5661] and this document. Different layout types may differ in:

  *   The method used to do IO operations directed to data storage devices.
  *   The requirements for communication between the Metadata server and the data storage devices.
  *   the means used to ensure that IO requests are only processed when the client holds an appropriate layout.
  *   The format and interpretation of nominally opaque data fields in pNFS-related NFSv4.x data structures.

Such matters are defined in a standards-track layout type specification. Except for the files layout type, which was defined in chapter 13 of [RFC5661], existing layout types are defined in their own RFC's and it is anticipated that new layout type will be defined in similar documents.


Evidently I added this as an introduction section.



It may be that we'll need a new concept (e.g. "layout-subtype" or "layout type variant") to deal with cases in which a layout type has variant classes of implementations that differs in some but not all of the items listed above. For example, loose and tight coupling variants of flex-files might fit this model and we should consider whether the same would apply for virtualized and non-virtualized variants of the block layout type.

  *   Control communication requirements: I think it is helpful here to separate requirements and mechanism. So i'm proposing:

For a particular layout type, defines the details regarding information (e.g layouts, stateids, file metadata, and file data) which must be  communicated between the metadata server and the storage devices.

  *   Control protocol:

Defines a particular mechanism that an implementation of a layout type would use to meet the control communication requirement for that layout type. This need not be a protocol as normally understood. In some cases the same protocol my be used as a control protocol and data access protocol.

  *   (file) data: suggest the following as a replacement

is that part of the file system object that contains the data to be read or written,  as opposed to attributes of the object .  That is, it is the file contents.

  *   Data server (DS):  I have a suggestion (below) designed to resolve the confusion and apparent self-contradiction regarding this term:

is one of the pNFS servers which provide the contents of a file system object which is a regular file, when the file system object is accessed over a file-based protocol.  Note that this usage differs from that in [RFC5661] which applies the term in some cases even when other sorts of protocols are being used. Depending on the layout, there might be one or more data servers over which the data is striped.  Note that while the metadata server is strictly accessed over the NFSv4.1 protocol, depending on the Layout Type, the data server could be accessed via any file access  protocol that meets the pNFS requirements.


See section 2.1 for a comparison of this term and"data storage device”.


Evidently I had made the above edits and not the below:


  *   fencing:  Suggest replacing "when" by "process by which”

Ack


  *   layout: suggest the following replacement:

contains information a client uses to access file data on pNFS storage device .  This

We can’t say pNFS storage device. I know what you mean here, but I can read it another way.


will include specification of the protocol (layout type) and the identity of the storage devices to be used.



The bulk of the contents of the layout are defined in [RFC5662] as nominally opaque, but individual layout types may specify their own interpretation of layout data.


Edited in...


  *   layout iomode: Suggest replacing "read or" by "read-only access or”

Ack


  *   layout stateid: Suggest replacing "the difference between a layout stateid and a normal stateid" by "differences in handling between layout stateids and other stateids”.

Ack

  *   Layout Type: propose removing this in favor of the introductory paragraph suggested above

I’ve added a pointer because:

A) We should have something here
B) I don’t want to repeat it verbatim.



  *   (file) metadata: Suggest the follow replacement:

is that part of the file system object that contains various descriptive data relevant to the file object, as opposed to the file data itself.  This could include the tie of last modification, access time, eof position, etc.

Ack


  *   Metadata server (MDS):

     *   In the second sentence, suggest replacing "generating" by "generating, recalling, and revoking"
     *   Suggest deleting the third sentence and appending the following material to the end of the second sentence

, for performing directory operations, and for performing I/O operations to regular files when the clients direct these to the MDS itself.

Ack


  *   recalling a layout:
     *   in the first sentence, suggest replacing "is" by "occurs"
     *   also in the first sentence, suggest replacing "uses a back channel" by "issues a callback”.

Ack

And do we need to define a callback?


  *   revoking a layout:  suggest the following replacement:

occurs when the metadata server invalidates a specific layout  Once revocation occurs, the metadata server will not accept as valid any reference to the revoked layout and the data storage device will not accept any client access based on the layout.

Ack

  *   stateid:  suggest the following replacement:

is a 128-bit quantity returned by a server that uniquely defines the some set of locking-related state provided by the server. Stateids may designate state related to open files, to byte-range locks, to delegations, or to layouts.

Ack


  *   storage device: suggest the following as a replacement

Designates the target to which clients may direct IO requests when they hold an appropriate layout. Note that each data server is a data storage device but that some data storage device are not data servers. See section 2.1 for further discussion.

Ack


  *   storage protocol:

is the protocol used by clients to do IO operations to the data storage device  Each layout type may specify  its own data access protocol.  It is possible for a layout type to specify multiple data access protocols

Ack


Section 2.1

First of all, suggest as a new title for this section: Use of the Terms "Data Server" and "Data storage Device"

Proposed new content for this section is below:

In [RFC5661]], these two terms are used somewhat inconsistently:

  *   In chapter 12, where pNFS in general is discussed, the term "data storage device" is used.
  *   In chapter 13, there the file layout type is discussed, the term "data server" is used.
  *   In other chapters, the term data "server: is used, even in contexts where the storage access type is not NFSv4.1 or any other file access protocol.

As this document deals with pNFS in general, it uses the more generic term "storage device" in preference to "data server".  The term "data server" is used only in contexts in which a file server is used as a data storage device.  Note that every data server is a storage device but that storage devices which use protocols which are not file access protocol are not data servers.

Since a given data storage device may support multiple layout types, a given device can potentially act as a data server for some set of storage protocols while simultaneously acting as a non-data-server data storage device for others.

And edited in...



Requirements Language

This document poses special issues regarding the RFC2119 terms in that it states requirements/recommendations that apply both to implementations and to future specifications.   My preference is to apply them only to implementations but I will follow your existing practice in my comments which follow.  Still, whichever way you choose to go on this particular issue, this section should say something about the question.

Thinking cap on