Re: [nfsv4] I-D Action: draft-ietf-nfsv4-flex-files-10.txt

[Thomas Haynes <[e-mail]>]

> On Jul 18, 2017, at 12:39 AM, Rick Macklem <[e-mail]> wrote:
> 
> First off. I'd like to thank both Thomas and David for their work on this. I am happy to see
> active work on it.
> 
> 1 - I think I mentioned this, but I no longer care about the stuff I posted last year.
>      (Basically the case where a server might only require one mirror to be written and
>       how quickly changes to a mirror were to be propagated. Others may still feel that
>       this is useful, but it no longer matters to me;-)
> 
> I am thinking of posting one issue at a time, since they probably require discussion.
> (If you'd prefer one post with everything in it, just let me know and I'll hold off on
> further individual posts and make one "collective" post.)
> Having said the above, here's the first one:
> 
>> 2.1.  LAYOUTCOMMIT
>> 
>>  When tightly coupled storage devices are used, the metadata server
>>  has the responsibility, upon receiving a LAYOUTCOMMIT (see
>>  Section 18.42 of [RFC5661]), of ensuring that the semantics of pNFS
>>  are respected (see Section 12.5.4 of [RFC5661]).  These do not
>>  include a requirement that data written to data storage device be
>>  stable upon completion of the LAYOUTCOMMIT.
> 
> - The above sentence seems to imply that the client is responsible to do COMMITs

The client is always responsible for doing COMMITs.

In a loosely coupled storage device, the client is responsible for doing the COMMIT because it has the knowledge of which write to commit.

With a tightly coupled storage device, if the client sends the COMMIT to the MDS, the MDS can ensure through the control protocol that the commit happens.

Actually, that is wrong, RFC 5661 states in Section 12.5.4 (on page 289): 

   The data should be written and
   committed to the appropriate storage devices before the LAYOUTCOMMIT
   occurs. 

And that completely invalidates the last sentence of the first paragraph of Section 2.1.

(see below)


>   and seems to contradict the following para., which suggests that the COMMITs
>   are required for "loose coupling" only.
> 
>> In the case of loosely coupled storage devices, it is the
>> responsibility of the client to make sure the data file is stable
>> before the metadata server begins to query the storage devices about
>> the changes to the file.  If any WRITE to a storage device did not
>> result with stable_how equal to FILE_SYNC, a LAYOUTCOMMIT to the
>> metadata server MUST be preceded by a COMMIT to the storage devices
>> written to.  Note that if the client has not done a COMMIT to the
>>  storage device, then the LAYOUTCOMMIT might not be synchronized to
>>  the last WRITE operation to the storage device.
> - I think it would be simpler for a client implementor (like me;-) to just apply
>  this para. to both loose and tight coupling (ie. just drop the part of the first
>  sentence up to the "," and say "It is the responsibility of the client..”


It is safe to say/implement that the client does the COMMIT in either scenario.

Taking both of these points into consideration, I’ve rewritten the section as:

2.1.  LAYOUTCOMMIT

   The metadata server has the responsibility, upon receiving a
   LAYOUTCOMMIT (see Section 18.42 of [RFC5661]), of ensuring that the
   semantics of pNFS are respected (see Section 12.5.4 of [RFC5661]).
   These do include a requirement that data written to data storage
   device be stable before the occurance of the LAYOUTCOMMIT.

   It is the responsibility of the client to make sure the data file is
   stable before the metadata server begins to query the storage devices
   about the changes to the file.  If any WRITE to a storage device did
   not result with stable_how equal to FILE_SYNC, a LAYOUTCOMMIT to the
   metadata server MUST be preceded by a COMMIT to the storage devices
   written to.  Note that if the client has not done a COMMIT to the
   storage device, then the LAYOUTCOMMIT might not be synchronized to
   the last WRITE operation to the storage device.

I.e., no mention of the model as they both have to obey the same requirements.


> OR
> - Make it conditional on a flag returned by the MDS.
> 
> In general, most of the "tight" vs "loose" coupling seems to be confusing to me.
> I can see that there may be the two cases w.r.t. security:
> 1 - Use the synthetic uid/gid and AUTH_SYS only.
> OR
> 2 - Use the same model as the MDS (same uid/gid/authenticator) and let the
>     MDS<->DS figure out how to implement it.
> Again, I think a flag returned by the MDS to indicate which security model is being
> used would be simpler than the "tight" vs "loose" concept. (The document can mention
> that some control protocol may be needed for the server implementation, but the
> client only cares what it needs to do and not how the server chooses to implement it.)


The concepts are presented to highlight the difficulties that a server developer will face.

I don’t think the client has to be aware of whether it is loosely or rightly coupled.

Consider a NFSv3 storage device made by X. A MDS made by Y can use NFSv3 as
 the control protocol. It could CREATE to instantiate the data files, it could SETATTR
the mode bits/uid/gid to fence, it can GETATTR to determine the change attribute,
the mtime, etc.

But if Y can recognize X and X provides some management APIs, then Y might be
able to leverage those APIs as an additional control protocol.

And as a client implementor, you are completely unaware of all of that.



> 
> I just find the "loose" vs "tight" coupling a somewhat confusing "catch all" and I think
> a few flags returned by the MDS to the client will clarify what the client needs to do,
> but this is just a suggestion and I'm happy with anything that works. (I think this also
> applies to Data Server vs Storage Server. To be honest, I'd just get rid of all mention
> of these and just call them all data servers?)
> 

In practice, they are the same. In the original 3 standard documents, they are
treated a bit differently. A data server has an OS, a storage device may or may not
have an OS.


> Again, thanks for working on this, rick
> 

And thanks for making me think!