Re: [nfsv4] Progressing RFC errata for RFC 5661

[Rick Macklem <[e-mail]>]

Trond Myklebust wrote:
>On Sun, 15 Sep 2019 at 09:44, David Noveck ><[e-mail]<mailto:[e-mail]>> wrote:
>>
>>An additional issue that needs to be addressed before incorporating these >>changes is that it is not appropriate to add extensive material about handling >>the file layout to Section 12.  That belongs in Section 13. We don't want to >>exacerbate the confusion that Tom has noted between statements about >>layouts in general and  about file layouts.  Rfc5661bis needs to do better in >>respect.
>>RFC5661 (2751<https://www.rfc-editor.org/verify_errata_select.php?eid=2751>)
>>GLOBAL
>>Technical
>>nfsv4 (tsv)
>>Ricardo Labiaga
>>
>>This is similar to the errata above although, because of the extensiveness of the 
>>changes proposed I think Rejected is more appropriate as an errata status.
>
>Rejecting this would break clients that have been encoded to rely on that >behaviour (i.e the Linux client, for one). Without it, we would be unable to >determine whether or not a LAYOUTCOMMIT is required for the files pNFS >layout type, and would have to default to assuming it does. That would put the >existing client base in violation. It would also force us to start sending >LAYOUTCOMMIT to servers that currently do not require it (i.e. NetApp).
Just fyi, this is what the FreeBSD client does. It always sends a Layoutcommit
to the MDS whenever an fsync(2) or similar is done and the client holds
a write layout for the file, unless the server replies NFS4ERR_NOTSUPP to Layoutcommit. (I think that the Netapp Filer replies NFS4ERR_NOTSUPP to Layoutcommit, which was why it disables doing Layoutcommits on a mount
once it sees a NFS4ERR_NOTSUPP reply.)
--> Layoutcommits are done even if the DS replies FILE_SYNC and/or the MDS
       sets commit-through-MDS.

>I'm not aware of any pNFS files server implementation that is in violation of this >errata and that is expecting a LAYOUTCOMMIT on a commit-through-MDS >implementation, or following a FILE_SYNC return from a commit-through-DS >implementation. Ccing Rick and Tigran for their input.
Well, when I first wrote the pNFS server, I assumed that the Layoutcommit
would be done and used it to synchronize the FileSize, Change attributes
between the DS and the ones cached in the MDS.

Then, I found out this didn't work for the Linux client, so I added a tunable
sysctl called vfs.nfsd.pnfsgetdsattr which tells the server to not cache
FileSize, Change when a write layout is issued to a client for the file.
As such, when this tunable is set (and that is the default), it works for the
Linux client, but is less efficient, since it must acquire FileSize, Change from
the DS whenever a Getattr requests them, if a write layout has been issued
for the file.
--> The FreeBSD server can be run either assuming the Layoutcommit is always
       done, or if the clients don't do them. In the latter case, it should work
       correctly even if a client never does a Layoutcommit.

I have not seen the wording of the actual errata (the links in these posts
require a login I don't have and I probably missed it when it was emailed),
but I am assuming that it says Layoutcommits should be done by the client
when the DS replies Unstable to writes and the MDS requires
commit-through-DS.

rick