Re: [nfsv4] Progressing RFC errata for RFC 5661

[David Noveck <[e-mail]>]

>>https://mailarchive.ietf.org/arch/msg/nfsv4/_KTtO6uz-MvRoStbhPuOXWZr6yI
>This actually appears to be a discussion related to the offset and length
arguments for LayoutCommit

True.   It appears that this change (essentially eliminating the use of
these arguments) was turned into *5* Held-over errata that just sat there
for over eight years, with not very many people knowing about them and
rfc5661 left describing a protocol thatsome people in the working group
felt to have been superseded.

The case of the meaning of  FILE_SYNC4 in pnfs-file was similar,
except that there was only one errata and it was left in Reported state.

In both cases, we will need information on the rationale for the change,
since the errata mechanism is not designed to record rationales. It appears
that it was presumed that changes big enough to require rationales are too
big for the errata mechanism.   The need for such rationales is not only to
satisfy your curiosity (and mine; it turns out I was in the hospital
while much of this discussion was going on).   These changes will have to
go into rfc5661bis which will need a "changes since rfc5661" section, which
will need, for each of these changes, a short paragraph explaining why the
change was made.

Even more important, if there are other errata that have already been
essentially approved by the working group, as the six we are talking about
have been, we need to hear about them right away, so they can be taken
account of as we develop plans to actually write and publish rfc5661bis.
We need to do this as soon as we can.   Given what we have just found out,
it appears that we are already eight years late in getting started on this
effort.

On Thu, Sep 19, 2019 at 12:07 AM Rick Macklem <[e-mail]> wrote:

> Trond Myklebust wrote:
> >Rick,
> >
> >That errata predates most of the Linux pNFS client implementation. We
> >wrote the implementation to conform to the errata.
> >
> >So no. It's not a bug. It's a deliberate design based on a decision
> >that was discussed in the IETF WG, on the mailing list
> >
> >https://mailarchive.ietf.org/arch/msg/nfsv4/_KTtO6uz-MvRoStbhPuOXWZr6yI
> This actually appears to be a discussion related to the offset and length
> arguments for LayoutCommit, but...
> >
> >and in a special session of the IETF:
> >
> >https://mailarchive.ietf.org/arch/msg/nfsv4/Rpw9XCwCARxfaU4ym5L2TauV6ao
> Ok. This was long before I got around to implementing it, so I wouldn't
> have
> understood the implications.
> --> I would have been interested in hearing the rationale behind not doing
>       LayoutCommit for FILE_SYNC4 writes, since it seems to me that
> RFC-5661
>       had gotten it right when it required them.
>
> As I said, the FreeBSD server can handle this case, it just results in a
> lot of
> overhead synchronizing Size, Change, Time_Modify between MDS and DS
> whenever a RW layout is issued to a client for the file.
>
> Thanks for pointing this out, rick
>
> On Wed, 18 Sep 2019 at 12:39, Rick Macklem <[e-mail]> wrote:
> >
> > Mkrtchyan, Tigran wrote:
> > [stuff snipped]
> > >Hi Rick,
> > >
> > >here is the public link to errata
> > >
> > >https://www.rfc-editor.org/errata/eid2751
> > >
> > >Tigran.
> > Thanks Tigran.
> >
> > Ok, so now that I've read it I have to admit I think it is rewriting the
> RFC
> > to conform with what the Linux client does.
> >
> > I think this para. from Sec. 13.10 of RFC-5661 is clear:
> >    The NFSv4.1 protocol only provides close-to-open file data cache
> >    semantics; meaning that when the file is closed, all modified data is
> >    written to the server.  When a subsequent OPEN of the file is done,
> >    the change attribute is inspected for a difference from a cached
> >    value for the change attribute.  For the case above, this means that
> >    a LAYOUTCOMMIT will be done at close (along with the data WRITEs) and
> >    will update the file's size and change attribute.  Access from
> >    another client after that point will result in the appropriate size
> >    being returned.
> >
> > It states "will be done". It doesn't say anything about UNSTABLE4 vs
> FILE_SYNC4.
> > (I think most POSIX-like clients would consider the fsync(2) syscall to
> require
> >  the same treatment as "close" above, but that is a POSIX-specific
> client issue.)
> > I can see the argument that, since there is no "must" in the statement,
> that a
> > client can choose not to do this, but that would also imply that the
> client will
> > need to live with the consequences of it.
> >
> > I think the second sentence of the first para. of the errata is bogus:
> > For file layouts, WRITEs to a Data Server that return a stable_how4
> value of
> > FILE_SYNC4 guarantee that data and file system metadata are on stable
> > storage.  This means that a LAYOUTCOMMIT is not needed in order to make
> the
> > data and metadata visible to the metadata server and other clients.
> >
> > Why?
> > The FILE_SYNC4 was returned by the DS. This would imply the DS
> > has committed data and metadata to stable storage on the DS.
> > However, I am not aware of anything in RFC-5661 that would imply that
> > the Size, Time_Modify and Change attributes or anything else must have
> > been updated or in stable storage on the MDS at this time.
> >
> > If a server does not require LayoutCommit operations for correct
> behaviour
> > then it can simply reply NFS4ERR_NOTSUPP (as I believe the Netapp filer
> > does) and the client then no longer needs to do them.
> >
> > If there is somewhere in RFC-5661 that it is stated that LayoutCommits
> are
> > not required when the DS replies FILE_SYNC4, then I missed it and there
> > is a problem with RFC-5661 that needs to be addressed.
> >
> > Otherwise, sorry, but it seems that the bug is in the Linux client
> implementation
> > and not RFC-5661.
> >
> > Is there a File Layout pNFS server implementation where the DSs return
> > FILE_SYNC4 that will break if the client does a LayoutCommit for this
> case?
> > (If so, then something may need to be done.)
> >
> > rick
> >
> >
> >
>