Re: [nfsv4] Progressing RFC errata for RFC 5661: Errata 2751

[Trond Myklebust <[e-mail]>]

At the time when this was raised to the IETF WG, we argued that it is
actually a clarification because the original RFC5661 does not specify
how to handle the different types of WRITE persistence.
IOW: it is trying to clarify an omitted description of client/server behaviour.

There is no text in RFC5661 that states what the behaviour of a pNFS
client should be when the server returns FILE_SYNC in response to a
WRITE (as NetApp servers always do). The DESCRIPTION in section
18.32.3 for a standard WRITE through the MDS states that means all
data+metadata has been persisted, but what does that even mean for
pNFS, where the data and metadata are handled by different servers?
You can't just port the description in 18.32.3, because it makes no
sense in the context of the different MDS/DS roles.

The errata presented by Ricardo therefore represents the consensus
interpretation for dealing with the question of what the different
WRITE modes mean for the 'files' pNFS type, and how they relate to
COMMIT and LAYOUTCOMMIT.

On Tue, 17 Sep 2019 at 04:41, Magnus Westerlund
<[e-mail]> wrote:
>
> Hi,
>
> Looking at Errata 2751:
> https://www.rfc-editor.org/errata_search.php?eid=2751
>
> I think it is fairly obvious that Approving this would not be a
> clarification of what RFC5661 descibes. It appears to me (with my lack
> of good understanding of NFS) to be a change of the consensus behavior
> that was established when RFC 5661 was published.
>
> That the WG consensus today may be to accept the text in to RFC 5661bis
> is not relevant and that the implementation follows this. The way to
> resolve this is to establish the consesus for implement these changes
> into RFC 5661bis and then publish it replacing RFC 5661.
>
> Cheers
>
> Magnus
>
>
>
> On Mon, 2019-09-16 at 23:50 +0200, Mkrtchyan, Tigran wrote:
> >
> > ----- Original Message -----
> > > From: "Dave Noveck" <[e-mail]>
> > > To: "Trond Myklebust" <[e-mail]>
> > > Cc: "Tigran Mkrtchyan" <[e-mail]>, "Rick Macklem" <
> > > [e-mail]>, "Magnus Westerlund"
> > > <[e-mail]>, "NFSv4" <[e-mail]>
> > > Sent: Monday, September 16, 2019 11:29:17 PM
> > > Subject: Re: [nfsv4] Progressing RFC errata for RFC 5661
> > > On Mon, Sep 16, 2019 at 4:26 PM Trond Myklebust <[e-mail]>
> > > wrote:
> > >
> > > >
> > > >
> > > > On Sun, 15 Sep 2019 at 09:44, David Noveck <[e-mail]>
> > > > wrote:
> > > >
> > > > >
> > > > > An additional issue that needs to be addressed before
> > > > > incorporating these
> > > > > changes is that it is not appropriate to add extensive material
> > > > > about
> > > > > handling of the file layout to Section 12.  That belongs in
> > > > > Section 13. We
> > > > > don't want to exacerbate the confusion that Tom has noted
> > > > > between
> > > > > statements about layouts in general and  about file
> > > > > layouts.  Rfc5661bis
> > > > > needs to do better in this respect.
> > > > >
> > > > > > RFC5661 (2751
> > > > > > <
> > > > > > https://protect2.fireeye.com/url?k=0f81077e-53130a61-0f8147e5-0cc47ad93db4-61f6126ac10042d2&q=1&u=https%3A%2F%2Fwww.rfc-editor.org%2Fverify_errata_select.php%3Feid%3D2751
> > > > > > >)
> > > > > >
> > > > > > GLOBAL
> > > > > >
> > > > > > Technical
> > > > > >
> > > > > > nfsv4 (tsv)
> > > > > >
> > > > > > Ricardo Labiaga
> > > > > >
> > > > >
> > > > > This is similar to the errata above although, because of the
> > > > > extensiveness of the changes proposed I think *Rejected* is
> > > > > more
> > > > > appropriate as an errata status.
> > > > >
> > > >
> > > > Rejecting this would break clients that have been encoded to rely
> > > > on that
> > > > behaviour (i.e the Linux client, for one).
> > > >
> > >
> > > Rejecting the errata (i.e. putting it in the state "Rejected")
> > > would not do
> > > that.
> > >
> > > What would do that is doing an rfc5661bis that does not incorporate
> > > these
> > > changes, but I don't think the working group is going to do that,
> > > for the
> > > reasons you state.
> > >
> > >
> > > > Without it, we would be unable to determine whether or not a
> > > > LAYOUTCOMMIT
> > > > is required for the files pNFS layout type, and would have to
> > > > default to
> > > > assuming it does. That would put the existing client base in
> > > > violation.
> > > >
> > >
> > > It appears that the existing client base is in violation of
> > > rfc5661.   I
> > > don't think we want rfc5661bis to be the same, so I expect the
> > > working
> > > group will decide to make this sort of change.
> > >
> > > >
> > > >
> > >
> > > It would also force us to start sending LAYOUTCOMMIT to servers
> > > that
> > > > currently do not require it (i.e. NetApp).
> > > >
> > > > I'm not aware of any pNFS files server implementation that is in
> > > > violation
> > > > of this errata and that is expecting a LAYOUTCOMMIT on a commit-
> > > > through-MDS
> > > > implementation, or following a FILE_SYNC return from a commit-
> > > > through-DS
> > > > implementation. Ccing Rick and Tigran for their input.
> > > >
> > >
> > > If there are such server implementations, people will be able to
> > > tell us
> > > about them, when we review the rfc5661bis section replacing Section
> > > 13 of
> > > rfc5661.
> >
> > If I read suggested errata correctly, then our server does exactly
> > what is suggested:
> >
> > We do not set NFL4_UFLG_COMMIT_THRU_MDS flag and return data server
> > returns
> > DATA_SYNC4 on WRITE, with expectation is that client will send a
> > LAYOUTCOMMIT.
> >
> >
> > >
> > > The basic issue is that errata are for correcting situations in
> > > which an
> > > RFC does not correctly represent the decisions of the working
> > > group.  I
> > > believe that, in this case, the working group wants to change its
> > > decision,
> > > for the good reasons you have indicated.   By saying that the
> > > errata should
> > > be put in status "Rejected", I do not mean to imply that these
> > > changes
> > > should not go into rfc5661bis.   I'm only saying that, according to
> > > the
> > > IETF rules that Magnus pointed us to, this is the correct
> > > status.   Magnus
> > > will make the final decision on errata status.   Regarding the
> > > incorporation of these changes into rfc5661bis, the working group
> > > will make
> > > a decision which the editor will then try to sell to the IESG,
> > > using the
> > > kind of arguments that you have made above.   The IESG does not
> > > want the
> > > spec to diverge from exsting implementations, so I don't expect a
> > > lot of
> > > pushback, once the situation is made clear to them.
> >
> >
> > This diverge is caused by 'on the field' experience, which covered by
> > rfc 5661
> > not 100% precise.
> >
> > Thanks,
> >    Tigran.
> --
> Cheers
>
> Magnus Westerlund
>
>
> ----------------------------------------------------------------------
> Network Architecture & Protocols, Ericsson Research
> ----------------------------------------------------------------------
> Ericsson AB                 | Phone  [phone]
> Torshamnsgatan 23           | Mobile [phone]
> SE-164 80 Stockholm, Sweden | mailto: [e-mail]
> ----------------------------------------------------------------------
>