Re: [nfsv4] New Version Notification for draft-haynes-nfsv4-layoutwcc-01.txt

[Rick Macklem <[e-mail]>]

On Fri, Mar 17, 2023 at 4:35 PM Thomas Haynes <[e-mail]> wrote:
>
>
>
> On Mar 17, 2023, at 4:19 PM, Rick Macklem <[e-mail]> wrote:
>
> Here are a few generic comments...
> As it stands, I could not implement this in a way that I would think
> would be useful.
>
> - There is no indication when the client should issue this operation to
>  the MDS. I'd suggest something like "whenever the client expects attributes
>  related to writing is up-to-date for a Getattr against the MDS" or something
>  like that.
>
>
>
> This is a fair ask, I’ll add wording to that effect.
>
>
>
>  --> As a client implementor, I need to know when to do this..
>
> Which attributes? It doesn't seem to get specific about what attributes.
> I'd suggest a specific list. Possibly:
> Size
> Change
> TimeModify
> SpaceUsed
> and maybe TimeAccess?
>
>
> Again, a good point. I’ll add text for this as well.
>
>
>
> Change is going to be problematic for NFSv3 DSs.
> Maybe a mechanism similar to what is defined for handling
> Change when a client holds a Write delegation can be used?
>
> When I did the pNFS flexfile layout client, I did a LayoutCommit
> whenever the client needs up-to-date attrbute values, because I
> thought that was what LayoutCommit would be useful for.
> --> If the client does this, the pNFS flexfiles layout server only needs
>      to get up-to-date attributes from the DS at that point in time and
>      can cache them in the MDS otherwise, unless there are Getattr
>      requests from client(s) that do not do pNFS, such as NFSv3/v4.0
>      clients.
> --> I discovered that the Linux client does not do LayoutCommit for
>      some cases where it expects the above attributes to be up-to-date,
>      so I was forced to query the DS whenever the MDS receives a
>      Getattr and any RW layout for the file has been given out.
>
>
> This is a GETATTR which asks for size, space_used, or one of the times, right?
Yes. The FreeBSD pNFS server keeps a copy of the above attributes in the MDS
and updates them when a Getattr for them (or any NFSv3 Getattr) shows up.

>
> Also, what do you do during grace recovery? Are you keeping track of open files across restarts?
>
> Or when you see the client doing an OPEN previous, do you refresh your attrs?
>
Nothing, but you are right, I probably should be updating the
attributes on the MDS during
restart.
However, I just (re)read Sec. 12.7.4 and 18.42 of RFC8881 and I don't see why
LayoutCommit is not sufficient for this? (It takes a "hint" of new
file size and modify
time in the arguments and when I read it, it seems that the pNFS
client should only
expect these file attributes to be up-to-date after doing it.

Also, if there was a Flex files specific lou_body<> for layoutupdate4,
a LayoutCommit
done during grace could report write errors. (That might be simpler
than your suggested
changes to LayoutReturn?)

If the client follows the advice in 12.7.4 to do a LayoutCommit during
grace, that would
update the attributes on the MDS if the client has done writing to DS(s).
Btw, only updating attributes on the MDS upon LayoutCommit reduces the MDS<->DS
attribute acquisitions (FreeBSD uses a MDS->DS Getattr RPC) significantly.
As I noted, I changed this because it didn't work for the Linux client.

rick

>
>
>      (The description of when to do LayoutCommit is vague in the RFCs.
>        I recall the Linux client doesn't do a LayoutCommit unless it needs
>        to also do a Commit to the DS.)
>
> My point is, this situation can be improved if there is a well defined
> "if the client expects up-to-date attributes, it must do this".
>
> In general, for this to be useful for me to implement (client and server),
> I need more specifics w.r.t. what attributes, when the Layout_wcc must
> be done.
>
>
> Okay, I can’t do a new version for two weeks because of the quiet period with the IETF.
>
> Thanks for the review!
>
>
>
> rick
>
> ----- Original Message -----
>
> From: "Thomas Haynes" <[e-mail]>
> To: "NFSv4" <[e-mail]>
> Sent: Monday, 20 February, 2023 16:48:45
> Subject: [nfsv4] Fwd: New Version Notification for draft-haynes-nfsv4-layoutwcc-01.txt
>
>
> This version addresses the issue that the draft is NFSv3 Flex Files only.
>
> Thanks,
> Tom
>
> Begin forwarded message:
>
> From: [e-mail]
> Subject: New Version Notification for draft-haynes-nfsv4-layoutwcc-01.txt
> Date: February 20, 2023 at 7:46:54 AM PST
> To: "Thomas Haynes" <[e-mail]>, "Trond Myklebust"
> <[e-mail]>
>
>
> A new version of I-D, draft-haynes-nfsv4-layoutwcc-01.txt
> has been successfully submitted by Thomas Haynes and posted to the
> IETF repository.
>
> Name: draft-haynes-nfsv4-layoutwcc
> Revision: 01
> Title: Add LAYOUT_WCC to NFSv4.2
> Document date: 2023-02-20
> Group: Individual Submission
> Pages: 9
> URL:
> https://www.ietf.org/archive/id/draft-haynes-nfsv4-layoutwcc-01.txt
> Status:         https://datatracker.ietf.org/doc/draft-haynes-nfsv4-layoutwcc/
> Html:
> https://www.ietf.org/archive/id/draft-haynes-nfsv4-layoutwcc-01.html
> Htmlized:
> https://datatracker.ietf.org/doc/html/draft-haynes-nfsv4-layoutwcc
> Diff:
> https://author-tools.ietf.org/iddiff?url2=draft-haynes-nfsv4-layoutwcc-01
>
> Abstract:
> The Parallel Network File System (pNFS) Flexible File Layout allows
> for a file's metadata (MDS) and data (DS) to be on different servers.
> It does not provide a mechanism for the data server to update the
> metadata server of changes to the data part of the file.  The client
> has knowledge of such updates, but lacks the ability to update the
> metadata server.  This document presents a refinement to RFC8434 to
> allow the client to update the metadata server to changes on the data
> server.
>
>
>
>
> The IETF Secretariat
>
>
>
>
> _______________________________________________
> nfsv4 mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/nfsv4
>
> _______________________________________________
> nfsv4 mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/nfsv4
>
>
> _______________________________________________
> nfsv4 mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/nfsv4
>
>