Re: [nfsv4] NFSv4.2 client does Seek to pNFS DS or MDS for File Layout?

[Rick Macklem <[e-mail]>]

Trond Myklebust wrote:
>On Aug 26, 2019, at 22:13, Rick Macklem ><[e-mail]<mailto:[e-mail]>> wrote:
>
>David Noveck wrote:
>rmacklem wrote:
>In reading RFC-7862, I can clearly see that, for a pNFS server, Seek may be done
>on the DS. (Actually, it's MAY in upper case.)
>
>I think the "MAY" you are referring to is basically saying that, unlike operation such
>as RENAME/REMOVE/OPEN, which are MDS-only, SEEK/READ/READ_PLUS
>are allowed in COMPOUNDs sent to the DS.
>
>In the case of the OPTIONAL operations among these (e.g. SEEK, READ_PLUS), the
>DS or the MDS may choose to support or not support them.
>
>What I can't find is any indication of whether the client should do Seek on the DS
>or MDS for this case.
>
>It's not there.
>Since others haven't replied to this, here's a couple of data points:
>- The FreeBSD server will support Seek on the MDS and DSs when it is configured
> as a pNFS server and the servers support NFSv4.2.
>- The FreeBSD NFSv4.2 client does Seek to the MDS at this time. (I may add
>doing Seek to the DS someday, but not unless I see enough need to justify
> doing it to avoid the overhead of Seek being done on the MDS as a proxy to the DS.)
>- The Linux NFSv4.2 client appears to do Seek to the MDS only. (From a quick look
> at Linux-4.17-rc2.)
> Trond, is this going to continue to be the case?
>
>If you are asking if I have any plans to change this, then the answer for now is “no”. >However if others are interested in working on it, then I’m happy to take patches.
>
>That said, I have a number of questions here. How are you dealing with striped files >on the client?
Good point. I hadn't thought about striping...

I'll admit I don't know how Seek to a DS would work for striped layouts?
Would it only return data/hole offsets for the stripes stored on that DS?
And for the densely packed case, would the offset be a logical offset for the
file or one mapped to the offset for that stripe?

--> This suggests that an MDS only implementation of Seek makes sense for striped
      File layouts.

> What about dense vs. sparsely packed layouts?
>In particular a DS serving a sparsely packed layout is supposed to deny you access to >the holes that lie between stripes (see NFS4ERR_PNFS_IO_HOLE) so what would a >server do when you ask it to seek across those boundaries?
--> I would have guessed it would ignore these I/O holes, since they aren't really
      a part of the file. Fortunately, I didn't need to worry about how this might
      be implemented because...
The FreeBSD server never does striping and simply provides the entire file on a
single DS for File layout (sparse, but no I/O holes caused by striping).

For Flexible file layout, the file can be mirrored, but that results in a copy
of the entire file on each of the mirrors. Again, no striping.

>I’m not really seeing any of those questions being answered directly in RFC7862. >There is a sentence or two describing how READ_PLUS should operate (but details >are missing), but there is nothing that describes SEEK behaviour w.r.t. pNFS, affaics.
>
Yep. Seems like a "hole" in the RFC (sorry, I couldn't resist;-).

rick

>Cheers
>  Trond