[nfsv4] Current-stateid/special-stateid for Layoutget

[Rick Macklem <[e-mail]>]

Boring background...
I created a pNFS server using Flex File layout and GlusterFS. It was
a dud. Basically worked, but was very slow.
I came up with Plan B, which is a simple, single MDS server using
File layout and FreeBSD's nfsd. It seems to work pretty well, which
brings me to...

When I have a client mounted with pNFS enabled, I see better I/O
performance for tests on fairly big files, however for software builds
(lots of I/O on small files), the performance is about the same for pNFS
as when pNFS is disabled and all I/O is being done through the MDS.

I am not 100% sure, but when pNFS is enabled, the client does two RPCs
when a file is opened:
- one that does the Open
- one that does the LayoutGet
and I think the extra RPC round trip time slows the pNFS case down.

The obvious fix is to do the Open and LayoutGet in the same compound RPC.
If the client already has a layout for the file, the LayoutGet can use the
Current-FH set by the Open operation and the layout_stateid and life is good.
However, if the client is getting the first layout for the file (this is usually the
case for the File Layout), then it doesn`t work, because LayoutGet needs
an open_stateid argument.

So, here`s what I am thinking of:
- Define a Current-OpenStateID, which is unset at the start of a compound.
- A successful Open op. sets the Current-OpenStateID to the one in its reply,
  so that it is set along with the Current-FH.
- Any other operation that sets the Current-FH unsets Current-OpenStateID.
- Redefine the case of LayoutGet with a special stateid as...
  If the stateid in LayoutGet`s argument is a special stateid, replace it with
  the Current-OpenStateID if one is set.
--> Then, do everything else just as RFC-5661 states.

I don't know about other implementations, but this would be easy to do in
the FreeBSD NFSv4.1 server and would allow LayoutGet to follow Open in
the same RPC, I think? (I haven't actually coded this yet.)

For server's that don't support this, I don't think it would be a problem,
since the compound would fail at the LayoutGet with NFS4ERR_BAD_STATEID.
At that point, the client would have the Open reply and could just do the
additional compound RPC with LayoutGet, using the OpenStateID in the Open
reply.

So, what do others think?

Also, I haven't followed the versioning discussion, so I have no idea if this
can be done without having a new minor version?

Thanks, rick
ps: I will probably code this up in FreeBSD experimentally, to see how it works.