Re: [nfsv4] FW: I-D ACTION:draft-faibish-nfsv4-pnfs-access-permissions-check-03.txt

[Tom Haynes <[e-mail]>]

[e-mail] wrote:
> This is the revised permissions check draft - it actually deals with any
> circumstance under which a client cannot access a pNFS data server and
> wants to report that inaccessibility.
>
> Thanks,
> --David
>
>   

1)  This:

   To the extent that an 
   MDS can determine whether storage devices are accessible to clients, 
   an MDS SHOULD NOT include a storage device in any pNFS layouts sent 
   to a client that cannot access that storage device. At a minimum, the 
   server SHOULD perform these storage device accessibility checks 
   before exporting a filesystem that supports pNFS and when the device 
   configuration for such an exported filesystem is changed (e.g., to 
   add a storage device).


implies to me that the MDS has to keep track of 
LAYOUT4_RET_REC_FSID_NO_ACCESS
and LAYOUT4_RET_REC_FILE_NO_ACCESS layout return types per client. I.e., 
once it
knows a client has problems with a specific storage device, it should 
avoid using that
device again.

Given that we need this mechanism for the client to report errors, how 
then does the server
know when it can start using these storage devices again? Even if the 
MDS knows a positive
change took place, it has to rely on the client to do the checking.

Is this a difference between MUST and SHOULD? I.e., does having SHOULD 
mean that
the MDS can hand out the storage devices again to see if the client can 
suddenly start
using them again?

2) Is it NFS4ERR_PERM or NFS4ERR_ACCESS for access permission denial?

   o  NFS4ERR_PERM SHOULD be used for access permission denial; and 


 From 5661:

15.1.6.1. NFS4ERR_ACCESS (Error Code 13)

   Indicates permission denied.  The caller does not have the correct
   permission to perform the requested operation.  Contrast this with
   NFS4ERR_PERM (Section 15.1.6.2), which restricts itself to owner or
   privileged-user permission failures, and NFS4ERR_WRONG_CRED
   (Section 15.1.6.4), which deals with appropriate permission to delete
   or modify transient objects based on the credentials of the user that
   created them.

15.1.6.2. NFS4ERR_PERM (Error Code 1)

   Indicates requester is not the owner.  The operation was not allowed
   because the caller is neither a privileged user (root) nor the owner
   of the target of the operation.

Since this document talks about mount issues, I went back to RFC 1813
and the MOUNT protocol. MNT can return MNT3ERR_ACCES if
the client does not have access rights to the export.

I think NFS4ERR_ACCESS is more consistent with prior protocols
than NFS4ERR_PERM.

Going back to this draft, in section 3.3, I see:

   There are two NO_ACCESS layoutreturn_type4 values that indicate lack 
   of storage device access, LAYOUT4_RET_REC_FSID_NO_ACCESS and 
   LAYOUT4_RET_REC_FILE_NO_ACCESS. 

and 

   An NFS error (nfsstat4) is 
   included in the layoutreturn data structures for these two types to 
   distinguish access permission problems from device inaccessibility: 


I think access has been overloaded here and to clarify things, 
NFS4ERR_PERM is selected
over NFS4ERR_ACCESS.

The only other reasons I can see for using NFS4ERR_PERM instead of 
NFS4ERR_ACCESS
are related to security:

a) if the user credentials were insufficient, i.e., kerberized access to 
the storage device failed.

b) Section 13.12 of 5661:

   If the metadata server would
   deny a READ or WRITE operation on a file due to its ACL, mode
   attribute, open access mode, open deny mode, mandatory byte-range
   lock state, or any other attributes and state, the data server MUST
   also deny the READ or WRITE operation.

Which seems to point out a need for error codes for:

a) No access granted (mount for files, block devices have other means)
b) Permission denied for the operation.
c) Permission denied because of security.

The difference between b) and c) is that b) is per fileid and c) is per 
fsid. So a MDS could
still use the storage device in the layout for b), but should avoid 
using it for c).

3) I find:

   An NFS error (nfsstat4) is 
   included in the layoutreturn data structures for these two types to 
   distinguish access permission problems from device inaccessibility: 

   o  NFS4ERR_PERM SHOULD be used for access permission denial; and 

   o  NFS4ERR_NXIO SHOULD be used for inability to access a device. 

   Other NFS errors MAY be used when they are appropriate. All uses of 
   these two layout return types that report errors SHOULD be logged by 
   the client. 


to be under-specified. What are the other errors that a server can see 
and how is it
supposed to react to those errors?

I'd like to see language about which errors are MANDATORY to be 
supported and which
are OPTIONAL. I know I can read the above to see that there are only two 
MANDATORY
ones, but I can also read it to see that all are MANDATORY.

I don't want clients shoehorning every error code back into these two. 
And I do want clarification
on what a server should do with the OPTIONAL codes. I.e. is it free to 
reuse those storage
devices the next time that client asks for a layout?

4)

What if the storage device A returns NFS4ERR_STALE to the client while 
storage device
B returns NFS4_OK for an operation on the same layout? But for a 
different file with the
same layout, it is given NFS4_OK from both DSs. This isn't necessarily 
either a permission issue nor
a device inaccessibility issue. (It could be either: the export was 
changed or the filesystem
indicated by the filehandle does not exist. It could also just mean that 
the indicated file does
not exist.)

Would this be where LAYOUT4_RET_REC_FILE_NO_ACCESS and NFS4ERR_NXIO is 
appropriate?

Which leads to an even more interesting question, what constitutes a 
NFS4ERR_NXIO error?
As NFS4ERR_NXIO is defined by 5661 to not be a valid return code from 
any operation
or CB, I would take it to mean simply that the storage device is not 
responding to the client.
If the client got any error back from the storage device, then it could 
not use NFS4ERR_NXIO.

As the NFS4ERR_STALE could simply mean that the filehandle refers to a 
file which doesn't
exist, is it appropriate to inform the server to no longer use that 
storage device in the layouts
assigned to that client?

Where this is going is perhaps now is a good time to add more 
informative error codes from
the storage device to the client. This would in turn allow the client to 
send these back to the
MDS. I.e., there is a world of difference between this device (fsid) 
does not exist and this file
does not exist.