Re: [nfsv4] https://datatracker.ietf.org/doc/draft-myklebust-nfsv4-pnfs-backend/

[Trond Myklebust <[e-mail]>]

On Wed, 2010-11-17 at 16:04 -0800, Thomas Haynes wrote:
> Trond,
> 
> Some questions I had after reading this:
> 
> 1) What keeps another client from spoofing a file handle to get at a cached file?
> 
> Note that there is no export control on cached files. The caching server
> has to assume that the MDS did the access checks. With AUTH_UNIX,
> this means that any client can get access with a spoofed file handle.
> 
> Another implementation issue here is that the caching server, if it does
> per op access checks, will now need to know to skip such checks on these
> files.

The client acting as the DS is supposed to check with the MDS that the
file handle is valid and that the client 2 is authorised to access it.
That's precisely what the PROXY_OPEN is for: to allow the MDS to check
that client 2 holds a layout.

> 2) What keeps a file handle clash from occurring?
> 
> I.e., the cached file has a file handle which maps to a valid file on the caching
> server.

It is up to the MDS to generate data server filehandles that do not
clash.

> 3) This also goes with the following from the draft:
> 
> >    When the data server receives a READ request from a client with a
> > 
> > 
> > 
> > 
> > Myklebust                Expires January 7, 2010                [Page 5]
> >  
> > Internet-Draft      pNFS back end protocol extensions          July 2009
> > 
> > 
> > 
> >    stateid or a data server filehandle that it does not recognise, it
> >    attempts to validate that request using the PROXY_OPEN call.  This
> >    operation will convert the data server filehandle as provided by the
> >    layout into a real filehandle, that the data server can use to access
> >    the file on the metadata server.  In order to make it easy for the
> >    data server to identify the file, the real filehandle SHOULD match
> >    the filehandle that was returned to the client when it received the
> >    read delegation.
> 
> 
> Which I take to imply that stateids must be unique on the MDS and
> these caching data servers. I.e., if a client presents a stateid that is
> already being used on the caching data server and it presents a valid
> filehandle from that caching data server, then the caching data server
> has to assume that the client is trying to access a non-cached file.

What do you mean by 'valid filehandle'? Data server filehandles are
_not_ the same as ordinary filehandles; they are filehandles that the
MDS has handed out as part of a layout and that need to be translated
via PROXY_OPEN. It is expected that they will be invalidated (and
presumably forgotten by the MDS and DS) when the layout is revoked.

> 4) I'm also reading this as saying that the data server filehandle SHOULD
> match the real filehandle presented in the layout. Indeed if it does not,
> then the data server will need to pass any filehandle it gets back to the
> MDS to see if it is a file handle it is caching.

It needs to do this once in order to translate the filehandle. Once that
is done, it can rely on the server to call it back when the layout is
revoked (and the filehandle is invalidated).

> 5) 
> 
> > 
> >    The PROXY_OPEN call also checks the access rights that were granted
> >    by the layout and the READ stateid for validity.  If the pNFS client
> >    in question does not hold a layout for this file, the PROXY_OPEN
> >    request from the data server will return NFS4ERR_PNFS_NO_LAYOUT.  In
> >    this case, the data server should not attempt to service the READ
> >    request, but should pass the error on to the pNFS client.
> > 
> 
> Is NFS4ERR_PNFS_NO_LAYOUT the only error that can be set here?

No. See the next paragraph.

> a) File does not exist (NFS4ERR_STALE)
> b) Client does not have permission to the underlying export (NFS4ERR_ACCESS)
> c) ...
> d) stateid is no longer valid, etc 
> 
> Ideally, the client should have gone through all of these steps in order to get
> the filehandle from the MDS, but nothing stops the client from holding onto the
> filehandle and the MDS deleting it, etc.

The DS holds a delegation for the file. The MDS can't delete it or
change access permissions; that would revoke the delegation, and hence
all layouts.

Cheers
  Trond