Re: [nfsv4] https://datatracker.ietf.org/doc/draft-myklebust-nfsv4-pnfs-backend/

[Thomas Haynes <[e-mail]>]

On Nov 17, 2010, at 5:59 PM, Trond Myklebust wrote:

> On Wed, 2010-11-17 at 16:04 -0800, Thomas Haynes wrote:
>> Trond,
>> 
>> Some questions I had after reading this:
>> 
>> 1) What keeps another client from spoofing a file handle to get at a cached file?
>> 
>> Note that there is no export control on cached files. The caching server
>> has to assume that the MDS did the access checks. With AUTH_UNIX,
>> this means that any client can get access with a spoofed file handle.
>> 
>> Another implementation issue here is that the caching server, if it does
>> per op access checks, will now need to know to skip such checks on these
>> files.
> 
> The client acting as the DS is supposed to check with the MDS that the
> file handle is valid and that the client 2 is authorised to access it.
> That's precisely what the PROXY_OPEN is for: to allow the MDS to check
> that client 2 holds a layout.


So I can read that in your text now. The first time through, that was not
what I got. In retrospect, I kept confusing whether the caching client or
client 2 was the one with the layout.



> 
>> 2) What keeps a file handle clash from occurring?
>> 
>> I.e., the cached file has a file handle which maps to a valid file on the caching
>> server.
> 
> It is up to the MDS to generate data server filehandles that do not
> clash.


The MDS can not know what filehandles the caching client has in use.



> 
>> 3) This also goes with the following from the draft:
>> 
>>>   When the data server receives a READ request from a client with a
>>> 
>>> 
>>> 
>>> 
>>> Myklebust                Expires January 7, 2010                [Page 5]
>>> 
>>> Internet-Draft      pNFS back end protocol extensions          July 2009
>>> 
>>> 
>>> 
>>>   stateid or a data server filehandle that it does not recognise, it
>>>   attempts to validate that request using the PROXY_OPEN call.  This
>>>   operation will convert the data server filehandle as provided by the
>>>   layout into a real filehandle, that the data server can use to access
>>>   the file on the metadata server.  In order to make it easy for the
>>>   data server to identify the file, the real filehandle SHOULD match
>>>   the filehandle that was returned to the client when it received the
>>>   read delegation.
>> 
>> 
>> Which I take to imply that stateids must be unique on the MDS and
>> these caching data servers. I.e., if a client presents a stateid that is
>> already being used on the caching data server and it presents a valid
>> filehandle from that caching data server, then the caching data server
>> has to assume that the client is trying to access a non-cached file.
> 
> What do you mean by 'valid filehandle'? Data server filehandles are
> _not_ the same as ordinary filehandles; they are filehandles that the
> MDS has handed out as part of a layout and that need to be translated
> via PROXY_OPEN. It is expected that they will be invalidated (and
> presumably forgotten by the MDS and DS) when the layout is revoked.


Again, there is nothing to prevent the MDS from handing out a filehandle
which happens to be one in use by the caching client. If client 2 already has
that filehandle from an earlier action, then it may be the case that it presents
it back to caching client.


> 
>> 4) I'm also reading this as saying that the data server filehandle SHOULD
>> match the real filehandle presented in the layout. Indeed if it does not,
>> then the data server will need to pass any filehandle it gets back to the
>> MDS to see if it is a file handle it is caching.
> 
> It needs to do this once in order to translate the filehandle. Once that
> is done, it can rely on the server to call it back when the layout is
> revoked (and the filehandle is invalidated).


Yes, once it has a mapping of valid data server filehandles, it can cache
them.


> 
>> 5) 
>> 
>>> 
>>>   The PROXY_OPEN call also checks the access rights that were granted
>>>   by the layout and the READ stateid for validity.  If the pNFS client
>>>   in question does not hold a layout for this file, the PROXY_OPEN
>>>   request from the data server will return NFS4ERR_PNFS_NO_LAYOUT.  In
>>>   this case, the data server should not attempt to service the READ
>>>   request, but should pass the error on to the pNFS client.
>>> 
>> 
>> Is NFS4ERR_PNFS_NO_LAYOUT the only error that can be set here?
> 
> No. See the next paragraph.

I'd swear you added that after I copied the text from the draft. :->

I probably became laser focused.


> 
>> a) File does not exist (NFS4ERR_STALE)
>> b) Client does not have permission to the underlying export (NFS4ERR_ACCESS)
>> c) ...
>> d) stateid is no longer valid, etc 
>> 
>> Ideally, the client should have gone through all of these steps in order to get
>> the filehandle from the MDS, but nothing stops the client from holding onto the
>> filehandle and the MDS deleting it, etc.
> 
> The DS holds a delegation for the file. The MDS can't delete it or
> change access permissions; that would revoke the delegation, and hence
> all layouts.
> 
> Cheers
>  Trond
> 

I'm thinking a couple of pictures would help in the final text - I can see all of this
now, but a concrete example could have helped.