Re: [nfsv4] https://datatracker.ietf.org/doc/draft-myklebust-nfsv4-pnfs-backend/

[Trond Myklebust <[e-mail]>]

On Thu, 2010-11-18 at 10:26 -0800, Thomas Haynes wrote:
> On Nov 17, 2010, at 5:59 PM, Trond Myklebust wrote:
> 
> > On Wed, 2010-11-17 at 16:04 -0800, Thomas Haynes wrote:
> >> Trond,
> >> 
> >> Some questions I had after reading this:
> >> 
> >> 1) What keeps another client from spoofing a file handle to get at a cached file?
> >> 
> >> Note that there is no export control on cached files. The caching server
> >> has to assume that the MDS did the access checks. With AUTH_UNIX,
> >> this means that any client can get access with a spoofed file handle.
> >> 
> >> Another implementation issue here is that the caching server, if it does
> >> per op access checks, will now need to know to skip such checks on these
> >> files.
> > 
> > The client acting as the DS is supposed to check with the MDS that the
> > file handle is valid and that the client 2 is authorised to access it.
> > That's precisely what the PROXY_OPEN is for: to allow the MDS to check
> > that client 2 holds a layout.
> 
> 
> So I can read that in your text now. The first time through, that was not
> what I got. In retrospect, I kept confusing whether the caching client or
> client 2 was the one with the layout.
> 
> 
> 
> > 
> >> 2) What keeps a file handle clash from occurring?
> >> 
> >> I.e., the cached file has a file handle which maps to a valid file on the caching
> >> server.
> > 
> > It is up to the MDS to generate data server filehandles that do not
> > clash.
> 
> 
> The MDS can not know what filehandles the caching client has in use.

When it sent the REGISTER_DS request, the DS client included a unique
rea_mds_identifier which the server is supposed to put as the first 8
bytes of the data server filehandle. This ensures that the DS client can
trace the data server filehandle as belonging to a particular MDS. The
MDS is then at liberty to define the rest of the file handle as it
pleases.

> > 
> >> 3) This also goes with the following from the draft:
> >> 
> >>>   When the data server receives a READ request from a client with a
> >>> 
> >>> 
> >>> 
> >>> 
> >>> Myklebust                Expires January 7, 2010                [Page 5]
> >>> 
> >>> Internet-Draft      pNFS back end protocol extensions          July 2009
> >>> 
> >>> 
> >>> 
> >>>   stateid or a data server filehandle that it does not recognise, it
> >>>   attempts to validate that request using the PROXY_OPEN call.  This
> >>>   operation will convert the data server filehandle as provided by the
> >>>   layout into a real filehandle, that the data server can use to access
> >>>   the file on the metadata server.  In order to make it easy for the
> >>>   data server to identify the file, the real filehandle SHOULD match
> >>>   the filehandle that was returned to the client when it received the
> >>>   read delegation.
> >> 
> >> 
> >> Which I take to imply that stateids must be unique on the MDS and
> >> these caching data servers. I.e., if a client presents a stateid that is
> >> already being used on the caching data server and it presents a valid
> >> filehandle from that caching data server, then the caching data server
> >> has to assume that the client is trying to access a non-cached file.
> > 
> > What do you mean by 'valid filehandle'? Data server filehandles are
> > _not_ the same as ordinary filehandles; they are filehandles that the
> > MDS has handed out as part of a layout and that need to be translated
> > via PROXY_OPEN. It is expected that they will be invalidated (and
> > presumably forgotten by the MDS and DS) when the layout is revoked.
> 
> 
> Again, there is nothing to prevent the MDS from handing out a filehandle
> which happens to be one in use by the caching client. If client 2 already has
> that filehandle from an earlier action, then it may be the case that it presents
> it back to caching client.

The DS client knows the difference between an ordinary filehandle and a
DS filehandle. The latter is _only_ used as part of a pNFS READ or WRITE
request by client 2 and will have been appropriately generated by the
MDS (see above).

Nothing stops the MDS from including information about the layout
stateid in the DS filehandle.

> >> 4) I'm also reading this as saying that the data server filehandle SHOULD
> >> match the real filehandle presented in the layout.

BTW: this is false. The DS filehandle MUST NOT match the real
filehandle. See above.

>  Indeed if it does not,
> >> then the data server will need to pass any filehandle it gets back to the
> >> MDS to see if it is a file handle it is caching.
> > 
> > It needs to do this once in order to translate the filehandle. Once that
> > is done, it can rely on the server to call it back when the layout is
> > revoked (and the filehandle is invalidated).
> 
> 
> Yes, once it has a mapping of valid data server filehandles, it can cache
> them.

That's the whole point of distinguishing DS filehandles and ordinary
filehandles. The former represents the layout.

> > 
> >> a) File does not exist (NFS4ERR_STALE)
> >> b) Client does not have permission to the underlying export (NFS4ERR_ACCESS)
> >> c) ...
> >> d) stateid is no longer valid, etc 
> >> 
> >> Ideally, the client should have gone through all of these steps in order to get
> >> the filehandle from the MDS, but nothing stops the client from holding onto the
> >> filehandle and the MDS deleting it, etc.
> > 
> > The DS holds a delegation for the file. The MDS can't delete it or
> > change access permissions; that would revoke the delegation, and hence
> > all layouts.
> > 
> > Cheers
> >  Trond
> > 
> 
> I'm thinking a couple of pictures would help in the final text - I can see all of this
> now, but a concrete example could have helped.

OK. I'll think about how to add this info.

I apparently also need to add in a paragraph about DS filehandles vs.
real filehandles.

Cheers
  Trond