Re: [nfsv4] server-side copy offload I-D

["Mike Eisler" <[e-mail]>]

On Wed, April 1, 2009 2:13 am, anshul wrote:
> On Wed, 2009-04-01 at 00:17 -0500, Tom Haynes wrote:
>> Mike Eisler wrote:
>> > On Tue, March 31, 2009 7:15 pm, Tom Haynes wrote:
>> >> Noveck, Dave wrote:
>> >>> The inter-server protocol is not defined in this proposal.  I think
>> you
>> >>> need to somehow say that it is up to the designer of the
>> implementation
>> >>> to prevent its normal security mechanisms (such as export access
>> >>> checking) from being defeated, without having to go into any detail
>> >>> about what those non-spec mechanisms might be.
>> >> I'm confused, is it being defined or not?
>> > The combination of RPCSEC_GSSv3 and COPY_NOTICE tells the source
>> > server authorize the destination server to make a copy on behalf of
>> > the user.
>> So RPCSEC_GSSv3 is sufficient to bypass export checking? I understand
>> how it will
>> conceptually pass along the user credentials, but export checking is
>> machine based and not
>> user based.

As Dave noted, export checking is not part of the NFS protocol.
Whether your server has export checking or not, and whether it
your server is willing to let the destination server acting as an NFS
client to copy data from it when authorized via COPY_NOTICE is
a decision for you, the implementer to make. How that is done is an
implementation choice. As specified, COPY_NOTICE provides a limited
override of any export policies that are in place.

> The destination server is being authorized read access to the source
> server via COPY_NOTICE, which is machine based.

More precisely, RPCSEC_GSSv3 + COPY_NOTICE authorizes a principal on
the destination server to read the file from the source server. The
principal on the destination server is likely a machine-based principal,
such as nfs/<dest-server-name>@EXAMPLE.COM. WHich makes sense, because
the only principals on the NFS servers that NFS clients are aware of, is
ths GSS principal name of nfs@<hostname> per the NFS standards.


>> Hold off replying to this until you read the next part....
>> > If RPCSEC_GSSV3 is not being used, then the source server will
>> > check if the user and the user's source network address has read
>> access
>> > according to the export policies of the destination server.
>> >
>> >
>>
>> Shouldn't that be write access? Or have you flipped your servers around?
>
> Read access should be checked at the source server and write access at
> the destination server via its export policies.

Yes. From the I-D:

"If the request is for a server-to-server copy, the source-fh is a
   filehandle from the source server and the compound procedure is being
   executed on the destination server."

>
>>
>> And to clarify, the source server can only do the check by attempting a
>> write as that user?

No, a READ.

>> And
>> if I'm the destination server, I'm not going to trust that the source
>> server has done the
>> authentication correctly.

The destination server doesn't care if the source server is checking
authentication correctly. All it cares about are:

1. does the user have write access to the destination?
2. will the source server let me (the destination server) read the file?

If both are yes, the COPY will happen.

>>
>> I.e., RPCSEC_GSSV3 allows us to not put user credentials in the COPY
>> procedures. But

COPY is inside COMPOUND. COMPOUND is an RPC. RPCs always have credentials.

If the destination server requires RPCSEC_GSS for the target file/directory,
the client will use RPCSEC_GSS. SECINFO/SECINFO_NO_NAME will negotiate this.
The credentials of the user that wants the file copied will be in
the RPCSEC_GSS or AUTH_SYS creds.

>> if it is not being used, how do the user credentials and the machine
>> credentials for that matter
>> get communicated from the source to destination server?
>>
>> Either you have to add them to the COPY procedures or can you point out
>> what in the
>> current protocol allows you to do this?
>
> Since COPY is always sent to the destination, the RPC headers should be
> able to provide the user credentials to the destination server. Machine
> credentials are provided by the COPY operation's arguments to the
> destination server.

The first sentence summarizes what I wrote. I believe what Anshul means by
the second sentence is that because the source server is identified in
COPY's arguments, the destination server will use its own principal
credentials when attempting to copy from the source server.

>> Also, even if the destination server did grant export access at the time
>> of the start of the
>> copy, the export may be changed at any point during the sequence of
>> WRITEs from the
>> source to destination. Thus any WRITE could trigger another access
>> check, so where is
>> the information coming from?
>
> The destination READs from the source and not vice-versa.

Yes. At first we were going to propose it the other way, but decided
that it would be easier if a source only had to grant read access to
a third party than granting write access to a third party. For the latter,
the issue isn't just that granting write access is more privilege, but that
we would have to deal with stuff like space reservations.

>
> Skipping some of next few questions, because they seem to based on a
> different premise.
>
>>
>> *Follow-up*: I really like this example:
>>
>> > If "cp" were re-implemented to use a copy() system call, then we
>> > have:
>> >
>> > clientC: COPY_NOTICE -> serverA
>> > clientC: COPY -> serverB
>> >
>> >
>> > serverB, perhaps using NFSv4.x, sends a series of READ requests to
>> > serverA.
>>
>> If we modified COPY_NOTICE to be:
>>
>>                    struct COPY_NOTIFY4args {
>>                            /* SAVED_FH: source file */
>>                            /* CURRENT_FH: destination directory */
>>                            netloc4         cna_destination_server;
>>                    };
>>
>> then the destination server could do the export check and directory
>> access check right then - we wouldn't have to worry about the source
>> server having to know this information. Conceptually, we wouldn't care
>> about RPCSEC_GSSV3 either - the access check is a metadata operation and
>> we don't need the user credentials during the writes...

As Anshul says, you appear to think that COPY is directed at the source.

Whether COPY is directed at the source or destination, some kind of
the delegation of authority from the client to another NFS server
is needed. We think that RPCSEC_GSSv3 holds a lot of promise.
We could do it without RPCSEC_GSSv3, such as by passing a secret in
a blob in COPY_NOTIFY, but if RPCSEC_GSSv3 is going to come to pass,
we'd rather do it with RPCSEC_GSSv3.

>>
>> The destination server would have to maintain state about the client
>> IP in case either the export access cache was flushed or the export
>> was modified. In either case, the server would be required to
>> redetermine
>> the access rights.
>>
>> Also, the destination server would not care whether the source server
>> had access rights or not. It would have to bypass export access checks
>> for this combination of (server, FH).
>>
>> Finally, CB_COPY would need to be modified such that if the client
>> failed
>> the export check as detailed above, the server could inform the client
>> that it no longer had permission to write the file.
>>
>> If the destination server reboots, then it will not understand the
>> WRITEs
>> coming from the source server. I.e., the source server may not have
>> export
>> access permissions. If it does have write permissions, everything will
>> seem
>> okay. If they fail, then the source server can contact the client and
>> inform
>> it that it needs to re-establish the COPY_NOTIFICATION.

Yes, all you are doing to moving the problem.

>>
>> In looking at the draft, I can't tell if the CB_COPY is maintained
>> between
>> the client and destination server or the client and the source server.
>
> The COPY is always sent to the destination server, thus the CB_COPY is
> always from the destination server to client.
>
>>
>> Like I said, I really liked this example you gave. I think providing
>> diagrams like this in the draft would make it much easier to understand
>> the flows going on here.

Point taken. Thanks.