Re: [nfsv4] seeking clarifications in server-side copy CB_OFFLOAD feature

[[e-mail] (J. Bruce Fields)]

On Thu, Mar 02, 2017 at 04:07:42PM -0500, Olga Kornievskaia wrote:
> I'm seeking some clarification for doing the server-side copy feature.
> EINVAL in read beyond the end of the file has been already raised in
> another thread so I'll leave that aside. So here are a few more :
> -- possible inconsistent behavior in presence of an error from doing
> an async copy vs sync. CB_OFFLOAD can return an error and a partial
> copy result but there is no mechanism to do so in synchronous COPY. If
> CB_OFFLOAD returned an error and a partial copy, is that really an
> error or should it be a short copy?

On the client side, in the Linux case, the client is implementing
copy_file_range(), which doesn't have the option of returning both an
error and a number of bytes copied.  Returning an error to the
application when the destination was actually modified is unacceptable.
Returning success and a short copy is OK, I think--if the error's
important then the application will get it when it tries to copy the
rest.

> I can see a case where an error should not be ignore -- if error is
> ENOSPC. Should the client if received an error in CB_OFFLOAD propagate
> the error to the application?

I think it's OK to ignore the ENOSPC.  The application will try to write
the rest, the client will turn that into a new COPY request, this time
the response will probably be 0 bytes copied and ENOSPC, so this time
the client can return the error to the client.

Alternatively I guess the client could cache that ENOSPC result and
return it on the next copy without talking to the server?  But then
there's a risk that the client could miss the fact that space was freed
up since the previous COPY.

> Should it be server's responsibility to then if some
> error occurred but a partial copy succeeded to then not return the
> error to the client (that leads to the 2nd question).

I'd rather servers didn't use this odd quirk of the protocol, but maybe
it's too late to stop them.  And it seems easy enough just to ignore the
error on the client side in the case bytes were copied.

> -- related to errors returned in CB_OFFLOAD, there is no guidance with
> regards to the reboot recovery. If the destination server unable to
> finish the copy due to the source server rebooting, should it reply
> back with a successful short copy or should it return some error (and
> short copy of possibly 0bytes).

A short copy sounds reasonable to me.

--b.

> To address these two questions I find some wording in the spec that
> confuse me (section 15.2.3):
> 
> "If a failure does occur for a synchronous copy, wr_count will be set
> to the number of bytes copied to the destination file before the error
> occurred." ... it also talks about consecutive bits there but then it
> proceeds to also say that "If the failure occurred for an asynchronous
> copy, .... It will be able to determine the bytes copied from the
> coa_bytes_copied in the CB_OFFLOAD argument."
> 
> Since it talks about "wr_count", then it must means that COPY's
> nfsstat4 status is not an error. Does this then mean that if copy
> "failed" then the error must be ignored on the server and a partial
> copy results propagated to the client. But for the async copy,
> "coa_bytes_copied" means that CB_OFFLOAD also set nfsstat4 to some
> error. So error is not ignored in the async case?
> 
> -- the spec lists the kind of errors CB_OFFLOAD reply can return but
> there is no wording for what kind of errors be included in
> CB_OFFLOAD's call. For instance, I can see that a source server reboot
> can translate to ERR_BAD_HANDLE or it could be BAD_SESSION or
> STALE_CLIENTID or BAD_STATEID. It is not clear what errors to check
> for and recover by restarting the copy and what errors shouldn't
> happen. Something like BAD_SESSION implies that server-to-server
> protocol was NFS4.x x>=1. Can the client receive an NFSv3 error in
> CB_OFFLOAD?
> 
> While I realize that server-to-server copy details is not a part of
> the RFC7862, but what the client receives in CB_OFFLOAD and how to
> interpret that is unclear.
> 
> Thank you.
> 
> _______________________________________________
> nfsv4 mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/nfsv4