Re: [nfsv4] Server-side copy question

[Olga Kornievskaia <[e-mail]>]

On Tue, Oct 24, 2017 at 7:43 AM, David Noveck <[e-mail]> wrote:
>> So in the short write case it sounds like it *is* the server's
>> responsibility to commit.
>
> Perhaps it should be the server's responsibility, but it is not,
> according to RFC7862
>
>> That's weird,
>
> I agree that it is wierd.  I feel it is well-intentioned but not viable
> since
> you are essentially creating your own protocol, if clients depend on
> this behavior an there is no point in doing it, if clients do not.
>
>> but as long as the short write
>> case is rare, maybe it's not a big deal?
>
> From a performance point of view, it is not a big deal.  However it is
> a big deal that the spec is broken. I hate when that happens :-(
>
>>Hm: "The coa_bytes_copied value indicates the number of bytes copied but
>> not which specific bytes have been copied."  That makes coa_bytes_copied
>> pretty much worthless.
>
> Any non-zero value of coa_bytes_copied is worthless because the client does
> not know:
>
> Which particular bytes were copied
> Whether those bytes were copied to the destination stably
> How to commit any  non-stably-written bytes.
>
>>I guess I'd be in favor of using the NFS_OK branch of CB_OFFLOAD
>> whenever a nonzero number of bytes is copied.
>
> That's likely to create an interoperability problem, since there are almost
> certainly clients who assume that when there is no error, all requested
> bytes
> have been copied.

I don't think it's true that clients would assume this. I think the
client can only assume server copied as many bytes as it was specified
in the CB_OFFLOAD bytes_copied regardless of how many bytes the client
had asked for. At least that's how I read the spec.

> Basically, the server is pretending there is no error and

Yes, pretending that there was no error is the part I'm not totally
comfortable with. And I agree with the comments that the client can
try again to get the same error.

> then
> relying on the client to figure out that there was one, which could only be
> done if the
> spec tells the clients about this.  If RFC7862 doesn't do this, you wind up
> essentially creating a private protocol.
>
>> If less than the
>> requested number of bytes were copied, the client can always find out
>> the error (if any) by attempting to copy the rest.
>
> Unfortunately there is no way of determining which bytes were copied so
> there
> is no way of knowing "the rest", i.e. the bytes not copied

I would like to understand the comment by Bruce and you about not
knowing which bytes were copied. I don't understand why it would not
be the bytes from the offset to offset+bytes_copied? And Dave,
wouldn't the "rest of the bytes" be the request length - bytes_copied?
What am I missing?

If you are suggested that the server is allowed to copy a requested
range in any order of smaller chunks then there is a problem with a
non-error case because we don't returned in COPY/CB_OFFLOAD offset and
bytes_copied (just bytes_copied).

I think we know which bytes are copied we just don't know how they
were copied: stable or not (if the server chooses to specify an error
in CB_OFFLOAD).

> On Mon, Oct 23, 2017 at 9:36 PM, J. Bruce Fields <[e-mail]>
> wrote:
>>
>> On Mon, Oct 23, 2017 at 05:43:42PM -0400, Olga Kornievskaia wrote:
>> > On Fri, Oct 20, 2017 at 3:33 PM, J. Bruce Fields <[e-mail]>
>> > wrote:
>> > > On Wed, Oct 18, 2017 at 05:22:08PM -0400, Olga Kornievskaia wrote:
>> > >> On Tue, Feb 28, 2017 at 12:32 PM, J. Bruce Fields
>> > >> <[e-mail]> wrote:
>> > >> > On Tue, Feb 28, 2017 at 05:04:27PM +0000, Adamson, Andy wrote:
>> > >> >> I also thought that bytes copied is only included in the success
>> > >> >> case – but as Olga pointed out, not true. The coa_status of NFS4_OK has the
>> > >> >> coa_resok4->wr_count showing the bytes copied, and the default (e.g. failure
>> > >> >> cases) has the coa_bytes_copied.
>> > >> >>
>> > >> >> So the server can send a CB_OFFLOAD with an error (such as the
>> > >> >> –EINVAL in the example) and send the bytes copied in coa_bytes_copied.
>> > >> >>
>> > >> >>    union offload_info4 switch (nfsstat4 coa_status) {
>> > >> >>    case NFS4_OK:
>> > >> >>            write_response4 coa_resok4;
>> > >> >>    default:
>> > >> >>            length4         coa_bytes_copied;
>> > >> >>    };
>> > >> >
>> > >> > Gah, I'm sorry, Olga said CB_OFFLOAD but for some reason I was
>> > >> > looking
>> > >> > at COPY.
>> > >> >
>> > >> > I have no idea what to think of that.  I don't see why it should be
>> > >> > different from COPY.
>> > >>
>> > >> I'd like to resurrect this thread...
>> > >>
>> > >> This discussion was initiated to allow for the server to do a partial
>> > >> copy instead of failing the copy with EINVAL when copy is requested
>> > >> beyond the end of the file.
>> > >>
>> > >> However, the problem I'm running into is that because CB_OFFLOAD only
>> > >> returns "coa_bytes_copied" and does not include stable_how and
>> > >> verifier. So even if client were to then send a COMMIT for the bytes
>> > >> that were copied, it has no verifier to match from the COMMIT to
>> > >> anything.
>> > >>
>> > >> Should the client then ignore the lack of the matching verifier and
>> > >> just assume the data was committed for the partial bytes?
>> > >
>> > > Yes, sounds like in the asynchronous case the server has to wait for
>> > > the
>> > > writes to reach disk before calling CB_OFFLOAD.  (Do we currently do
>> > > that?).
>> >
>> > Why? No the spec does not require the server to do no such thing. The
>> > server sends back in CB_OFFLOAD if the bytes were written as UNSTABLE
>> > or FILE_SYNC.
>>
>> Look back at what you said before....  So the problem you were
>> explaining (missing stable_how and verifier) was only for the short
>> write case?

Yes.

>> So in the short write case it sounds like it *is* the server's
>> responsibility to commit.

I disagree. I don't see anywhere in the spec that requires the server
to do that.

>>   That's weird, but as long as the short write
>> case is rare, maybe it's not a big deal?
>>
>> Hm: "The coa_bytes_copied value indicates the number of bytes copied but
>> not which specific bytes have been copied."  That makes coa_bytes_copied
>> pretty much worthless.

Yep that's what I thought.

>> I guess I'd be in favor of using the NFS_OK branch of CB_OFFLOAD
>> whenever a nonzero number of bytes is copied.  If less than the
>> requested number of bytes were copied, the client can always find out
>> the error (if any) by attempting to copy the rest.

Ok

>>
>> --b.
>>
>> >
>> > > In theory that shouldn't be too expensive because asynchronous COPY
>> > > should normally be done with a very large range.
>> > >
>> > > (I wonder if the server should just do a synchronous copy if the copy
>> > > would be less than a few megs?)
>> >
>> > Yes the server could determine to convert an asynchronous copy into a
>> > synchronous one based on size.
>> >
>> > I guess I'm still confused how to handle the implementation so that it
>> > doesn't kill interoperability with other possible (future)
>> > implementations...
>>
>> _______________________________________________
>> nfsv4 mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/nfsv4
>
>