Re: [nfsv4] Server-side copy question

[Olga Kornievskaia <[e-mail]>]

On Wed, Oct 25, 2017 at 6:30 AM, David Noveck <[e-mail]> wrote:
>
>>I would like to understand the comment by Bruce and you about not
>> knowing which bytes were copied. I don't understand why it would not
>> be the bytes from the offset to offset+bytes_copied? And Dave,
>> wouldn't the "rest of the bytes" be the request length - bytes_copied?s
>> have been copied."
>
> It might be and it is resonable if it is but RFC7862, says, as Bruce noted,
> "The coa_bytes_copied value indicates the number of bytes copied but
> not which specific bytes.".  I don't know wny it says that or how those
> words came
> to be in the spec.  If the spec did not say that, it would reasonable to
> assume that it
> would be the bytes from the offset to offset+bytes_copied.  But given
> RFC7862,
> as it is, I think clients cannot assume that.  It may be that these words
> are just
> plaiin wrong, but assuming that is risky, and fixing the spec to remove them
> is very
> difficult, even if everyone were to agree that they shouldn't be there.

Ah, got it. Thank you.

> ,
>
> On Tue, Oct 24, 2017 at 11:52 AM, Olga Kornievskaia <[e-mail]>
> wrote:
>>
>> On Tue, Oct 24, 2017 at 7:43 AM, David Noveck <[e-mail]>
>> wrote:
>> >> So in the short write case it sounds like it *is* the server's
>> >> responsibility to commit.
>> >
>> > Perhaps it should be the server's responsibility, but it is not,
>> > according to RFC7862
>> >
>> >> That's weird,
>> >
>> > I agree that it is wierd.  I feel it is well-intentioned but not viable
>> > since
>> > you are essentially creating your own protocol, if clients depend on
>> > this behavior an there is no point in doing it, if clients do not.
>> >
>> >> but as long as the short write
>> >> case is rare, maybe it's not a big deal?
>> >
>> > From a performance point of view, it is not a big deal.  However it is
>> > a big deal that the spec is broken. I hate when that happens :-(
>> >
>> >>Hm: "The coa_bytes_copied value indicates the number of bytes copied but
>> >> not which specific bytes have been copied."  That makes
>> >> coa_bytes_copied
>> >> pretty much worthless.
>> >
>> > Any non-zero value of coa_bytes_copied is worthless because the client
>> > does
>> > not know:
>> >
>> > Which particular bytes were copied
>> > Whether those bytes were copied to the destination stably
>> > How to commit any  non-stably-written bytes.
>> >
>> >>I guess I'd be in favor of using the NFS_OK branch of CB_OFFLOAD
>> >> whenever a nonzero number of bytes is copied.
>> >
>> > That's likely to create an interoperability problem, since there are
>> > almost
>> > certainly clients who assume that when there is no error, all requested
>> > bytes
>> > have been copied.
>>
>> I don't think it's true that clients would assume this. I think the
>> client can only assume server copied as many bytes as it was specified
>> in the CB_OFFLOAD bytes_copied regardless of how many bytes the client
>> had asked for. At least that's how I read the spec.
>>
>> > Basically, the server is pretending there is no error and
>>
>> Yes, pretending that there was no error is the part I'm not totally
>> comfortable with. And I agree with the comments that the client can
>> try again to get the same error.
>>
>> > then
>> > relying on the client to figure out that there was one, which could only
>> > be
>> > done if the
>> > spec tells the clients about this.  If RFC7862 doesn't do this, you wind
>> > up
>> > essentially creating a private protocol.
>> >
>> >> If less than the
>> >> requested number of bytes were copied, the client can always find out
>> >> the error (if any) by attempting to copy the rest.
>> >
>> > Unfortunately there is no way of determining which bytes were copied so
>> > there
>> > is no way of knowing "the rest", i.e. the bytes not copied
>>
>> I would like to understand the comment by Bruce and you about not
>> knowing which bytes were copied. I don't understand why it would not
>> be the bytes from the offset to offset+bytes_copied? And Dave,
>> wouldn't the "rest of the bytes" be the request length - bytes_copied?
>> What am I missing?
>>
>> If you are suggested that the server is allowed to copy a requested
>> range in any order of smaller chunks then there is a problem with a
>> non-error case because we don't returned in COPY/CB_OFFLOAD offset and
>> bytes_copied (just bytes_copied).
>>
>> I think we know which bytes are copied we just don't know how they
>> were copied: stable or not (if the server chooses to specify an error
>> in CB_OFFLOAD).
>>
>> > On Mon, Oct 23, 2017 at 9:36 PM, J. Bruce Fields <[e-mail]>
>> > wrote:
>> >>
>> >> On Mon, Oct 23, 2017 at 05:43:42PM -0400, Olga Kornievskaia wrote:
>> >> > On Fri, Oct 20, 2017 at 3:33 PM, J. Bruce Fields
>> >> > <[e-mail]>
>> >> > wrote:
>> >> > > On Wed, Oct 18, 2017 at 05:22:08PM -0400, Olga Kornievskaia wrote:
>> >> > >> On Tue, Feb 28, 2017 at 12:32 PM, J. Bruce Fields
>> >> > >> <[e-mail]> wrote:
>> >> > >> > On Tue, Feb 28, 2017 at 05:04:27PM +0000, Adamson, Andy wrote:
>> >> > >> >> I also thought that bytes copied is only included in the
>> >> > >> >> success
>> >> > >> >> case – but as Olga pointed out, not true. The coa_status of
>> >> > >> >> NFS4_OK has the
>> >> > >> >> coa_resok4->wr_count showing the bytes copied, and the default
>> >> > >> >> (e.g. failure
>> >> > >> >> cases) has the coa_bytes_copied.
>> >> > >> >>
>> >> > >> >> So the server can send a CB_OFFLOAD with an error (such as the
>> >> > >> >> –EINVAL in the example) and send the bytes copied in
>> >> > >> >> coa_bytes_copied.
>> >> > >> >>
>> >> > >> >>    union offload_info4 switch (nfsstat4 coa_status) {
>> >> > >> >>    case NFS4_OK:
>> >> > >> >>            write_response4 coa_resok4;
>> >> > >> >>    default:
>> >> > >> >>            length4         coa_bytes_copied;
>> >> > >> >>    };
>> >> > >> >
>> >> > >> > Gah, I'm sorry, Olga said CB_OFFLOAD but for some reason I was
>> >> > >> > looking
>> >> > >> > at COPY.
>> >> > >> >
>> >> > >> > I have no idea what to think of that.  I don't see why it should
>> >> > >> > be
>> >> > >> > different from COPY.
>> >> > >>
>> >> > >> I'd like to resurrect this thread...
>> >> > >>
>> >> > >> This discussion was initiated to allow for the server to do a
>> >> > >> partial
>> >> > >> copy instead of failing the copy with EINVAL when copy is
>> >> > >> requested
>> >> > >> beyond the end of the file.
>> >> > >>
>> >> > >> However, the problem I'm running into is that because CB_OFFLOAD
>> >> > >> only
>> >> > >> returns "coa_bytes_copied" and does not include stable_how and
>> >> > >> verifier. So even if client were to then send a COMMIT for the
>> >> > >> bytes
>> >> > >> that were copied, it has no verifier to match from the COMMIT to
>> >> > >> anything.
>> >> > >>
>> >> > >> Should the client then ignore the lack of the matching verifier
>> >> > >> and
>> >> > >> just assume the data was committed for the partial bytes?
>> >> > >
>> >> > > Yes, sounds like in the asynchronous case the server has to wait
>> >> > > for
>> >> > > the
>> >> > > writes to reach disk before calling CB_OFFLOAD.  (Do we currently
>> >> > > do
>> >> > > that?).
>> >> >
>> >> > Why? No the spec does not require the server to do no such thing. The
>> >> > server sends back in CB_OFFLOAD if the bytes were written as UNSTABLE
>> >> > or FILE_SYNC.
>> >>
>> >> Look back at what you said before....  So the problem you were
>> >> explaining (missing stable_how and verifier) was only for the short
>> >> write case?
>>
>> Yes.
>>
>> >> So in the short write case it sounds like it *is* the server's
>> >> responsibility to commit.
>>
>> I disagree. I don't see anywhere in the spec that requires the server
>> to do that.
>>
>> >>   That's weird, but as long as the short write
>> >> case is rare, maybe it's not a big deal?
>> >>
>> >> Hm: "The coa_bytes_copied value indicates the number of bytes copied
>> >> but
>> >> not which specific bytes have been copied."  That makes
>> >> coa_bytes_copied
>> >> pretty much worthless.
>>
>> Yep that's what I thought.
>>
>> >> I guess I'd be in favor of using the NFS_OK branch of CB_OFFLOAD
>> >> whenever a nonzero number of bytes is copied.  If less than the
>> >> requested number of bytes were copied, the client can always find out
>> >> the error (if any) by attempting to copy the rest.
>>
>> Ok
>>
>> >>
>> >> --b.
>> >>
>> >> >
>> >> > > In theory that shouldn't be too expensive because asynchronous COPY
>> >> > > should normally be done with a very large range.
>> >> > >
>> >> > > (I wonder if the server should just do a synchronous copy if the
>> >> > > copy
>> >> > > would be less than a few megs?)
>> >> >
>> >> > Yes the server could determine to convert an asynchronous copy into a
>> >> > synchronous one based on size.
>> >> >
>> >> > I guess I'm still confused how to handle the implementation so that
>> >> > it
>> >> > doesn't kill interoperability with other possible (future)
>> >> > implementations...
>> >>
>> >> _______________________________________________
>> >> nfsv4 mailing list
>> >> [e-mail]
>> >> https://www.ietf.org/mailman/listinfo/nfsv4
>> >
>> >
>
>