Re: [nfsv4] Server-side copy question

[David Noveck <[e-mail]>]

>I would like to understand the comment by Bruce and you about not
> knowing which bytes were copied. I don't understand why it would not
> be the bytes from the offset to offset+bytes_copied? And Dave,
> wouldn't the "rest of the bytes" be the request length - bytes_copied?s
have been copied."

It might be and it is resonable if it is but RFC7862, says, as Bruce noted,
"The coa_bytes_copied value indicates the number of bytes copied but
not which specific bytes.".  I don't know wny it says that or how those
words came
to be in the spec.  If the spec did not say that, it would reasonable to
assume that it
would be the bytes from the offset to offset+bytes_copied.  But given
RFC7862,
as it is, I think clients cannot assume that.  It may be that these words
are just
plaiin wrong, but assuming that is risky, and fixing the spec to remove
them is very
difficult, even if everyone were to agree that they shouldn't be there.
,

On Tue, Oct 24, 2017 at 11:52 AM, Olga Kornievskaia <[e-mail]>
wrote:

> On Tue, Oct 24, 2017 at 7:43 AM, David Noveck <[e-mail]>
> wrote:
> >> So in the short write case it sounds like it *is* the server's
> >> responsibility to commit.
> >
> > Perhaps it should be the server's responsibility, but it is not,
> > according to RFC7862
> >
> >> That's weird,
> >
> > I agree that it is wierd.  I feel it is well-intentioned but not viable
> > since
> > you are essentially creating your own protocol, if clients depend on
> > this behavior an there is no point in doing it, if clients do not.
> >
> >> but as long as the short write
> >> case is rare, maybe it's not a big deal?
> >
> > From a performance point of view, it is not a big deal.  However it is
> > a big deal that the spec is broken. I hate when that happens :-(
> >
> >>Hm: "The coa_bytes_copied value indicates the number of bytes copied but
> >> not which specific bytes have been copied."  That makes coa_bytes_copied
> >> pretty much worthless.
> >
> > Any non-zero value of coa_bytes_copied is worthless because the client
> does
> > not know:
> >
> > Which particular bytes were copied
> > Whether those bytes were copied to the destination stably
> > How to commit any  non-stably-written bytes.
> >
> >>I guess I'd be in favor of using the NFS_OK branch of CB_OFFLOAD
> >> whenever a nonzero number of bytes is copied.
> >
> > That's likely to create an interoperability problem, since there are
> almost
> > certainly clients who assume that when there is no error, all requested
> > bytes
> > have been copied.
>
> I don't think it's true that clients would assume this. I think the
> client can only assume server copied as many bytes as it was specified
> in the CB_OFFLOAD bytes_copied regardless of how many bytes the client
> had asked for. At least that's how I read the spec.
>
> > Basically, the server is pretending there is no error and
>
> Yes, pretending that there was no error is the part I'm not totally
> comfortable with. And I agree with the comments that the client can
> try again to get the same error.
>
> > then
> > relying on the client to figure out that there was one, which could only
> be
> > done if the
> > spec tells the clients about this.  If RFC7862 doesn't do this, you wind
> up
> > essentially creating a private protocol.
> >
> >> If less than the
> >> requested number of bytes were copied, the client can always find out
> >> the error (if any) by attempting to copy the rest.
> >
> > Unfortunately there is no way of determining which bytes were copied so
> > there
> > is no way of knowing "the rest", i.e. the bytes not copied
>
> I would like to understand the comment by Bruce and you about not
> knowing which bytes were copied. I don't understand why it would not
> be the bytes from the offset to offset+bytes_copied? And Dave,
> wouldn't the "rest of the bytes" be the request length - bytes_copied?
> What am I missing?
>
> If you are suggested that the server is allowed to copy a requested
> range in any order of smaller chunks then there is a problem with a
> non-error case because we don't returned in COPY/CB_OFFLOAD offset and
> bytes_copied (just bytes_copied).
>
> I think we know which bytes are copied we just don't know how they
> were copied: stable or not (if the server chooses to specify an error
> in CB_OFFLOAD).
>
> > On Mon, Oct 23, 2017 at 9:36 PM, J. Bruce Fields <[e-mail]>
> > wrote:
> >>
> >> On Mon, Oct 23, 2017 at 05:43:42PM -0400, Olga Kornievskaia wrote:
> >> > On Fri, Oct 20, 2017 at 3:33 PM, J. Bruce Fields <
> [e-mail]>
> >> > wrote:
> >> > > On Wed, Oct 18, 2017 at 05:22:08PM -0400, Olga Kornievskaia wrote:
> >> > >> On Tue, Feb 28, 2017 at 12:32 PM, J. Bruce Fields
> >> > >> <[e-mail]> wrote:
> >> > >> > On Tue, Feb 28, 2017 at 05:04:27PM +0000, Adamson, Andy wrote:
> >> > >> >> I also thought that bytes copied is only included in the success
> >> > >> >> case – but as Olga pointed out, not true. The coa_status of
> NFS4_OK has the
> >> > >> >> coa_resok4->wr_count showing the bytes copied, and the default
> (e.g. failure
> >> > >> >> cases) has the coa_bytes_copied.
> >> > >> >>
> >> > >> >> So the server can send a CB_OFFLOAD with an error (such as the
> >> > >> >> –EINVAL in the example) and send the bytes copied in
> coa_bytes_copied.
> >> > >> >>
> >> > >> >>    union offload_info4 switch (nfsstat4 coa_status) {
> >> > >> >>    case NFS4_OK:
> >> > >> >>            write_response4 coa_resok4;
> >> > >> >>    default:
> >> > >> >>            length4         coa_bytes_copied;
> >> > >> >>    };
> >> > >> >
> >> > >> > Gah, I'm sorry, Olga said CB_OFFLOAD but for some reason I was
> >> > >> > looking
> >> > >> > at COPY.
> >> > >> >
> >> > >> > I have no idea what to think of that.  I don't see why it should
> be
> >> > >> > different from COPY.
> >> > >>
> >> > >> I'd like to resurrect this thread...
> >> > >>
> >> > >> This discussion was initiated to allow for the server to do a
> partial
> >> > >> copy instead of failing the copy with EINVAL when copy is requested
> >> > >> beyond the end of the file.
> >> > >>
> >> > >> However, the problem I'm running into is that because CB_OFFLOAD
> only
> >> > >> returns "coa_bytes_copied" and does not include stable_how and
> >> > >> verifier. So even if client were to then send a COMMIT for the
> bytes
> >> > >> that were copied, it has no verifier to match from the COMMIT to
> >> > >> anything.
> >> > >>
> >> > >> Should the client then ignore the lack of the matching verifier and
> >> > >> just assume the data was committed for the partial bytes?
> >> > >
> >> > > Yes, sounds like in the asynchronous case the server has to wait for
> >> > > the
> >> > > writes to reach disk before calling CB_OFFLOAD.  (Do we currently do
> >> > > that?).
> >> >
> >> > Why? No the spec does not require the server to do no such thing. The
> >> > server sends back in CB_OFFLOAD if the bytes were written as UNSTABLE
> >> > or FILE_SYNC.
> >>
> >> Look back at what you said before....  So the problem you were
> >> explaining (missing stable_how and verifier) was only for the short
> >> write case?
>
> Yes.
>
> >> So in the short write case it sounds like it *is* the server's
> >> responsibility to commit.
>
> I disagree. I don't see anywhere in the spec that requires the server
> to do that.
>
> >>   That's weird, but as long as the short write
> >> case is rare, maybe it's not a big deal?
> >>
> >> Hm: "The coa_bytes_copied value indicates the number of bytes copied but
> >> not which specific bytes have been copied."  That makes coa_bytes_copied
> >> pretty much worthless.
>
> Yep that's what I thought.
>
> >> I guess I'd be in favor of using the NFS_OK branch of CB_OFFLOAD
> >> whenever a nonzero number of bytes is copied.  If less than the
> >> requested number of bytes were copied, the client can always find out
> >> the error (if any) by attempting to copy the rest.
>
> Ok
>
> >>
> >> --b.
> >>
> >> >
> >> > > In theory that shouldn't be too expensive because asynchronous COPY
> >> > > should normally be done with a very large range.
> >> > >
> >> > > (I wonder if the server should just do a synchronous copy if the
> copy
> >> > > would be less than a few megs?)
> >> >
> >> > Yes the server could determine to convert an asynchronous copy into a
> >> > synchronous one based on size.
> >> >
> >> > I guess I'm still confused how to handle the implementation so that it
> >> > doesn't kill interoperability with other possible (future)
> >> > implementations...
> >>
> >> _______________________________________________
> >> nfsv4 mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/nfsv4
> >
> >
>