Re: [nfsv4] Review of draft-ietf-nfsv4-rfc5667bis-06

[David Noveck <[e-mail]>]

>  As far as I know the extant implementations do
> not support this construction, and we have to
> preserve interoperation with them.

I can see that the relevant factor is the need
to preserve interoperability, which is just as well,
since, whenever we try to interpret what RFC5667
really meant, my head starts to hurt.

If the reason for forbidding something is driven by
interoperability concerns, it would make the spec
more clear if this were said explicitly

> That reasoning could also apply in the NFSv4
> case, where it is almost a certainty that typical
> NFSv4 WRITE requests can be larger than the
> inline threshold.

Perhaps so but note that section 5.4.1 of -06 says:

Therefore, an NFS version 4 client MAY send a reduced Payload stream
in a Long Call.  An NFS version 4 client MAY enable an NFS version 4
server to send a reduced Payload stream in a Long Reply.

This wasn't in -05. It doesn't seem to be needed, but if it is wrong,
simply deleting it would not be enough. You might have to put in
something forbidding it, as is done in the legacy case.




On Sun, Feb 26, 2017 at 3:29 PM, Chuck Lever <[e-mail]> wrote:

>
> > On Feb 25, 2017, at 3:54 PM, David Noveck <[e-mail]> wrote:
> >
> > > RFC 5667 Section 4 says:
> >
> > > > Similarly, a single RDMA Read list entry MAY be posted by the client
> > > > to supply the opaque file data for a WRITE request or the pathname
> > > > for a SYMLINK request.
> >
> > Part of the problem here is that, as you discuss later, this statement is
> > ambiguous, as the meaning of "read list entry" is not clear.
> >
> > > > The server MUST ignore any Read list for
> > > > other NFS procedures,
> >
> > As I understand it, this statement cannot apply to PZRCs, and rfc5666bis
> > has already dealt with that issue.  So, if one tried to maintain this
> paragraph,
> > in something like the RFC5667-form, some modification would have been
> > necessary to avoid essentially preventing any use of PZRCs
> >
> > > > as well as additional Read list entries beyond
> > > > the first in the list.
> >
> > > I take "Read list entry" to mean Read chunk, composed of
> > > multiple list entries that share the same XDR position.
> > > This comports with similar language describing Write
> > > chunks where a single list entry is indeed allowed to
> > > have multiple segments.
> >
> > Makes sense to me.
> >
> > > However, the original intent might have been "single
> > > Read segment".
> >
> > It might have been but there is no way to be sure.
>
> We can ask Tom Talpey. If he does not recall, then
> we have no way to be sure.
>
>
> > In some
> > cases, one might look to how existing implementations behave,
> > to come up with an interpretation even though it might not
> > have been what the author intended.
> >
> > > In that case, this language clearly states
> > > that Legacy clients can send only one contiguous memory
> > > region
> >
> > I think you mean that,  if that were the case, this language would
> > mean that.
> >
> > > in Legacy NFS READ and SYMLINK requests.
> >
> > I think we are talking about WRITE and SYMLINK, given
> > that read chunks are mentioned.
> >
> > While I can easily believe that existing implementations
> > limit SYMLINKs to a single contiguous buffer, I have trouble
> > believing the same about WRITEs.
>
> All server implementations I'm aware of allow WRITE
> payloads with multiple segments. That suggests the
> former interpretation is correct.
>
>
> > > At your behest I replaced the RFC 5667 language with
> > > just the definitions of which XDR data items are DDP
> > > eligible, and removed the discussion about Read list
> > > entries.
> >
> > One benefit of that is that we now know what the spec says, while,
> > as with the section 4 text you quoted, we weren't.
> >
> > > This new language permits a Legacy client to reduce the
> > > Payload stream of an NFS READ or SYMLINK request and put
> > > that stream into a Position-Zero Read chunk. The client
> > > would then send the PZRC and the non-zero position chunk:
> > > two distinct Read chunks, which is not allowed by the
> > > RFC 5667 language discussing NFS READ or SYMLINK.
> >
> > It is not clear that section 4 would disallow that.  Beyond the
> > uncertainty about what "read list entry" means and a possible need
> > to treat PZRCs separately as done in rfc5666bis, the issue is that
> > you are only supposed to ignore "additional Read list entries beyond
> > the first in the list."  To me, this would allow a PZRC, if it is the
> first in the
> > list, which it might well be.
>
> A PZRC is allowed. A non-zero position Read chunk is
> allowed. What might not be allowed is both at the
> same time.
>
>
> > Anyway, your view and my view of this obscure text doesn't really
> > count.  We have to work off the behavior of implementations.  If existing
> > implementations did or didn't do something because of this presumed
> > restriction, that would provide an answer.  Unfortunately, I don't think
> this
> > provides an answer either,
>
> At least Linux does not currently support a reduced
> Payload stream in a PZRC. Solaris is not likely to
> either.
>
> Either the original implementers interpreted the
> original specification as not allowing this
> construction, or so far there hasn't been a need
> to support it.
>
>
> > > To remain interoperable with RFC 5667-compliant
> > > implementations, only one Read chunk is allowed, which
> > > excludes the use of reduced Payload streams in a PZRC,
> > > even though RPC-over-RDMA Version One allows this
> > > construction.
> >
> > True but it is not clear to me me how this situation could ever
> > arise given:
> >       • The XDR of legacy NFS protocols
> >       • The definition of what is DDP-eligible in the ULB
>
> The situation MAY arise because it is not specifically
> excluded by the new language.
>
> Unless the Transport header is exceptionally large
> or there is exceptional authentication material,
> a reduced NFS WRITE or SYMLINK can always fit inline.
>
> However, a client MAY still choose to reduce the
> Payload stream and send it in a PZRC, even if it
> could be sent inline. There is nothing that says
> an implementation cannot use RDMA_NOMSG for all
> transactions.
>
> The point here is that if an NFSv3 WRITE is large,
> there are only two legal ways to send it:
>
>  - Reduce the payload into a Read chunk and send
> the rest inline
>
>  - Send the whole request in a PZRC (RDMA_NOMSG)
>
>
> > > There isn't a need to support sending a reduced Payload
> > > stream this way in these NFS versions, so I added the new
> > > paragraph to forbid it explicitly.
> >
> > If there is no way for this situation to arise, there is no need to
> > forbid it explicitly.
>
> There is a need to forbid it explicitly:
>
> - The new language definitely does not forbid this
> construction.
>
> - It is possible a client might use this
> construction for reasons we can't predict.
>
> - As far as I know the extant implementations do
> not support this construction, and we have to
> preserve interoperation with them.
>
> That reasoning could also apply in the NFSv4
> case, where it is almost a certainty that typical
> NFSv4 WRITE requests can be larger than the
> inline threshold.
>
> --
> Chuck Lever
>
>
>
>