Re: [nfsv4] draft-ietf-nfsv4-rpcrdma-bidirection-03 review

[Chuck Lever <[e-mail]>]

Hi Karen-

Thanks for the review. I'll respond below to questions
and non-editorial comments. I'll merge the other comments
before the next revision.


> On May 20, 2016, at 12:11 PM, Karen <[e-mail]> wrote:
> 
> Hi Chuck,
> 
> Here is some feedback on draft-ietf-nfsv4-rpcrdma-bidirection-03
> Karen
> 
> 
> Abstract section:
> 
> The use of the word "recent" seems relative to time.  5 years later, recent will no longer be recent.  Not sure how much of a big deal this is, just seems maybe instead of putting "recent minor versions of NFSv4", a more precise explanation would be "With the introduction of NFSv4.0".
> 
> 
> 2.1
> 
> "This traditional form of ONC RPC message passing is referred to as operation in the "forward direction"."
> 
> "as operation in the forward direction" seems awkward.  "is referred to as the "forward direction"." seems a bit more concise and clearer.
> 2.2 
> Same as in section 2.2 in "This form of message passin is referred to as operation in the "backward direction."
> 
> Also again the use of the word recently, as in "Not until recently", seems that "Not until NFSV4.0" would be clearer.
> 
> 3.1 
> "An NFSv4 "delegation" is simply a promise made by a server that it will notify a client before another agent is allowed access to a file"
> 
> another agent? seems "a different client" would be clearer.

A server may also host application programs, which can
open files too. When a local application opens a file
delegated to a client, the server would recall that
delegation.

So "agent" means other clients, or programs running on
the server.


> 3.2
> 
> "Without a backward direction RPC-over-RDMA capability, an NFSv4.1 client must additionally connect using a transport with backward direction capability to use as a backchannel."
> 
> I understand the point, and agree, but the sentence seems mumbled.  Maybe its confusing to me because of the use of the word backward direction.  Maybe backward direction needs to be defined as a connection that is initiated by a requestor, but also used to accept requests from the respondor, in which the requestor then reponds?  Not sure if that wording is any clearer, but the idea that the requests are sent from the side that did not create the connection?

Isn't the definition in section 2.2 sufficient?


> 4
> 
> "For an RDMA Send operation to work, the receiving peer must have posted an RDMA Receive Work Request (WR) to provide a receive buffer in which to land the incoming message.
> 
> why is the posted receive buffer being called a posted receive work request?  Seems this would be clearer 
> "For an RDMA Send operation to work, the receiving peer must have posted an RDMA receive buffer as the location in which the RDMA send request will land."

Each receive buffer is posted by posting a Receive Work
Request on the QP's receive queue. "Posting a receive
buffer" is a more colloquial way of saying it, posting
a Receive Work Request is perhaps more precise.

An argument could be made for switching to one terminology
or the other throughout the document. But which one is
more clear? Opinions?


> And similarly
> 
> 
> "If a receiver hasn't posted enough Receive WRs to land incoming Send operations..."
> 
> If a receiver hasn't posted enough receive buffers to accept incoming send requests"
> 
> (Why is Receive capitalized?)
> 
> 
> 4.1
> 
> "When message direction is not fully determined by context"
> 
> "fully determined by context" thats confusing, what does that really mean?  I think this means when the rdma header does not directly indicate if the message is a call or reply, but the wording is confusing.

That means that in some cases the receiver can guess
accurately which direction the message was going, based
on the context of the operation, even without having
an RPC message payload.

I'm open to suggestions.

I think some prefer that the document simply state that
direction is always unknown in cases where an RPC
message payload is not present. That kind of opens a
can of worms with RDMA_ERROR, which is needed to report
that the client does not support backward direction
operation.


> 4.2
> 
> "If a sender transmits a message for a receiver which has no prepared receive buffer"
> 
> the word "posted" not "prepared" was used earlier, to be consisted "posted" would read more clearly than "prepared"
> 
> 4.2.2 
> "A forward direction RPC-over-RDMA service endpoint"
> 
> 
> did you mean "server" instead of "service"?
> 
> "To receive incoming backward direction replies, an RPC-over-RDMA server endpoint must pre-post a receive buffer for each backward direction Call it sends"
> 
> What prevents a buggy, or hacking client from sending more requests that forward credits and preventing a reply in the backward direction from arriving?  Could a buggy or hacky client cause a denial of service by sending more than credit number of requests?  Guess that could happen in the standard single direction and would result in the connection closing.  Maybe not an issue?

The same issue exists in the forward direction.


> 5.1
> 
> last sentence, "and the header's msg_type field MUST conatin the value CALL", 
> 
> for more clarification "and the RPC header's msg_type field", this is a nit...
> 
> 5.4
> 
> "If an ONC RPC client has no work to do, it may be some time before it re-establishes a transport connection. Backward direction Requesters must be prepared to wait indefinitely for a connection to be established before a pending backward direction ONC RPC Call can be retransmitted."
> 
> I don't believe there are currently an non-idempotent backchannel operations, but if there ever were to be, seems that the client should detect if it has any drc replies and re-create the bi-directional, or even one way backchannel connection, right after it detects the connection is  gone.

Does backward direction RPC/TCP do this?

It certainly would be an issue for a backward-only connection.
The client would have to notice that the connection was down
and re-establish it quickly.


> Not addressed, and obvious to the implementer, but should it be mentioned that send buffers in the backward direction MUST be the same size as the posted receive buffers on the receiving side?  That is the backward connection must use the same send/recv buffer sizes and this can not be negotiated?

Well, the I-D used to say that the buffer sizes are the
same in both direction, but I guess I chopped it out at
some point. I'll have to go back and look for it.


--
Chuck Lever