Re: [nfsv4] draft-ietf-nfsv4-rpcrdma-bidirection-03 review

[Chuck Lever <[e-mail]>]

> On Jun 6, 2016, at 1:10 PM, David Noveck <[e-mail]> wrote:
> 
> >> The responsibility for defining the basic structure of bidirectional operation belongs to rpcrdma-bidirection.  
> >> This includes the use of two separate credit pools and the fact each message may contain either a request or a grant applying to the credit pool owned by the message receiver.
> 
> > OK: the receiver has to decide whether each message
> > contains a credit request (the receiver is acting as
> > a responder) or a credit grant (the receiver is
> > acting as a requester).
> 
> Yes but I'd boil this down to the receiver deciding whether the credit field is a request or a grant (and the
> RPC-over-RDMA version telling him how to decide.
> 
> The place where this gets "interesting" is a subcase of the feature you brought up.  If a transmission can
> contain multiple rpc messages, it can be the case the sender would want to send both requests and replies
> and thus want/need to send both a request and a grant.
> 
> I think we want to be careful that, where we specify how the credit field is to be used, we don't foreclose
> passing credits or grants in other fields.    If an XMOPT_GROUP were defined, that allowed a group of
> RPC messages to be sent, the version two sender would make the direction field reflect the majority, but
> I would want it to have the option to have an other-credit field so it could convey a request and a grant in
> the same transmission. 
> 
> > I might refine the text of rpcrdma-bidirection in this
> > area, but otherwise I don't think there's a significant
> > change needed.
> 
> there might be minor text tweaking required.
> 
> 
> > >       • The responsibility for defining how it is to be determined, whether any particular message has a grant or a request in the credit field is the responsibility of the specific RPC-over-RDMA version being used.
> 
> > I might add this suggestion to rpcrdma-bidrection.
> 
> OK, bu so far I hven't seen any obvious place to put it.
> 
> 
> > >       • The responsibility of deciding whether bidirectional operation is OPTIONAL, REQUIRED, or not allowed is the responsibility of the specific RPC-over-RDMA version being used.
> 
> > I'm uncertain how that aligns with rfc5666bis, which
> > does not mention backwards direction operation at all.
> > This implies that it is allowed, and not REQUIRED. Is
> > that enough?
> 
> I think what is in 5666bis is fine.  Section 8.1 introduces it as a
> potential conventional extension and references rpcrdma-bidirection. 
> 
> 
> >>       • The text that now defines how the credit field for specfic Version One messages is interpreted as illustrative of version one only.
> 
> > Is a text change needed for this, or are you simply
> > stating an operating assumption for spec writers?
> 
> I was thinking of rewriting the last paragraph of section 4.1 of rpcrdma-bidirection to read as follows:
> 
> When message direction is not fully determined by context or by an accompanying RPC message with a call direction field, it may not be clear whether the header credit value is a request or grant.  In such cases the definition of the RPC-over-RDMA version used may provide guidance as to how the credit value is to be interpreted. In such cases in which such guidance is not provided or is adequate, the receiver MUST NOT use the header's credit value.
> I didn't see anything else that need to change.


OK, here's how rpcrdma-bidirection Section 4.1 reads now:

> 4.1.  Backward Credits
> 
>    RPC-over-RDMA credits work the same way in the backward direction as
>    they do in the forward direction.  However, forward direction credits
>    and backward direction credits on the same connection are accounted
>    separately.
> 
>    The forward direction credit value retains the same meaning whether
>    or not there are backward direction resources associated with an RPC-
>    over-RDMA transport connection.  This is the number of RPC requests
>    the forward direction responder (the RPC server) is prepared to
>    receive concurrently.
> 
>    The backward direction credit value is the number of RPC requests the
>    backward direction responder (the RPC client) is prepared to receive
>    concurrently.  Observe that the backward direction credit value MAY
>    be different than the forward direction credit value.
> 
>    During bi-directional operation, each receiver has to decide whether
>    an incoming message contains a credit request (the receiver is acting
>    as a responder) or a credit grant (the receiver is acting as a
>    requester) and apply the credit value accordingly.
> 
>    When message direction is not fully determined by context (e.g.,
>    suggested by the definition of the RPC-over-RDMA version that is in
>    use) or by an accompanying RPC message payload with a call direction
>    field, it is not possible for the receiver to tell with certainty
>    whether the header credit value is a request or grant.  In such
>    cases, the receiver MUST NOT use the header's credit value.



--
Chuck Lever