Re: [nfsv4] draft-ietf-nfsv4-rpcrdma-bidirection-03 review

[David Noveck <[e-mail]>]

Since Chuck has cleared his comment about bidirection and submitted -05, I
think it is
time to decide that WGLC has concluded (successfully) for
draft-ietf-nfsv4-rfc5666bis
draft-ietf-nfsv4-rfc5666-implementation, and
draft-ietf-nfsv4-rpcrdma-bidirection.

These should be able to move forward.

On Thu, Jun 9, 2016 at 9:02 PM, Spencer Shepler <[e-mail]>
wrote:

>
> Thanks, Chuck.  Will do.
>
> -----Original Message-----
> From: Chuck Lever [mailto:[e-mail]]
> Sent: Thursday, June 9, 2016 11:15 AM
> To: Spencer Shepler <[e-mail]>
> Cc: David Noveck <[e-mail]>; NFSv4 <[e-mail]>
> Subject: Re: [nfsv4] draft-ietf-nfsv4-rpcrdma-bidirection-03 review
>
> Hi Spencer-
>
> Let's proceed with revision 05. I'm clearing my recent comment about the
> readiness of rpcrdma-bidirection.
>
>
>
> > On Jun 3, 2016, at 12:20 PM, Spencer Shepler <[e-mail]>
> wrote:
> >
> >
> > Thanks for the discussion.  I will wait on you, Chuck, to "clear" the
> comment you made below on readiness.
> >
> > Just to be clear, the WGLC is complete and from my point of view we have
> consensus but want to ensure the author is comfortable with moving forward.
> >
> > -----Original Message-----
> > From: Chuck Lever [mailto:[e-mail]]
> > Sent: Friday, June 3, 2016 8:05 AM
> > To: David Noveck <[e-mail]>; Spencer Shepler
> > <[e-mail]>
> > Cc: NFSv4 <[e-mail]>
> > Subject: Re: [nfsv4] draft-ietf-nfsv4-rpcrdma-bidirection-03 review
> >
> >
> >> On Jun 3, 2016, at 10:33 AM, David Noveck <[e-mail]> wrote:
> >>
> >> A week ago I wrote:
> >>> The problem is that, in a multi-direction context, a given
> >>> implementation only knows whether it is a client or a server.  It
> >>> only knows if it is a requester or a responder based on the message
> >>> it is processing at any given instant and if it can't figure that out,
> it is unsure whether it is a requester or responder and may be neither at
> particular times.
> >>
> >> It's now been at least two weeks since last call was supposed to end
> >> for the Version One RDMA documents and a week since Chuck posted any
> >> necessary updates.  I'm not sure why the documents are still listed as
> in WGLC, but I wanted to make it clear that my comments above should not be
> holding these up.
> >>
> >> Those comments refer to a problem that is inherent in Version One
> >> and cannot be fixed within the constraints of a Version One
> >> no-XDR-change update.  Since the Working group has decided to do a
> >> Version One no-XDR-change update, the three existing RDMA documents
> should now go forward.  I don;t believe there is any disagreement on that
> point.
> >
> > I concur that rfc5666bis and rfc5666-implementation-experience are ready
> to move forward.
> >
> > However, I've been thinking about an earlier comment Dave made in this
> thread:
> >
> >> My only concern is with regard to a possible future extension in which
> multiple RPC messages are carried in a single SEND (i.e., the precise
> opposite of message continuation).  When the language for the new field is
> drafted, we should make sure that it doesn't assume all the messages in a
> SEND are all related to a single direction of operation.
> >
> >
> > There seem to be two related issues, and one of them has ramifications
> for rpcrdma-bidirection.
> >
> >
> > I. Ambiguity of the meaning of the credit field
> >
> > In the rfc5666bis world, the credit field in all messages flowing on a
> connection had an unambiguous meaning. If the message was going from
> requester to responder, the credit field was a credit request. In the other
> direction, it was a credit grant.
> >
> > rpcrdma-bidirection introduced the idea of two separate credit flows,
> which depend on whether the message was part of forward RPC operation or
> backward RPC operation on that connection.
> >
> > This is the source of the problem.
> >
> > The meaning of the credit field depends on a field in the upper layer,
> so it's a layering violation, to say the least.
> >
> > Also, the original concept of credit was to manage RDMA receive buffers,
> but rpcrdma-bidirection interpretation of the credit value is
> one-credit-per-RPC.
> >
> > Now if we want to add support for partial RPC message per credit
> (message continuation) or multiple RPC messages per credit (as proposed
> above), or NO RPC message per credit (for RDMA2_OPTIONAL control messages)
> the credit field is unusable.
> >
> > Would we add an independent pool of credits for each of these
> transmission mechanisms? Probably not.
> >
> > A logical course of action, then, would be to alter the
> rpcrdma-bidirection I-D so that forward and backward direction operation
> use the same pool of credits.
> >
> > The question is whether it is feasible for the two directions to share
> the credit pools without deadlocking.
> > Some prototyping and/or careful thought would be needed to answer it.
> >
> > (Spencer, I may be changing my mind about the readiness of
> rpcrdma-bidirection).
> >
> >
> > II. Ambiguity of the meaning of the XID field
> >
> > Today we have a one-to-one match between the XID value in the
> RPC-over-RDMA header and the XID associated with the RPC payload in that
> message.
> >
> > With an extension for message continuation, there is still a one-to-one
> match: the RPC-over-RDMA XID can match the XID of the partial RPC payload.
> >
> > But for multiple RPC payloads per Send, or no RPC payloads per Send
> (control messages), we no longer have a match. How should the RPC-over-RDMA
> header's XID field be set in those cases?
> >
> > RFC 5666 and rfc5666bis have similar language regarding how the XID
> field in the RPC-over-RDMA header is to be set. Here's rfc5666bis, Section
> 5.2.1:
> >
> >>   The
> >>   receiver MAY perform its processing based solely on the XID in the
> >>   RPC-over-RDMA header, and thereby ignore the XID in the RPC message,
> >>   if it so chooses.
> >
> >
> > In other words, implementations are already allowed to rely on the value
> of the RPC-over-RDMA header's XID field, and ignore the XID field in the
> payload.
> >
> > This issue has to be addressed for rpcrdma-version-two to support
> extensions that enable multiple or no RPCs per message.
> >
> > I propose that here, an independent XID space for RDMA2_OPTIONAL
> messages makes sense. This would allow receivers to match RDMA2_OPTIONAL
> call and reply messages, but would not tie the header's XID value to the
> upper layer payload in any way.
> >
> >
> > --
> > Chuck Lever
> >
> >
> > _______________________________________________
> > nfsv4 mailing list
> > [e-mail]
> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fwww.i
> > etf.org%2fmailman%2flistinfo%2fnfsv4&data=01%7c01%7csshepler%40microso
> > ft.com%7cc7fb211893674020da6d08d39091ec90%7c72f988bf86f141af91ab2d7cd0
> > 11db47%7c1&sdata=KiMVRSXc2Sj8%2blfR0vjWLJFQ4KAYKx95%2bMkReAuNQjA%3d
>
> --
> Chuck Lever
>
>
>
>