Re: [nfsv4] draft-ietf-nfsv4-rpcrdma-bidirection-03 review

[David Noveck <[e-mail]>]

On June 3, Chuck wrote:
> I concur that rfc5666bis and rfc5666-implementation-experience are ready
to move forward.

On June 9, Chuck wrote:
> Let's proceed with revision 05. I'm clearing my recent comment about the
readiness of rpcrdma-bidirection.

On June 9, Spencer wrote:
> Thanks, Chuck.  Will do.

Given the above, is there any reason that all of these documents are still
listed as "In WG Last Call"?.

I think we need to record the (successful) completion of WGLC before the
holiday weekend intervenes,
and then proceed to move these documents forward.   Chuck has already done
the hard part.  We need
to turn the administrative crank to get this work published. The whole
"Work in Progress" thing is getting
kind of old :-(


On Thu, Jun 16, 2016 at 4:13 PM, David Noveck <[e-mail]> wrote:

> Since Chuck has cleared his comment about bidirection and submitted -05, I
> think it is
> time to decide that WGLC has concluded (successfully) for
> draft-ietf-nfsv4-rfc5666bis
> draft-ietf-nfsv4-rfc5666-implementation, and
> draft-ietf-nfsv4-rpcrdma-bidirection.
>
> These should be able to move forward.
>
> On Thu, Jun 9, 2016 at 9:02 PM, Spencer Shepler <[e-mail]>
> wrote:
>
>>
>> Thanks, Chuck.  Will do.
>>
>> -----Original Message-----
>> From: Chuck Lever [mailto:[e-mail]]
>> Sent: Thursday, June 9, 2016 11:15 AM
>> To: Spencer Shepler <[e-mail]>
>> Cc: David Noveck <[e-mail]>; NFSv4 <[e-mail]>
>> Subject: Re: [nfsv4] draft-ietf-nfsv4-rpcrdma-bidirection-03 review
>>
>> Hi Spencer-
>>
>> Let's proceed with revision 05. I'm clearing my recent comment about the
>> readiness of rpcrdma-bidirection.
>>
>>
>>
>> > On Jun 3, 2016, at 12:20 PM, Spencer Shepler <[e-mail]>
>> wrote:
>> >
>> >
>> > Thanks for the discussion.  I will wait on you, Chuck, to "clear" the
>> comment you made below on readiness.
>> >
>> > Just to be clear, the WGLC is complete and from my point of view we
>> have consensus but want to ensure the author is comfortable with moving
>> forward.
>> >
>> > -----Original Message-----
>> > From: Chuck Lever [mailto:[e-mail]]
>> > Sent: Friday, June 3, 2016 8:05 AM
>> > To: David Noveck <[e-mail]>; Spencer Shepler
>> > <[e-mail]>
>> > Cc: NFSv4 <[e-mail]>
>> > Subject: Re: [nfsv4] draft-ietf-nfsv4-rpcrdma-bidirection-03 review
>> >
>> >
>> >> On Jun 3, 2016, at 10:33 AM, David Noveck <[e-mail]>
>> wrote:
>> >>
>> >> A week ago I wrote:
>> >>> The problem is that, in a multi-direction context, a given
>> >>> implementation only knows whether it is a client or a server.  It
>> >>> only knows if it is a requester or a responder based on the message
>> >>> it is processing at any given instant and if it can't figure that
>> out, it is unsure whether it is a requester or responder and may be neither
>> at particular times.
>> >>
>> >> It's now been at least two weeks since last call was supposed to end
>> >> for the Version One RDMA documents and a week since Chuck posted any
>> >> necessary updates.  I'm not sure why the documents are still listed as
>> in WGLC, but I wanted to make it clear that my comments above should not be
>> holding these up.
>> >>
>> >> Those comments refer to a problem that is inherent in Version One
>> >> and cannot be fixed within the constraints of a Version One
>> >> no-XDR-change update.  Since the Working group has decided to do a
>> >> Version One no-XDR-change update, the three existing RDMA documents
>> should now go forward.  I don;t believe there is any disagreement on that
>> point.
>> >
>> > I concur that rfc5666bis and rfc5666-implementation-experience are
>> ready to move forward.
>> >
>> > However, I've been thinking about an earlier comment Dave made in this
>> thread:
>> >
>> >> My only concern is with regard to a possible future extension in which
>> multiple RPC messages are carried in a single SEND (i.e., the precise
>> opposite of message continuation).  When the language for the new field is
>> drafted, we should make sure that it doesn't assume all the messages in a
>> SEND are all related to a single direction of operation.
>> >
>> >
>> > There seem to be two related issues, and one of them has ramifications
>> for rpcrdma-bidirection.
>> >
>> >
>> > I. Ambiguity of the meaning of the credit field
>> >
>> > In the rfc5666bis world, the credit field in all messages flowing on a
>> connection had an unambiguous meaning. If the message was going from
>> requester to responder, the credit field was a credit request. In the other
>> direction, it was a credit grant.
>> >
>> > rpcrdma-bidirection introduced the idea of two separate credit flows,
>> which depend on whether the message was part of forward RPC operation or
>> backward RPC operation on that connection.
>> >
>> > This is the source of the problem.
>> >
>> > The meaning of the credit field depends on a field in the upper layer,
>> so it's a layering violation, to say the least.
>> >
>> > Also, the original concept of credit was to manage RDMA receive
>> buffers, but rpcrdma-bidirection interpretation of the credit value is
>> one-credit-per-RPC.
>> >
>> > Now if we want to add support for partial RPC message per credit
>> (message continuation) or multiple RPC messages per credit (as proposed
>> above), or NO RPC message per credit (for RDMA2_OPTIONAL control messages)
>> the credit field is unusable.
>> >
>> > Would we add an independent pool of credits for each of these
>> transmission mechanisms? Probably not.
>> >
>> > A logical course of action, then, would be to alter the
>> rpcrdma-bidirection I-D so that forward and backward direction operation
>> use the same pool of credits.
>> >
>> > The question is whether it is feasible for the two directions to share
>> the credit pools without deadlocking.
>> > Some prototyping and/or careful thought would be needed to answer it.
>> >
>> > (Spencer, I may be changing my mind about the readiness of
>> rpcrdma-bidirection).
>> >
>> >
>> > II. Ambiguity of the meaning of the XID field
>> >
>> > Today we have a one-to-one match between the XID value in the
>> RPC-over-RDMA header and the XID associated with the RPC payload in that
>> message.
>> >
>> > With an extension for message continuation, there is still a one-to-one
>> match: the RPC-over-RDMA XID can match the XID of the partial RPC payload.
>> >
>> > But for multiple RPC payloads per Send, or no RPC payloads per Send
>> (control messages), we no longer have a match. How should the RPC-over-RDMA
>> header's XID field be set in those cases?
>> >
>> > RFC 5666 and rfc5666bis have similar language regarding how the XID
>> field in the RPC-over-RDMA header is to be set. Here's rfc5666bis, Section
>> 5.2.1:
>> >
>> >>   The
>> >>   receiver MAY perform its processing based solely on the XID in the
>> >>   RPC-over-RDMA header, and thereby ignore the XID in the RPC message,
>> >>   if it so chooses.
>> >
>> >
>> > In other words, implementations are already allowed to rely on the
>> value of the RPC-over-RDMA header's XID field, and ignore the XID field in
>> the payload.
>> >
>> > This issue has to be addressed for rpcrdma-version-two to support
>> extensions that enable multiple or no RPCs per message.
>> >
>> > I propose that here, an independent XID space for RDMA2_OPTIONAL
>> messages makes sense. This would allow receivers to match RDMA2_OPTIONAL
>> call and reply messages, but would not tie the header's XID value to the
>> upper layer payload in any way.
>> >
>> >
>> > --
>> > Chuck Lever
>> >
>> >
>> > _______________________________________________
>> > nfsv4 mailing list
>> > [e-mail]
>> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fwww.i
>> > etf.org%2fmailman%2flistinfo%2fnfsv4&data=01%7c01%7csshepler%40microso
>> > ft.com%7cc7fb211893674020da6d08d39091ec90%7c72f988bf86f141af91ab2d7cd0
>> > 11db47%7c1&sdata=KiMVRSXc2Sj8%2blfR0vjWLJFQ4KAYKx95%2bMkReAuNQjA%3d
>>
>> --
>> Chuck Lever
>>
>>
>>
>>
>