Re: [nfsv4] pNFS file and use of RPC-over-RDMA to access an NFSv4.1 data server

Tom Talpey <tom@talpey.com> Fri, 27 April 2018 12:51 UTC

Return-Path: <tom@talpey.com>
X-Original-To: nfsv4@ietfa.amsl.com
Delivered-To: nfsv4@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AA4A0120713 for <nfsv4@ietfa.amsl.com>; Fri, 27 Apr 2018 05:51:07 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3QJpVB_bUm_z for <nfsv4@ietfa.amsl.com>; Fri, 27 Apr 2018 05:51:05 -0700 (PDT)
Received: from p3plsmtpa11-07.prod.phx3.secureserver.net (p3plsmtpa11-07.prod.phx3.secureserver.net [68.178.252.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DCC7F1200F1 for <nfsv4@ietf.org>; Fri, 27 Apr 2018 05:51:05 -0700 (PDT)
Received: from [192.168.0.55] ([24.218.182.144]) by :SMTPAUTH: with SMTP id C2q4fchImmOqYC2q7fMDoQ; Fri, 27 Apr 2018 05:51:04 -0700
To: nfsv4@ietf.org
References: <CADaq8jfcEa3xz1UNwrWkcvLZAr=eHKzQKppR+Kuqbzyq8H2ueQ@mail.gmail.com> <57F19FA8-2491-4CAB-B3B0-09A194BF09F9@oracle.com> <MWHPR2101MB0809103188F9DDFECCCB7A3CA0B60@MWHPR2101MB0809.namprd21.prod.outlook.com> <CADaq8jdqitBPXond=K3kZu7mzc0+uWuZ1BTn7pxN9TBsUgR9LA@mail.gmail.com> <22B3EA02-3085-41FF-9393-EB827235CB1A@oracle.com> <094be1f2-cdd7-4b1f-d059-a4017ccfa9d3@talpey.com> <CE03DB3D7B45C245BCA0D24327794936300F7FAE@MX307CL04.corp.emc.com>
From: Tom Talpey <tom@talpey.com>
Message-ID: <d156a5d0-9a57-9d71-13e4-3d0ed47a4e7d@talpey.com>
Date: Fri, 27 Apr 2018 08:50:59 -0400
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0
MIME-Version: 1.0
In-Reply-To: <CE03DB3D7B45C245BCA0D24327794936300F7FAE@MX307CL04.corp.emc.com>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-CMAE-Envelope: MS4wfOAIoGjvcqo7nz+Awvt876UGbsrbTWiZFYgB1RsRHOHR4O2T/hwoebVdP01Ma2gH/ueEkRzEpe+8YBgruLTtNwBCzef0qvF9JxXfODn0bgmMcKC0YsbB W36hQXEa/KuEVA1/zvCuKkAF9NrJyhH4a8bPcLv2Auw+7UzYaNsdah7T
Archived-At: <https://mailarchive.ietf.org/arch/msg/nfsv4/x5Aq99FKGfiItOAm4mI4SL41Peg>
Subject: Re: [nfsv4] pNFS file and use of RPC-over-RDMA to access an NFSv4.1 data server
X-BeenThere: nfsv4@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: NFSv4 Working Group <nfsv4.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/nfsv4/>
List-Post: <mailto:nfsv4@ietf.org>
List-Help: <mailto:nfsv4-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 27 Apr 2018 12:51:08 -0000

On 4/26/2018 11:40 AM, Black, David wrote:
>> On 4/25/2018 3:30 PM, Chuck Lever wrote:
>>> NFSv4.1 has a "gear shift" capability where, after establishment
>>> of a TCP connection, the client and server detect that an RDMA
>>> connection is possible. At this point, I'm not aware of any
>>> implementation of this capability. Are we as a community deciding
>>> to abandon this approach for NFS?
>>
>> I hope not! Note, the iSCSI/RDMA mapping, iSER, has a similar
>> capability in its "RDMAExtensions" key at logon.
>>
>> But there are some challenges. RoCE and InfiniBand only support an
>> RDMA connection, and can't perform the step-up. And while the iWARP
>> (specifically, MPAv2) protocol does, the local APIs to drive the shift
>> are immature.
> 
> When the revised iSER RFC 7145 was published in 2014, there were no iSER implementations that supported this "gear shift" functionality.
> 
> The original iSER RFC 5046 required "gear shift" ... but implementations ignored that requirement.

Yes, that's a valid point, and the iSCSI "Portal Group" could be a
framework to reference. It provides some of the necessary cookies
for the client/initiator to select a protocol.

However, it does require external side protocols for discovery, and
most importantly requires a single port to serve a single protocol.
This leads to requiring more than one logical or physical interface,
when a target/server wishes to serve both RDMA and other transports.
Step-up/gear shift mode supports a single interface, seamlessly.

Tom.


>> I believe the iSER and NFS/RDMA communities could work together to
>> enable this type of interaction. But as you point out, these are
>> implementation questions. The protocols are in place.
>>
>> Tom.
>>
>> _______________________________________________
>> nfsv4 mailing list
>> nfsv4@ietf.org
>> https://www.ietf.org/mailman/listinfo/nfsv4
> 
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4
> 
>