IETF Logo Mail Archive

[nfsv4] questions about flow control

Chuck Lever III <chuck.lever@oracle.com> Thu, 29 April 2021 18:16 UTC

Return-Path: <chuck.lever@oracle.com>
X-Original-To: nfsv4@ietfa.amsl.com
Delivered-To: nfsv4@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CF3E93A11A0; Thu, 29 Apr 2021 11:16:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level:
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=oracle.com header.b=XA4fYV+x; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.b=kJEx6DIl
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qwH5FiZzNEoP; Thu, 29 Apr 2021 11:16:55 -0700 (PDT)
Received: from aserp2130.oracle.com (aserp2130.oracle.com [141.146.126.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5F8BC3A11A1; Thu, 29 Apr 2021 11:16:54 -0700 (PDT)
Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 13TIFCvR136833; Thu, 29 Apr 2021 18:16:51 GMT
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : content-type : content-id : content-transfer-encoding : mime-version; s=corp-2020-01-29; bh=XpJyMJa4E877HELjT5oyu45S5/f2FjM6mZ+G3quxedI=; b=XA4fYV+xU6rIOYchaGtzPt25fXR8LcdFyJnhH2bumIQmSLZUtSKXu5va735IX98SXFWx 4cJX6+rUvH5BuQ3fYYtA9r51+bnBGwVrGNdDBkZ6lbEd3FTop6sZ93AOKJmzu+xmwiBR O2rC5vJ/2ZKLEK1xZ/oVhyoFURRm6Ea+ByHPDQhWd8o8qiIlolTTCmUnRNYOAAt/hj4v LWQzuxA8sLldO5qtr8crDlZ52KOrlRW+6laIhWEMD3Of3DD2YbLXh36hwS9QAZbVxAxT eYbG78C6XSU/0nWXxZcc1Heu5Uu7PY+JBrxOeTzwfFPpOg4bIAFcLZVUIC2F4WYtDAoo yg==
Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2130.oracle.com with ESMTP id 385afq5dqd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 29 Apr 2021 18:16:51 +0000
Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 13TIAlIX098239; Thu, 29 Apr 2021 18:16:50 GMT
Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2170.outbound.protection.outlook.com [104.47.59.170]) by userp3020.oracle.com with ESMTP id 384w3wnsx3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 29 Apr 2021 18:16:50 +0000
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dObAu2KZF0skPc7ZdyeKbxyLE7QXULjRcyVTx988dXwCWTEqqaf5jJt9BOfOW8OVcHjPPseaxF7dxpXvFtIzxOx646EWSVxufUbFf6v1NUSGRkRh2+sUWjcnspGZ9o3q2ZBtMyf6brm3mXi1+hayX1Jilk2sxlprnzkDYZgS+m9aCSQJRdPAriQvPGJMhd5M5Y2PlWD9oqJZoIHzUjBfgdc2D5lAS/GnpyxHV76jwnhUz/pAqBcagxy3dXWRjeWfrn5PcdcqP51rD7A4KB3xnI8GkWQJamULngkOjr1v0braU3NHVdshWeWzluFiKKbduwZALpRSD9O5SGi66yX+mw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XpJyMJa4E877HELjT5oyu45S5/f2FjM6mZ+G3quxedI=; b=Lq/TlesxcJI7ks2iwcNPqu5OlWiU1v+I3sM52qyOK4P2o5YBAwyJkafigprXzuCDVHAdjxBf3yHy9qD1r3IRQY8HLji7yp8bc6TD5zGa3J1NNIQz0TeNr69Z9dORx2St06eopuSj84XBBHmDKeKHGPPryYuFVMKAidZF5+3k8dvBCaWOQ0uUayvH8NPmGCY9FaHD1W47k8ttvfqLiHrDsyNQWRPxD5zNCkLZRBZ35vOZJFNSdAexX5RqqNLRy4q3iJmn5cX1mn/jrNMUt4KLT7H4rZ1rQwwwMA1PLhNs3AqJ1V5OxEPmLC4e6DaG9nlaf2KLB7UePsFp45L+zh8HyA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XpJyMJa4E877HELjT5oyu45S5/f2FjM6mZ+G3quxedI=; b=kJEx6DIlyHWy9vFKlw6kAgorSpJn3VcwEAikQaDa5pRH7YJqFiz/unkzP3cs2pG091UmbNKbTIIRwVQgb3ZprhFXm5AVqhsV7PYGnCne/TWHWiMlBnADeHfeFICYKjwGqbVfdTdiMI7QEKg6WHQgX4OjrKcA75pOaltyuSPF1WI=
Received: from SJ0PR10MB4688.namprd10.prod.outlook.com (2603:10b6:a03:2db::24) by SJ0PR10MB4510.namprd10.prod.outlook.com (2603:10b6:a03:2d6::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22; Thu, 29 Apr 2021 18:16:48 +0000
Received: from SJ0PR10MB4688.namprd10.prod.outlook.com ([fe80::50bf:7319:321c:96c9]) by SJ0PR10MB4688.namprd10.prod.outlook.com ([fe80::50bf:7319:321c:96c9%4]) with mapi id 15.20.4065.027; Thu, 29 Apr 2021 18:16:48 +0000
From: Chuck Lever III <chuck.lever@oracle.com>
To: NFSv4 <nfsv4@ietf.org>, "nfsv4-ads@ietf.org" <nfsv4-ads@ietf.org>
Thread-Topic: questions about flow control
Thread-Index: AQHXPSPRfQXNwmK1sUuTiIL+uA/nAQ==
Date: Thu, 29 Apr 2021 18:16:48 +0000
Message-ID: <EAF8A6D7-5E55-4078-945A-8DA8CF28496D@oracle.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=oracle.com;
x-originating-ip: [68.61.232.219]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: bfac4f11-4e46-4a5f-07b5-08d90b3af42a
x-ms-traffictypediagnostic: SJ0PR10MB4510:
x-microsoft-antispam-prvs: <SJ0PR10MB45100A08A648BF10C2A73758935F9@SJ0PR10MB4510.namprd10.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: Q7Bs1Zrgmd9NZWDzjVlFDXfyUIuE+HX+kl0cANjX9v1VDPMyReFST2GqGZprMW0t1vo+3dfValr0IRZPzXEf+e2Z1a8w3/Y2ncU+7HBdvAncJPjj0AYpRySDNZ+Ul4VBJtW06ka6Gcnl1N3yrrUr7/m5KFFjH7Wn28Ftz8QORkhTtqmf9yO+3KOAw1uy3MEdVSKW5Ro3s+GjFwhn+tkvR+/nncyXhav0TwEU6xYPiy/J5erndHactToREmXWrOA+ASAdZr0mcEIN7sgEfywWsL/f1MkXj9t81YOiLtakzsbdyFLy8MiRcHvCSSUbfjvShoaO1gjESWvlTveHHdmkBUDqlslj/Mx6BqOmny0qz9LB7T1moExa9Tln9jPYAiuFCMWNAQHZ+A08wc4j6QaSwucQpFHclienATrXYr22giPPBEQTude5PrM2pV1jtHmsSNustzrgA0tsFS2CcxSB3l5saqNs6TGvcYV7tRY+fhrF3fupb5R7vERmf9KlK5qgeqTnK7004ODgqiVfKqY1WVuO/abxObxtIB3nI/X2yCOS89NNXrsHBC2W+Nr/zqdC7hr33rJggTZVea0QqjOAecLTeI5bBZ9D8wq/07rIoAbIWsmOK6vpVjmm/mVwyey4pDfrHe0yMP0PBwoLXSbQjhDIOaxbwZ/ysJnFRPyMI4s0jhxef4NHd3D1892Tz1Aj12L2dIZXpPo5oWphjzfgF17ItUAqDddRGCSlGDsTldg=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR10MB4688.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(376002)(396003)(39860400002)(366004)(346002)(136003)(71200400001)(122000001)(8936002)(76116006)(8676002)(3480700007)(91956017)(2906002)(66476007)(966005)(66946007)(186003)(66556008)(2616005)(66446008)(6486002)(83380400001)(6512007)(316002)(86362001)(6506007)(33656002)(38100700002)(110136005)(26005)(4326008)(64756008)(36756003)(478600001)(5660300002)(45980500001); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata: e5z4QLEly7LFXCva6ZQLQSFY0abbe8L7vNEAfIFggHPbVNLpAp5CsOXl+n4v9E/94f3CCecjI7N7gPei1vy7okzcxndxZWQS9ag3cTxZ10KBcj6tBQ9HV2uTRNdoj2me2z/1Bai3YH4AgedpFU6SPdI29pOFSNE+DphYYll7aQ9DmOT1dsN81Yq+ZrBg+GGXHHBBmCiU4RrzerbdazcoX7/4o3lE+qgRdozTyc6SG38d3ao4PMxFNf+ZEVYgSrfXCzRa8d4BmPcbKCm3pWsCeCyOEkIId7759R9x7nKkCvWrzCFY1wFL6LzWawlB4eeijCaw+rokx0J1KdyaDy4cTGIaDxAIhXmd4uy54uA7DFP3eilAlCuLdasnGpl3HcoK/kTVKmJIlzpv1w4EP6NEVUnAMX83iSho8eUHuJYS5P9g+PWrO6pbL5oVtQmuQZhkDB3LYOQQT2IY1K2/i0Jzax1IXGmWgbPpOLNAuc8GaWRH+Eh4UKj9JOlouSHXmnZ5LjlPQNr1b6qBU1M8++lRYyYIB0/g9L3fAAQYFBClPWD0hr+i9IyV0tUransR66KmlmZETBhnZmqdEOWs2MlqveRqq95/M80Sc9GcDwynlyo0BGrsK+oI27QauFX4rxpXgIcRSKobKw0druXsl1fHUH9wqgQWYMkMGXlQNbJeILCgthfDJ1k3hpgHe3w842YN+U7fPn/FvriagUJXlTNm3SBDNB81vnQ7pVt4jqI+JpH21UxGq1j5wD+RqvY/GhDpiVhxHgqavH/8uKf2HpOlwHimcV3EweYejhyeyg3rRMksuiuUTjjU83JnFNHq/pzY341HIypLdzi6hE+xEEND1aLJ+X96KCbWm16p+MSJ0ctjloDPjO0faUSecdMuG2pEAEcdncCoKoEpFPgREaRSyR/8BQXNICh+9TT4gQMKhbxhFV2go+PmdYA6W+fRy5/d9fd7n0uybgeq9VUQaI9PjkR011pt2VGGDaUrtkpT+fWJIYsyCbiLzXTXwoTVD1R+MydAwqd/z3j/iudU47kOzvx0BSvp+fwqB4i2vUnCwdi8mLgTzo5KCe6pszqsVKxxw+8I7q5MeAj5ImQ2XDAXJMJuarvBWTvbnHHMnJaEAEWQKYh2uZYBXXYeYveRzAulu6zXUcZ9P78/vHhqy/bAtB51by17wu6hst84KbHjpL/DijW3NO3TzYEoZTegd6B1OOc2l7RkUjr5zKRuXZ+FfQ347QieuurKyvPwyb7PC3JTfPq9W+ADdvFK0u9BZLxBLl5gxSMkU1gTgYXXJxTNGkGlc5bAVPXeYFjfQ/pU92KwaSLooncn5zhLcX2fmWSU
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <6ED80235A151B54FBEDE6E9870326310@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4688.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bfac4f11-4e46-4a5f-07b5-08d90b3af42a
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Apr 2021 18:16:48.1362 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: qjatLzPNpvxNDj8e1ls9Www8OJB6peVHvBmJzMnz5UyMr0epngTlrxa79lWaHZ5hNthyCN8pW0QY/lMcC4+4JA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4510
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9969 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 mlxscore=0 mlxlogscore=959 bulkscore=0 phishscore=0 spamscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104060000 definitions=main-2104290117
X-Proofpoint-ORIG-GUID: rz4yjkgSrto6ZAm23QwOVcGP6-Uu76jr
X-Proofpoint-GUID: rz4yjkgSrto6ZAm23QwOVcGP6-Uu76jr
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9969 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 phishscore=0 clxscore=1031 suspectscore=0 lowpriorityscore=0 mlxlogscore=999 mlxscore=0 adultscore=0 malwarescore=0 impostorscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104060000 definitions=main-2104290117
Archived-At: <https://mailarchive.ietf.org/arch/msg/nfsv4/-g00mFldNSnoB11OVhMhmvUWsR0>
Subject: [nfsv4] questions about flow control
X-BeenThere: nfsv4@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NFSv4 Working Group <nfsv4.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/nfsv4/>
List-Post: <mailto:nfsv4@ietf.org>
List-Help: <mailto:nfsv4-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 29 Apr 2021 18:17:00 -0000

Howdy-

We're working on prototyping RPC/RDMA version two. As many of
you know, RPC/RDMA uses credit-based flow control.

I've presented to the WG before on the kinds of improvements
to credit accounting we need to make over version one of
RPC/RDMA in order to support control plane operations and
message continuation -- cases where we no longer have
perfectly symmetrical Call/Reply pairing.

I'm looking at Section 4.2.1.1 of draft-ietf-nfsv4-rpcrdma-version-two
as it is currently constructed and I'm finding it ...
underwhelming.

I'm thinking of replacing it with something more akin to the
original forms of credit-based flow control, as described in
Chapter 4 of:

https://dl.acm.org/doi/pdf/10.1145/190314.190324

and implemented in the form of Chapter 5 of that paper. The
rdma_credits field would be filled in with the sender's Vr,
in both directions, and N2 + N3 would be the credit limit. We
would need to add some kind of "reset credit accounting"
message as well.

I'm not feeling confident about this choice, however. Does
anyone know a person or people who could answer some questions
about flow control design? Or is there a good reference I could
read to help me understand fundamentals and common pitfalls?

Many thanks in advance!


--
Chuck Lever

v2.23.0 | Report a Bug | By Email