Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done before/after Reclaim_complete?
Trond Myklebust <trondmy@hammerspace.com> Sat, 06 April 2024 03:25 UTC
Return-Path: <trondmy@hammerspace.com>
X-Original-To: nfsv4@ietfa.amsl.com
Delivered-To: nfsv4@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EDBABC14F6BB for <nfsv4@ietfa.amsl.com>; Fri, 5 Apr 2024 20:25:03 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.096
X-Spam-Level:
X-Spam-Status: No, score=-2.096 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=hammerspace.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mPv4fzt1o8Ym for <nfsv4@ietfa.amsl.com>; Fri, 5 Apr 2024 20:25:00 -0700 (PDT)
Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2111.outbound.protection.outlook.com [40.107.223.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E0A37C14F71A for <nfsv4@ietf.org>; Fri, 5 Apr 2024 20:24:59 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hndnqQ5voC6AJzZ6g1sCb7gZxx0ltpKbxyNpqLzc6pTOf492nAucMAA4fEDKiPjSSSdf/QhJ6FfgFajXU73znxWgmdPiATpueubecu+YdPQmeB9IWV7/jMtVMbNAYxms5TbjyineMx04mnibmuoOJhOyBlYKrhLpDBzfFOyNJThhPxqwkmip9gTSTkuLfqVL0FHkUTN4LeraXPBC0YaxUheVP8kYMfk2sJ6n+tJQQddcTINydJVQJnnW5YeeLAxzqxuU1a92FevbZjy3AM0FlY0NFBshT0VUzzWp+PLWlSielO2cvkpluO69bOAGR3MqeDDW+/8h8ckcLXSXdocBVw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MXvrGbsCN1OHpmZTRT/FKbNsyzm98a9DaoJC/y3XR3k=; b=I5ei9lnmCPBuH8ZQXCjVSUZDQtm5mgRpaP2VT8z3itST+7fW8SVviuma4KjcJ5YcaispZ9j5KJr9YfwVIzbwB48SkwmXFad6eJOLQZZe8rUWmlPR7gpfb9LxqLKI/n1q8URuPbgQeu0lZ9aAHcevUsNwudGizMbR1aY4vLc4GvfGAbh7c24F3clEFYIGpRYXdlAldc1Q1LABf4AsTsVk+btodYzCO4l9pncGt0QQ7HMaYFAf38mIOBkDMZCvjqIW4YKZGS3RJHTneRH95YzI0WtMBrbOyvlsobXLBkUCa7tQrY12RphcgNdAjP8V/ADH0mKSUe+Xe3vRxiTzpgpPrg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=hammerspace.com; dmarc=pass action=none header.from=hammerspace.com; dkim=pass header.d=hammerspace.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hammerspace.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MXvrGbsCN1OHpmZTRT/FKbNsyzm98a9DaoJC/y3XR3k=; b=BgGLuu3PEK+9TS5VT0IZpWFiKxd/xbdi+LaN4roOAh3Nk9UaUf2rHyRqC2JdcySCC7PhnBOtwHoT5FxIX6odTOA+5Ke8GqVtXStQTouRWyE8mde+rFoO2+0beYOKYS14xd/GuoSmZ3MFM5YHSEvBy9cqBhwvkdUxGBl33xJXrX0=
Received: from CH0PR13MB5084.namprd13.prod.outlook.com (2603:10b6:610:111::7) by SA0PR13MB3934.namprd13.prod.outlook.com (2603:10b6:806:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7409.46; Sat, 6 Apr 2024 03:24:55 +0000
Received: from CH0PR13MB5084.namprd13.prod.outlook.com ([fe80::5bb5:501a:fb40:5057]) by CH0PR13MB5084.namprd13.prod.outlook.com ([fe80::5bb5:501a:fb40:5057%6]) with mapi id 15.20.7409.042; Sat, 6 Apr 2024 03:24:55 +0000
From: Trond Myklebust <trondmy@hammerspace.com>
To: Rick Macklem <rick.macklem@gmail.com>
CC: "nfsv4@ietf.org" <nfsv4@ietf.org>
Thread-Topic: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done before/after Reclaim_complete?
Thread-Index: AQHahxBMvCkGUO5qpkColHKPwUmNU7FaT3oAgABAGACAAAY9AA==
Date: Sat, 06 Apr 2024 03:24:55 +0000
Message-ID: <28CDDCED-0056-427A-8D71-49AB937F8884@hammerspace.com>
References: <CAM5tNy4y36AEyD5SeOZ2F1t8AYhu8-ZKPyNvCd-dmydFg1kJtA@mail.gmail.com> <19e3502ca1859608975bd17dc18e6774bf0bfab3.camel@hammerspace.com> <CAM5tNy41j0BLi0ixu=VA_rT14GxL7ZKKwbSyb1UAgf-xAE8m+g@mail.gmail.com> <0d1c12d3e85a5feb165c13b44b0f25111224166f.camel@hammerspace.com>
In-Reply-To: <0d1c12d3e85a5feb165c13b44b0f25111224166f.camel@hammerspace.com>
Accept-Language: en-US, en-GB
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.500.171.1.1)
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: CH0PR13MB5084:EE_|SA0PR13MB3934:EE_
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: p8KKkYtP9MfXEVoYT8ZQDEjW7SPSW2s7/gdjwWrdb4AZfWEou6UHYK0Znyti8rakNHafF/598+NG8VkfF5CybdNprWbFCwqbqt1WyOwiOZFWGtgWjNGQhR90bEHJBBsHIW+Et8wT35RsQ2j2pULETz1TVZrM3WqHks1qRFdCQiSFzHnJOeImLzQJTHyHp9iQAMG8c9y1vS2jVDjM0lQn8TtQHY2WgTb0BVZGmPP3Qi3QMfenfnZl0bRSUtst/cr7+Fra99ySC3SiqvD3auVkqcpmlOnsfved8SOW8vgUUXFE3kqYr6Htvp1uTSsCeOf4SIOP/m9NP+9c8q8Nwrw39p2Nastmj6N8wBjBZy3c+6lfgyMy1hAp7+JPXPvEU1xFeN6bMwoWwMzEVi7dxSMeLAcW7FUcVUKayQEjnt2rnpnpVOc5ZsYBxlK/3gc+2eaqu2kphbDnP7ecxgdpQi8yDqEhsZxiH7ZXYeDuAVuJfosbr/iOBuRSeedJGRXzsfJgXZTiFhbRj7KaeAw8hGpWJQ+Cb4nSNafh7y9dVujAFiGYreEShY8A/QyGmjE4Ta/wML5S33mD0ywR+tRkvgEciAE9ih1rX+cXdfEQpdLAJWWrrHJUHSKp5go9rfESB5TMtVLyAAMf1Qacx2skacJRbwy2aiXOx5DVcINn+vPmAKc=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH0PR13MB5084.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366007)(376005)(1800799015); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: XjP/lV5PrLCjCEydeMMUHuODQ+nY9WqIKOKdooJPls3GL+Y0Hlp2e/1JPc/TdL82ydAsjg5bpDc3L8jNTuogXp3Bim/OHPKS7BRC7WPfNEs/lAOP7VrpHxTh1Iktr/TPMzfm35Sb4DJISkgwG1tIucYZFFz/gQw9G15tgaHrvG4SRS77K08SRB/LtZ5J9SyVtyzBhwrAfKqZ0+E6DFi25a5KHNx9oiRUSkZeQCywRn/Idtlj9ZmzRmB4fam7u63c+uFE14E1tE2036hd7J64qmmas2lbcPbwFp1WqQRe7Rc/EXqeqhvq+pcASfsosy0JrqW5TTP9hQ9c+HxIVrqaSSsx75wucjCpbN30pxgnaw7uKBF5a3zogo6x8Bne4etQMUwZn4mAS0E4JTSgBZaOlQj02m1HNlbVuSo1RbqITei1bIScW6bN37xsEWj+Hv8gYhJYTXNvlAzU//AKIf8j5jZKVLecJtyeCG1HViPhr1w53sb5xa9Ga9+eiOI9DNVs58aaGF2ZVDJG1fg8AFRrdanVo5AKkKDqjHyJhEA1hfYeQ+0nRYcKdEBTCO9v0+LPAaiIWCRNFH58arwzlXB07D+HiB2AX9jnp7rS5CyHTyYM4jEmkLH79jPdddLUtC8Ccc1RI5qt6IeyIsk6M3l+JlN9tULwj923hCwTR4FyfpmSMDSALF2TeajLeE1vhyY0JKL7LxSuzBM7WrLHWRppENtBgcPZTeM1UorgRk+6St83I84woC0zkN0VSuRxKbUQEAB8JJEPk9P9wKTv7twYU9sRhpI1nCn/6cCS3ubZrXV0pdO7joy+fSwIttSyDnEaewuagrXiTbVX06jFtUOrBxuwlRirADToHnecT5nNfYAqa1fhvmHBPw3xHmJ4KA6hQ0HxUKo9ldpadLBR1yflMUUdYHH3Gsh/WDRnZ+BwO47VMybQ8gyxYvM6VV4tItCfh3SMB1E286pn3t6uC8UgmMWvpgeU7piiZzrQtMU7tnv3xwqJcRRpTWtn5yZTSUrHsKxAHVJj8mCo2Olorc0nJswETchXLIl5hgND4yadE4qykHuHFKaReo9eJj4HkgBAskXwn+EZrI0K7mycR58D22B59VPQTGF700ENR523sS040Rm2nyTKVkUHxA5Sa2KQ6vhxAQsdfiGc6MKLS6hVJJ7XL10MLfup0JWehj+BXsK5USBTJaCSOba/hg8M827iglKFHTL2E9w5CoUMtp/XIjNrfLnpDy+iDMxl5Q6amQ10vH+VrPcLO203NUYOTIPpg98vaGOBopqjsxUtLKi0CfZZ4nNFj3r+5jojewPuyTuSxC+Hxp247OwEwQLypLK17e07ZYg1RXKetlZS9/JwSsdGVEIdRNSYZRS83NGJDxV1hNV3Co2jGbhFqQ7QqXJPgEC1i7n8b3tMPBDMzy1ha1rrmGq+3ePv70TYp2K/po1/hHwIQ8pw3zcKsBzgTEpscZqPGgfxZwKJk79kJDDDCyZgYzkC8OjXijY5QR7CcSsme3F3WVUY2b+RToUEJxQOxgs+nMUtGXjF9GSNxB5u0gXJdysafcA3k31wn19xBf9tGHWFeO9t9UcTTRiTlvuIZTSDO5qUUFx4zhaleYQL925mZ5EXOukS8+twqOehxvFI3xwclJJZKEyIOnCbNYJfGO3tV0zzjGbV3SPhxnd++A==
Content-Type: multipart/alternative; boundary="_000_28CDDCED0056427A8D7149AB937F8884hammerspacecom_"
MIME-Version: 1.0
X-OriginatorOrg: hammerspace.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: CH0PR13MB5084.namprd13.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e0488475-9be9-4397-f018-08dc55e9218d
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Apr 2024 03:24:55.8246 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 0d4fed5c-3a70-46fe-9430-ece41741f59e
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: rL9X2QsxMfp1BhsPUFMLUd48YWTogW030DACUCVqtOygcfdWAjUqnWBeF7bqKsNTMDCYNWHLg4y7yVkS9+wKWQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR13MB3934
Archived-At: <https://mailarchive.ietf.org/arch/msg/nfsv4/2jcBwDVSEtOXd2BeQMTJCAXfCXo>
Subject: Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done before/after Reclaim_complete?
X-BeenThere: nfsv4@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: NFSv4 Working Group <nfsv4.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/nfsv4/>
List-Post: <mailto:nfsv4@ietf.org>
List-Help: <mailto:nfsv4-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 06 Apr 2024 03:25:04 -0000
On Apr 5, 2024, at 23:02, Trond Myklebust <trondmy@hammerspace.com> wrote: On Fri, 2024-04-05 at 16:13 -0700, Rick Macklem wrote: On Thu, Apr 4, 2024 at 9:18 PM Trond Myklebust <trondmy@hammerspace.com> wrote: On Thu, 2024-04-04 at 16:23 -0700, Rick Macklem wrote: Hi, I am in the process of implementing Open/Claim_Delegate_Prev and have run into a question I cannot seem to answer from reading RFC8881. There is this snippet: enum open_claim_type4 { /* * Not a reclaim. */ CLAIM_NULL = 0, CLAIM_PREVIOUS = 1, CLAIM_DELEGATE_CUR = 2, CLAIM_DELEGATE_PREV = 3, /* * Not a reclaim. * * Like CLAIM_NULL, but object identified * by the current filehandle. */ CLAIM_FH = 4, /* new to v4.1 */ The comments in the above seem to "hint" that CLAIM_DELEGATE_PREV is a reclaim type operation. If that is the case, I'd assume it/should be performed by the client before it does a Reclaim_Complete? Then there is this snippet: For OPEN requests that reach the server during the grace period, the server returns an error of NFS4ERR_GRACE. The following claim types are exceptions: * OPEN requests specifying the claim type CLAIM_PREVIOUS are devoted to reclaiming opens after a server restart and are typically only valid during the grace period. * OPEN requests specifying the claim types CLAIM_DELEGATE_CUR and CLAIM_DELEG_CUR_FH are valid both during and after the grace period. Since the granting of the delegation that they are subordinate to assures that there is no conflict with locks to be reclaimed by other clients, the server need not return NFS4ERR_GRACE when these are received during the grace period. This clearly states that CLAIM_DELEGATE_PREV cannot be performed during the server's grace period. The above two statements are not necessarily contradictory but it does seem that, if the server is in its grace period, the Reclaim_Complete needs to be done before any Open/Claim_Delegate_Prev, since Reclaim_Complete tells the server when to end its grace period. Note that, usually, a client would not be rebooting when a server is in its grace period, but that could happen. Bottom line... Should a client perform Open/Claim_Delegate_Prev before or after Reclaim_Complete? And, if the answer is "after", how does the server know when a client is done recovering opens/delegations via Open/Claim_Delegate_Prev? Thanks for any help with this, rick CLAIM_DELEGATE_PREV has now survived at least 24 years as part of a "standard" and has yet to be properly described. I can find references to it in RFC3010 and all NFSv4 descriptions since then, but I have yet to meet anybody who can claim to have implemented it in a product. IMO, it is time to drop it, and start afresh with something that has actually been road tested before someone started putting pen to paper. Well, I am going to respond to David's post in some more detail, but I will note here that, with the exception of whether to do it before or after Reclaim_complete, I have not found serious issues with it. (Btw, the FreeBSD server has had a "dormant" implementation for about 20years, although I am just now trying to use it by the FreeBSD client. I was wondering if I would find any other server implementation at the Bakeathon in a couple of weeks.) I see. So how do you plan to deal with the case where the server denies the request for a delegation? Will your client just toss away the dirty data? My problem with CLAIM_DELEGATE_PREV is that the “spec” has no description whatsoever for what happens in the “unhappy path” case, where the application has done everything right to apply a share lock, or a byte range lock to the parts that it plans to modify, and yet because the server cannot be contacted, the client lets everything proceed because it wants to assert a delegation once the server is back online. …. and yet the assertion of the CLAIM_DELEGATE_PREV delegation can trivially fail if someone has modified or deleted the file during the period when the client was unable to talk to the server, and there is nothing in the spec that advises what to expect next. _________________________________ Trond Myklebust Linux NFS client maintainer, Hammerspace trond.myklebust@hammerspace.com
- [nfsv4] RFC: Is Open/Claim_Delegate_Prev done bef… Rick Macklem
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Trond Myklebust
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Rick Macklem
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… David Noveck
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Trond Myklebust
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Trond Myklebust
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Rick Macklem
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Rick Macklem
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… David Noveck
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Rick Macklem
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Rick Macklem
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Rick Macklem
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Rick Macklem
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… David Noveck
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Rick Macklem
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Rick Macklem
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Rick Macklem
- Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done… Rick Macklem