Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done before/after Reclaim_complete?

David Noveck <davenoveck@gmail.com> Sat, 06 April 2024 10:22 UTC

Return-Path: <davenoveck@gmail.com>
X-Original-To: nfsv4@ietfa.amsl.com
Delivered-To: nfsv4@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3244EC14F6AC for <nfsv4@ietfa.amsl.com>; Sat, 6 Apr 2024 03:22:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.094
X-Spam-Level:
X-Spam-Status: No, score=-2.094 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D0Q-e7nF4jYU for <nfsv4@ietfa.amsl.com>; Sat, 6 Apr 2024 03:22:27 -0700 (PDT)
Received: from mail-qv1-xf34.google.com (mail-qv1-xf34.google.com [IPv6:2607:f8b0:4864:20::f34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F882C14F69D for <nfsv4@ietf.org>; Sat, 6 Apr 2024 03:22:27 -0700 (PDT)
Received: by mail-qv1-xf34.google.com with SMTP id 6a1803df08f44-6994313546aso6394996d6.2 for <nfsv4@ietf.org>; Sat, 06 Apr 2024 03:22:27 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712398946; x=1713003746; darn=ietf.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=0JaJxTJvX3uEGg0GbAVkdo383+TJLeYL4aR/fc2H00s=; b=it1VW5kucfsgPR4DmCG9OvkrI0fC7feaF2W2t2zK6ZRVL16QqazY+w1gZMAIajBQjA FQYy+0hOp1sHkgGWFCR+z4Jk/HpDxEmpSRS3iiZmV37O36Y3uXdNVyi0BB+oYrHg6TvR EKaEHES8j/bcteqOAt+uWr0xTDkzvnnBy7W20Rf8lNNhkISYfgoC5F4NzBLAsasKutav a24y6j3a0LHhfRMe8yLqilCEUbqI85sMH1yyb4IUkC0Y0EziA+tARhdiVvRv4hToYUIr Rwa3Gx/00Jsj/2ZyhiMJTG30sFj/ohwloBPq1X/h1tCSnF8bkoWKMzqC3qchoJl3mp/z zuWg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712398946; x=1713003746; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0JaJxTJvX3uEGg0GbAVkdo383+TJLeYL4aR/fc2H00s=; b=oujkFlF6WzfIr0zIhNrDQpEMVVDNfcwIlIcLTZODzb0ZtfDsfw6TIhvzkdSpUVWVkt tQkHROAI2LdZoniSUqpdN8+zFr99XYhGT+gANgJpPa+fD/86K2gSjwRDp0mJ9MAla2yT /xU+N/o0x4KIVoZMLGR5En1fcPPGm1ww7/ycuuvUTwAW1Dm9Mi06TqpL0haja2/0pYat yvlkQTh6BjEDAPeH0TLtbY3j8lS1lGcEAgLCtIz/kZS+kjPBGSNFxYyIAOwaM21NXRIv pmaM8iQz9vwinjN/0Q4s5Txn0Lp6AFAx/SgwuHkumwVh5D2RbJTLKgCdWUf8/qyboFGV 4Y0Q==
X-Forwarded-Encrypted: i=1; AJvYcCXwp2An5LMtssfuffoLZ8B7X3mZvK7+d+JJFX0+VKteIh05dxFx5psmTEq4PyJfjSmtdY4Sfujhpi5pFTkHYA==
X-Gm-Message-State: AOJu0Yzv2Q2nc7fsoQtk9mAkonbA7imxXs99GkyM4vnKnyaKm6FyhJAZ R39erHct8iBTdoZ/V5pZFYz0Sa/kGRX4XuHUNArOLkhVnVMcxlnR3pgWzsfk9l86u62qrPJBaTO 9UlwFuev7C9qclMjv1kJI3wf6jII=
X-Google-Smtp-Source: AGHT+IGZlaIQ1ZSJyKpttfvcSos9GGzQT4owOCNBeQqIsJy8MICbrssiR0YtGRfYvfwMTEc5BH8kOmHWJxVTm6QAd9s=
X-Received: by 2002:a05:6214:29ee:b0:699:406d:9030 with SMTP id jv14-20020a05621429ee00b00699406d9030mr4046857qvb.38.1712398946172; Sat, 06 Apr 2024 03:22:26 -0700 (PDT)
MIME-Version: 1.0
References: <CAM5tNy4y36AEyD5SeOZ2F1t8AYhu8-ZKPyNvCd-dmydFg1kJtA@mail.gmail.com> <19e3502ca1859608975bd17dc18e6774bf0bfab3.camel@hammerspace.com> <CAM5tNy41j0BLi0ixu=VA_rT14GxL7ZKKwbSyb1UAgf-xAE8m+g@mail.gmail.com> <0d1c12d3e85a5feb165c13b44b0f25111224166f.camel@hammerspace.com> <28CDDCED-0056-427A-8D71-49AB937F8884@hammerspace.com> <CAM5tNy7pkz6q5fY_NvDhLs91Eq-DNXx5gvFOeh3fqsNS0YwA9A@mail.gmail.com>
In-Reply-To: <CAM5tNy7pkz6q5fY_NvDhLs91Eq-DNXx5gvFOeh3fqsNS0YwA9A@mail.gmail.com>
From: David Noveck <davenoveck@gmail.com>
Date: Sat, 06 Apr 2024 06:22:14 -0400
Message-ID: <CADaq8jeFBA=D50SwXeKRRzqx2dLy7JR2KKz6OD8q9WiRK3AxNA@mail.gmail.com>
To: Rick Macklem <rick.macklem@gmail.com>
Cc: Trond Myklebust <trondmy@hammerspace.com>, "nfsv4@ietf.org" <nfsv4@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000e6879b06156af277"
Archived-At: <https://mailarchive.ietf.org/arch/msg/nfsv4/6qScVwcEmYrS57wG-ihlKYVK2rQ>
Subject: Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done before/after Reclaim_complete?
X-BeenThere: nfsv4@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: NFSv4 Working Group <nfsv4.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/nfsv4/>
List-Post: <mailto:nfsv4@ietf.org>
List-Help: <mailto:nfsv4-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 06 Apr 2024 10:22:31 -0000

>
>
> I have assumed that Open/Claim_delegate_prev will only be used by a
> client after it has rebooted.


That makes sense.


> I cannot see it being
> any more/less useful for recovery from network partitioning than the
> rest of the Opens.
>

I don't think it is useful at all in the case of network partitioning.  It
wasn't intended for that even though there is some text in the section on
delegation recovery. that mentions detecting the need for recovery as
preceding from a partition and not mentioning client restart as it should.
That needs to be cleaned up in -04.


> I assume that a server will reply to an Open/Claim_delegate_prev with
> NFS4ERR_EXPIRED if it does not have such a
> delegation for the client, since it was not able to recall it.
> I'm not sure if the spec states that, but since delegations are
> covered by the lease, it makes
> sense to me?
>

Not to me.  I would prefer NFS4ERR_RECLAIM_BAD and -04 will say that, but
we can discuss that as part of the rfc5661bis review process.

No doubt network partitioning and NFS4ERR_EXPIRED is ugly.


I agree that network partitioning is ugly.

But I do
> not see anything different for CLAIM_DELEGATE_PREV
> than other open/delegation handling.


Me either.


> Typically a client mount will
> hang pretty quickly during a network partitioning, but applications
> can certainly write a lot of dirty data into a client's data cache
> before the network partitioning is noticed and a client does not
> have any good way to deal with the dirty data.
>

I think the best way is to write it to local persistent storage, as you are
exploring.  This brings us to
CLAIM_DELEGATE_PREV which certainly wouldn't be winning any v4.1 feature
beauty contests.


>
> > …. and yet the assertion of the CLAIM_DELEGATE_PREV delegation can
> trivially fail if someone has modified or deleted the file during the
> period when the client was unable to talk to the server,


The assumption here is that the client is holding a write delegation for
the file in question, so that modification cannot happen until the
delegation is successfully recalled or revoked.  As noted before, network
partitioning is UGLY.

and there is nothing in the spec that advises what to expect next.
>

I'll try to address that in -04.


> If the file has been deleted, all opens will fail (NFS4ERR_NOENT or
> NFS4ERR_STALE?).
>

Yes.


>
> If the file was modified and the client does a Claim_null open, the
> only indication
> that it has been modified is the change attribute's value, which it
> might not have kept.
> At least the open/Claim_delegate_prev fails (presumably with
> NFS4ERR_EXPIRED)
> and the client knows the file might have been modified by someone
> else. Not great,
> but I do not see how it is worse off than without Claim_delegate_prev?
>
> It's better since, without the ability to open the file, despite the
pending-recall state of the delegation, there is no way to do the cached
updates even in the more boring case in which nobody else is trying to
modify the file.

rick
>
> >
> >
> > _________________________________
> > Trond Myklebust
> > Linux NFS client maintainer, Hammerspace
> > trond.myklebust@hammerspace.com
> >
>
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4
>