Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done before/after Reclaim_complete?

David Noveck <davenoveck@gmail.com> Sun, 07 April 2024 14:52 UTC

Return-Path: <davenoveck@gmail.com>
X-Original-To: nfsv4@ietfa.amsl.com
Delivered-To: nfsv4@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 74DE8C14F5FC for <nfsv4@ietfa.amsl.com>; Sun, 7 Apr 2024 07:52:40 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.094
X-Spam-Level:
X-Spam-Status: No, score=-2.094 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AxdwvcY3NzDG for <nfsv4@ietfa.amsl.com>; Sun, 7 Apr 2024 07:52:36 -0700 (PDT)
Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com [IPv6:2607:f8b0:4864:20::f2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8BE02C14F5E7 for <nfsv4@ietf.org>; Sun, 7 Apr 2024 07:52:36 -0700 (PDT)
Received: by mail-qv1-xf2a.google.com with SMTP id 6a1803df08f44-690be110d0dso19565406d6.3 for <nfsv4@ietf.org>; Sun, 07 Apr 2024 07:52:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712501555; x=1713106355; darn=ietf.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=6BDidZpkymLfZnsCI1zW2y0Ok+5W7Ul57mwBXT04ijE=; b=KCI8tNdOze9aVKycjvX4HuQwjIPkUtxafoqfvYFCsSRT7aZXEVWIxv9sDPbAVwDTZq /dTLus3MApokK+95VFKKY6/rG2u28PER0tGQ3zo+MO/pGaxJETYLZHkU6jO4n5/6hmSk u39eS17RFWwQvEPWfIZans/nG+T7frcNHxgwnUXEuOeV8V5Eb0g0m+5wxH0Ik+bY2kzC WF1XODLsKB285bbM7Xk8VgDOVacKWbT7rBYeARX2W3zbMX7s9BgaB+yMLJjsE6gtyf6q O3NT+f4uCXX0sfBHd8GN1piyLKfVEvCjPS9KuHKolsm/ZQXvwKZXE76EMxY1sFNzbYYU lR3w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712501555; x=1713106355; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6BDidZpkymLfZnsCI1zW2y0Ok+5W7Ul57mwBXT04ijE=; b=JQp29PhTxYA6aocvWs/n0NHdGQreaEcmdO+MkAQahhGaGv2QKVUnM+MHUaJvY43Hz9 6iM7VWLfgkC3lC6z3h2a1Dtv63PRNtodRFBrITTT46jyRoFbRdsRaLqdBS5DXfGqyy92 PCInC2eWYO0fDM+VijWHDVbt58jAm8FKzxxfhPL+MIH1IgW0pip3IHdCuIKJdCyAc9PK N8lPCKD5QKggbpdNa8FbW0BOpLjby5QMxphL8+PHLTtD37gHuZ+GeclqRrIpEbfaYHym nsDddLkShIzKexmYAEqkmU82f9eSG7ugBWARL3lgatNIP2yx/1g4kLOXQm60vJgIb5t0 jZtg==
X-Forwarded-Encrypted: i=1; AJvYcCWzGdc8K7KwTihQY7066ttrOfENAXvMm4wBrRXGpXCNKpdt4SMfD6Gt1ffMDXmSgmVUcvMgtgjllCuhdQ8NCA==
X-Gm-Message-State: AOJu0Yw5lqyuNb9Z5ME98hUBNHRoIYwC5ye4YnIr29VLRbwa5gGFY87U WnL2TnGzsEPaxnlWVayd0vEnG000DACfi9BjqHEAmVp2o1RPkiT6b91xhAf+8tFk7n4XF2vEZyo 4vRIM7aKqCqr9Ls+g6JJSPeVW6A1XD91MEyI=
X-Google-Smtp-Source: AGHT+IGx12xfU8IaTZvXL9POd7c5yuaV7OZVmdvkkfbRzMJB6Fn5hYCNj59g7diTO3eLsbVZIijTSUIOOlPSR4jYc+4=
X-Received: by 2002:a05:6214:403:b0:696:7ffc:4617 with SMTP id z3-20020a056214040300b006967ffc4617mr9197808qvx.11.1712501555220; Sun, 07 Apr 2024 07:52:35 -0700 (PDT)
MIME-Version: 1.0
References: <CAM5tNy4y36AEyD5SeOZ2F1t8AYhu8-ZKPyNvCd-dmydFg1kJtA@mail.gmail.com> <19e3502ca1859608975bd17dc18e6774bf0bfab3.camel@hammerspace.com> <CAM5tNy41j0BLi0ixu=VA_rT14GxL7ZKKwbSyb1UAgf-xAE8m+g@mail.gmail.com> <0d1c12d3e85a5feb165c13b44b0f25111224166f.camel@hammerspace.com> <CAM5tNy7i3SyY0mLbd+vsmO-jpkZ6Bdr8hz3w5T-PhR8GdbMpnA@mail.gmail.com>
In-Reply-To: <CAM5tNy7i3SyY0mLbd+vsmO-jpkZ6Bdr8hz3w5T-PhR8GdbMpnA@mail.gmail.com>
From: David Noveck <davenoveck@gmail.com>
Date: Sun, 07 Apr 2024 10:52:23 -0400
Message-ID: <CADaq8jdEUrf7Bp5njeKEbzLFW+vYdvNAaZtqeyfCcVVHmaYMxg@mail.gmail.com>
To: Rick Macklem <rick.macklem@gmail.com>
Cc: Trond Myklebust <trondmy@hammerspace.com>, "nfsv4@ietf.org" <nfsv4@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000e05ae9061582d698"
Archived-At: <https://mailarchive.ietf.org/arch/msg/nfsv4/SBtSu-F7nk9UnxCjnd25moOru_U>
Subject: Re: [nfsv4] RFC: Is Open/Claim_Delegate_Prev done before/after Reclaim_complete?
X-BeenThere: nfsv4@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: NFSv4 Working Group <nfsv4.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/nfsv4/>
List-Post: <mailto:nfsv4@ietf.org>
List-Help: <mailto:nfsv4-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/nfsv4>, <mailto:nfsv4-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 07 Apr 2024 14:52:40 -0000

>
>
>
> The more I think about it, the more I think you are correct, at least
> by default.
>

Not fully correct.  Some of what I told you is wrong and will be corrected
in -04.    The current target for submission is 4/23 but it is likely I
canpull that forward by a week.   In any case, I hope the following
fragment will be helpful:

/*
 * Includes multiple types of operations:
 *
 *    - Non-reclaim operations valid independent of grace period status.
 *    - Reclaim operations only used during a grace period.
 *    - Reclaim operations only used during a special delegation recovery
 *      period.
 */

enum open_claim_type4 {

        CLAIM_NULL              = 0,    /* Non-reclaim operation, */
        CLAIM_PREVIOUS          = 1,    /* Reclaim operation -- grace
                                                               period only.
*/
        CLAIM_DELEGATE_CUR      = 2,    /* Non-reclaim operation. */
        CLAIM_DELEGATE_PREV     = 3,    /* Reclaim operation -- special

  delegation recovery period
                                                                      only.
*/
        /*
         * Beyond this point, all values are new to v4.1.
         */

       /*
         * Like CLAIM_NULL, but object identified
         * by the current filehandle.
         */
        CLAIM_FH                = 4,    /* Non-reclaim operation. */

        /*
         * Like CLAIM_DELEGATE_CUR, but object identified
         * by current filehandle.
         */
        CLAIM_DELEG_CUR_FH      = 5,    /* Non-reclaim operation. */

        /*
         * Like CLAIM_DELEGATE_PREV, but object identified
         * by current filehandle.
         */
        CLAIM_DELEG_PREV_FH     = 6     /* Reclaim operation -- special

  delgation recovery period

 only. */
};

Why?
> Because when a client crashes/reboots and there is dirty data in the
> RAM cache it gets
> tossed, so tossing dirty data in the non-volatile cache when it cannot
> safely
> write it back to the server would be consistent with what happens now.
>

Yes, but the idea is to do better than that, when it can be written safely.

>
> For the case where the server is in grace, the client can use
> Open/Claim_previous
> and then, if the reply indicates an immediate recall of the
> delegation, it can write the dirty data
> back to the server before doing a Delegreturn.
>

True but that is an unusual case.

>
> For the case where the server is not in grace and the
> Open/Claim_delegate_prev
> fails, I think there is another way that the client can safely write
> the dirty data back.
> - If the client keeps track of the server's value of the change
> attribute for the file
>   in non-volatile storage.
>   - The client can try an
> Open/Claim_null/Open_share_access_write/Open_share_deny_write
>      followed by a Getattr of change in the same compound.
>      - if this succeeds and the change attribute is the same as the
> stored one, then I think the client can
>

If the  Open/Claim_delegate_prev fails, it would be because the delegation
was revoked, in which
case, change attribute will be different.   Also, in that case, checking
change attribute and proceding
to write is not safe, since the attribute can change after the check.  You
need continuous posession
of the delegation which is why clain-delege-prev was created.

       safely write the dirty data back to the server using the open_stateid
>        - then close the open_stateid
>
> An option other than tossing the data would be to flag it and let a
> sysadmin/user decide if
> merging it back to the server's file makes sense. Ugly but doable when
> the dirty data is in
> client non-volatile storage.
>

He could decide but I'm not sure on what basis he could decide,
especially since telling
him all the relevant information could be a security hole a megaparsec
wide. :-(

>
> rick
>
> >
> > --
> > Trond Myklebust
> > Linux NFS client maintainer, Hammerspace
> > trond.myklebust@hammerspace.com
> >
> >
>
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4
>