Re: [nfsv4] GETDEVICEINFO returning ENOENT

Olga Kornievskaia <> Wed, 18 January 2023 15:23 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 87421C14CF15 for <>; Wed, 18 Jan 2023 07:23:23 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.65
X-Spam-Status: No, score=-1.65 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.096, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id BaZ-WdjZhWrX for <>; Wed, 18 Jan 2023 07:23:19 -0800 (PST)
Received: from ( [IPv6:2607:f8b0:4864:20::535]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by (Postfix) with ESMTPS id 969F3C14EB18 for <>; Wed, 18 Jan 2023 07:23:19 -0800 (PST)
Received: by with SMTP id g68so23649843pgc.11 for <>; Wed, 18 Jan 2023 07:23:19 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=google-2016-06-03; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ezD7bTenS+YYaq1HvSelwcCsB37G26EPbO+ZwMn7QFM=; b=bkFgjgbE3E4syas7rDdL2Bi637GLSNeuuAqRZ52muo/2PRKoPd10ROqRIigLXQHFXZ yfbumNvSx8RANyCaUUAe7ZkPw4btCh951r+SamiUwCwtZpaD2fg1eTM7Zhu3ApSbdUWe ZqkEs7T1p7ofYGRy7QPLqc5o+kO5fqMhakE/SPxDBPY0qnXyJ/JH28ehPzOA9zUV3yjx 0KYmTfWZ5GxCFvFoazrJ9lfHcQYebkhgQkO7IwNAyqZJHrvcaj33P6QQ9jd50Z2U0Bhu 3PPD+FAhpTPG1kQiZKdxQ0GdLtEl2uVysDPHtAGKtLRo8ODI5UggWtCzxnc9u2fboN76 whwQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ezD7bTenS+YYaq1HvSelwcCsB37G26EPbO+ZwMn7QFM=; b=MVLlFAtKFxaHoWMmkuZ6ldsZ/HU6jSuKU173ceV24zkbdpJnCO7BKBjtNauqg0ilMr TSzP/qVl25uZUX9v70a+7rnEMbTrW8HpRyzJVpJcoJubykRktwy7BARZZn2aZyO2mtgc q/1Jf6Kg4v1Tl8yMh1JyXanU4E1Dss/cKmB6lEMIjgd393bEBVUG1dga9IC/MoFqe9k1 9rPSvZxv0CreBrZOZQXphmaDiCVGHz7qk0Wk1SbP3K7+X03tnf7Ds+aXRhNUm+rTIXr0 gFSk4iWLxYJMvMhmMAEXWCyQWN2f7IZd0nvVYzesxII00+hepwDdxEa+F1Hoo7YgDZin uXRA==
X-Gm-Message-State: AFqh2kr4Uk25fkA8guC15iXGySXK00QjAgkKsctpl1sQu3SdZk98SOz3 NOfN80O9UabHtIdIQbnSCrW/SS1TMoCADjTvKWeXeaQo
X-Google-Smtp-Source: AMrXdXtoT897qHkyjx1zvAHC/EQNI4/Ijiz/fCoDdEooMEczpZ/NkobmyIC2CTbny9PgjiHWB7aJ2tQAqg8lJo2MBrU=
X-Received: by 2002:a65:4884:0:b0:476:a862:53d2 with SMTP id n4-20020a654884000000b00476a86253d2mr673459pgs.163.1674055398564; Wed, 18 Jan 2023 07:23:18 -0800 (PST)
MIME-Version: 1.0
References: <> <> <> <>
In-Reply-To: <>
From: Olga Kornievskaia <>
Date: Wed, 18 Jan 2023 10:23:07 -0500
Message-ID: <>
To: Trond Myklebust <>
Cc: "" <>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <>
Subject: Re: [nfsv4] GETDEVICEINFO returning ENOENT
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: NFSv4 Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 18 Jan 2023 15:23:23 -0000

On Wed, Jan 18, 2023 at 10:16 AM Trond Myklebust
<> wrote:
> On Jan 18, 2023, at 10:01, Olga Kornievskaia <> wrote:
> On Tue, Jan 17, 2023 at 5:54 PM Trond Myklebust <> wrote:
> On Jan 17, 2023, at 17:37, Olga Kornievskaia <> wrote:
> Hi folks,
> Is there anything in the spec or elsewhere to let the client decide if
> NFS4ERR_NOENT is a temporary or a permanent error for GETDEVICEINFO
> operation?
> Right now a linux client treats it as a temporary error and triggers a
> GETDEVICEINFO call before every IO call (until it gets a positive
> answer). The client then proceeds to do the operations against the
> MDS. But shouldn't ENOENT be equivalent to LAYOUTUNAVAILABLE error
> which the client treats like a bit more permanent error and doesn't
> ask for a layout on every IO after failing. It would seem logical that
> if ENOENT means the server doesn't know this deviceid which was gotten
> from a layout then should the client throw away the layout and ask
> again for the layout?
> Thank you.
> Why would a server deliberately hand out a layout containing a deviceid that it cannot translate? That makes no sense, and can only be considered to be buggy server behaviour.
> The current Linux client code assumes that if GETDEVICEINFO returns NFS4ERR_NOENT, then the deviceid, and hence the layout have been revoked by the server, and that retrying the layoutget will yield a new, valid layout with a valid deviceid.
> As I mentioned, the current linux code doesn't treat the NFS4ERR_NOENT
> as what you describe. It re-tries (GETDEVICEINFO) on every single IO
> call. I was looking for clarification to fix the linux client to treat
> Hmm... That behaviour appears to be specific to the files layout driver, and looks like a bug. The flex files layout driver will report the error using the layout error mechanism, and then return the layout, while the block layout marks the layout  as unavailable.
> I suggest we fix the files layout driver to drop the layout, and perhaps try to send a layout error, if the server supports that.

Yes indeed I should have said that it was just for filelayout
implementation. I will propose a fix on the mailing list then. I do
btw agree with you that it's rather questionable to make the deviceid
invalid just as it was given out in the LAYOUTGET, but it is valid as
per spec. And I thought the client should treat that the same as
LAYOUTUNAVAILABLE. But I also think that the server should then recall
the layout...

Thank you for the feedback!

> _________________________________
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace