Re: [nfsv4] Disk protection problem for pNFS block

sfaibish <> Thu, 21 October 2010 19:11 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id BBFC43A6975 for <>; Thu, 21 Oct 2010 12:11:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -6.329
X-Spam-Status: No, score=-6.329 tagged_above=-999 required=5 tests=[AWL=0.270, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 8WHAiXBpFpeM for <>; Thu, 21 Oct 2010 12:11:43 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id 60B0C3A6A6B for <>; Thu, 21 Oct 2010 12:11:34 -0700 (PDT)
Received: from ( []) by (Switch-3.4.3/Switch-3.4.3) with ESMTP id o9LJD1bj023456 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 21 Oct 2010 15:13:01 -0400
Received: from ( []) by (RSA Interceptor); Thu, 21 Oct 2010 15:12:47 -0400
Received: from ( []) by (Switch-3.4.3/Switch-3.4.3) with ESMTP id o9LJBXMZ003067; Thu, 21 Oct 2010 15:11:33 -0400
Date: Thu, 21 Oct 2010 15:11:33 -0400
To: "J. Bruce Fields" <>
From: sfaibish <>
Organization: EMC
Content-Type: text/plain; format="flowed"; delsp="yes"; charset="iso-8859-15"
MIME-Version: 1.0
References: <> <>
Content-Transfer-Encoding: 8bit
Message-ID: <>
In-Reply-To: <>
User-Agent: Opera Mail/9.10 (Win32)
Cc: nfsv4 list <>
Subject: Re: [nfsv4] Disk protection problem for pNFS block
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: NFSv4 Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 21 Oct 2010 19:11:59 -0000

On Thu, 21 Oct 2010 14:19:22 -0400, J. Bruce Fields <>  

> On Thu, Oct 21, 2010 at 01:21:25PM -0400, sfaibish wrote:
>> I want to discuss this issue in preparation for the WG discussion on
>> this topic. In the current pNFS block protocol the client has to
>> contact the MDS in order to get the signature offset of the devices
>> used by the block client.
>> Users of pNFS block in Linux want to be able to identify a LUN being  
>> used
>> to store pNFS data without having to contact the metadata server but the
>> protocol doesn't define a fixed location; the MDS will send it at mount
>> time. The location of the signature is not defined in the protocol but
>> we can make it a configuration parameter in the client that will define
>> the signature location for each pNFS block server. My prefered solution
>> would be to enhance the block protocol.
>> In more details the pNFS FS writes a signature to the LUN, but to find
>> out the offset of the signature the client first needs to talk with the
>> MDS to be told the offset that the signature is located when using
>> GETDEVICEINFO. But this is too late and can only be done after the boot
>> ends and as a result some boot time applications can destroy the FS.
> I'm confused: you're assuming such applications may write to every disk
> they discover that isn't on some list of disks not to touch?
Yes. We encountered such cases of DB applications that use fixed device
name that put a signature on devices. There is no such thing as list
of devices not to touch. How do you identify them if there is no signature?

> The pnfs block protocol has to assume *some* level of sanity from
> clients.
Well this was the premise when w5663 was written but I can show you cases
when happens specially with applications using raw SCSI devices FC  
We had this problem with our private MPFS many times. And this is in the  
sequence and you cannot change that as some applications are started at  
time. I discussed this with Jason and he agrees to this.


> --b.
>> One
>> possible solution is to "hide" pNFS devices or write protect the devices
>> that contain pNFS data even if that host doesn't mount the pNFS
>> block volume.
>> The concern was that there is a window of time before the devices can be
>> protected at mount time but it leaves a window of possible data  
>> corruption.
>> I would also like to include some kind of protection mechanism in
>> the protocol
>> as an extenssion to RFS5663 for example. So Jason and I propose
>> something like
>> this:
>> 1) Clients can have configuration file that specifies the signatures
>> to protect.
>>    The signature should be vendor agnostic: "pNFS block device" but
>> at a fixed offset.
>> 2) Clients could come pre-configured with a protection file that  
>> recognizes
>>    "most signatures"
>> 3) pNFS servers could define a function that will return such  
>> signatures.
>>    It will be late, but every time one does a mount, the client
>> could retrieve
>>    the signatures, and then make sure that they are listed in the
>> configuration
>>    file.  This way the system learns what it needs to protect in
>> cases where a
>>    new non-standard pNFS server is introduced.
>> What do people think about the problem and the solution idea? I will
>> collect
>> all the opinion and include in the discussion in Beijing. Thank you
>> for your
>> consideration
>> /Sorin
>> --
>> Best Regards
>> Sorin Faibish
>> Corporate Distinguished Engineer
>> Unified Storage Division
>>         EMC²
>> where information lives
>> Phone: 508-249-5745
>> Cellphone: 617-510-0422
>> Email :
>> _______________________________________________
>> nfsv4 mailing list

Best Regards

Sorin Faibish
Corporate Distinguished Engineer
Unified Storage Division
where information lives

Phone: 508-249-5745
Cellphone: 617-510-0422
Email :