Re: [nfsv4] Current implementation status of NFSv4.1

Here is the relevant information:

SETUP

server:
   sw: dCache-1.9.10-1 on MDS and DS
   hw: 5x DELL R510 2x4 core 2.27 GHz, 10 Gb/s NIC (intel), 12 GB RAM, 
15 TB RAID6 SATA

client:
  sw: RHEL 5.3 + 2.6.36-rc3-pnfs-all-latest + nfs-utils-1.2.1,
  hw: 32x DELL M600 blades  2x4 cores 2.5 GHz, 16 GB RAM, 1 Gb/s NIC
  mount options:/* */*/-t nfs4 -o minorversion=1,rsize=32768,wsize=32768/*

network:
   hw: Force 10 Gbit switch

In total: 256 cores vs. 5 DS + 1 MDS

We run 8248  typical analysis batch jobs processing 33 TB of data.
It took 4 days with ~330 MB/s. No crashes.

We written in 10 days ~13 TB data from Stanford University to Hamburg
with FTP
client on a single node. Each file was between 100-200 GB. No crashes.

With 50 parallel jobs we get up to network limit. With mo jobs performance
started to degrade. I suspect, that there are no enough io thread in our
server
( only 5 per DS ). Our admins say that we hit the disk limits on a
single server
as we do not stripe the files. This have to be sill investigated.

All clients committed a group suicide after attempt to write into a
single file
(this was triggered by operational mistake). As this is not supported by our
server I suspect that IOERROR or LAYOUT_TRYLATER error was returned.
Unfortunately this happened during my vacation and I was not able to
look at it.

We have declared test success full. In total we have seen quite a good
performance
and reasonable stability.  We was able to show our users that NFSv4.1 is
real and
can be used even if it's still experimental.

We look forward to do similar test with some of the vendor systems as well.

Regards,
   Tigran.

P.S: here is the link to original presentation, but it full of 'foreign'
words.
http://117.103.105.177/MaKaC/materialDisplay.py?contribId=217&sessionId=60&materialId=slides&confId=3

On 10/19/2010 05:44 PM, Marc Eshel wrote:
> Hi Tigran,
> We are all interested, can you post it to this email list.
> Thanks, Marc.
>
>
>
> From:
> "William A. (Andy) Adamson" <androsadamson@gmail.com>
> To:
> Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
> Cc:
> nfsv4@ietf.org
> Date:
> 10/19/2010 06:16 AM
> Subject:
> Re: [nfsv4] Current implementation status of NFSv4.1
> Sent by:
> nfsv4-bounces@ietf.org
>
>
>
> On Tue, Oct 19, 2010 at 3:34 AM, Tigran Mkrtchyan
> <tigran.mkrtchyan@desy.de> wrote:
>   
>> We did extensive pNFS test last couple of mounts which we present this 
>>     
> week
>   
>> at Conference on Computing in High Energy and Nuclear Physics in Taipei.
>> I can provide numbers and slides if you need.
>>     
>
> Hi Tigran
>
> Very interesting - please forward them to me.
>
> Thanks
>
> -->Andy
>
>   
>> Regards,
>>    Tigran
>>
>> On 10/18/2010 10:30 PM, Spencer Shepler wrote:
>>
>> I am curious if anyone is interested and able to do a
>> presentation at the upcoming WG meeting on 4.1 implementation
>> status?  Given the nature of the interop testing (NDAs) I am
>> not asking for an industry report but more of an experience
>> so far and status for single implementations.  Covering
>> things like: functionality implemented, issues found, issues
>> unresolved, areas of concern, lessons learned for future work.
>> That type of thing.
>>
>> Anyone interested?
>>
>> Thanks,
>> Spencer
>>
>> p.s. Obviously, this could include NFSv4.0 as well. :-)
>> _______________________________________________
>> nfsv4 mailing list
>> nfsv4@ietf.org
>> https://www.ietf.org/mailman/listinfo/nfsv4
>>
>>
>> _______________________________________________
>> nfsv4 mailing list
>> nfsv4@ietf.org
>> https://www.ietf.org/mailman/listinfo/nfsv4
>>
>>
>>     
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4
>
>
>
>   