NFS write performance 6 times slower than read

Question

Hi all,
I built my self a new homeserver and want to use NFS to export stuff to the clients. Problem is that I get a big difference in writing and reading from/to the share. Everything is connected by GBit Network, and raw network speed is fine.
Reading on the clients yields about 31MByte/s which is almost the native speed of the disks (which are luks-encrypted). But writing to the share gives only about 5.1MByte/s in the best case. Writing to the disks internally gives about 30MByte/s too. Also writing with unencrypted rsync from the client to the server gives about 25-30MByte/s, so it is definitely not a network or disk problem. So I wonder if there is anything that I could do to improve the Write-Performance of my NFS-shares. Here is my config which gives the best results so far:
Server-Side:
/etc/exports
/mnt/data 192.168.0.0/24(rw,async,no_subtree_check,crossmnt,fsid=0)
/mnt/udata 192.168.0.0/24(rw,async,no_subtree_check,crossmnt,fsid=1)
/etc/conf.d/nfs-server.conf
NFSD_OPTS=""
NFSD_COUNT="32"
PROCNFSD_MOUNTPOINT=""
PROCNFSD_MOUNTOPTS=""
MOUNTD_OPTS="--no-nfs-version 1 --no-nfs-version 2"
NEED_SVCGSSD=""
SVCGSSD_OPTS=""
Client-Side:
/etc/fstab
192.168.0.1:/mnt/data /mnt/NFS nfs rsize=32768,wsize=32768,intr,noatime 0 0
Additional Infos:
NFS to the unencrypted /mnt/udata gives about 20MByte/s reading and 10MByte/s writing.
Internal Speed of the discs is about 37-38MByte/s reading/writing for the encrypted one, and 44-45MByte/s for the unencrypted (notebook-hdd)
I Noticed that the load average on the server goes over 10 while the CPU stays at 10-20%
So if anyone has any idea what might go wrong here please let me know. If you need more information I will gladly provide it.
TIA
seiichiro0185
Last edited by seiichiro0185 (2010-02-06 13:05:23)

Answer 1

Your rsize and wsize looks way too big. I just use defaults and it runs fine.
I don't know what your server is but I plucked this from BSD Magazine.
There is one point worth mentioning here, modern Linux usually uses wsize and rsize 8192 by default and that can cause problems with BSD servers as many support only wsize and rsize 1024. I suggest you add the option -o wsize=1024,rsize=1024 when you mount the share on your Linux machines.
You also might want to check here for some optimisations http://www.linuxselfhelp.com/howtos/NFS … WTO-4.html
A trick to increase NFS write performance is to disable synchronous writes on the server. The NFS specification states that NFS write requests shall not be considered finished before the data written is on a non-volatile medium (normally the disk). This restricts the write performance somewhat, asynchronous writes will speed NFS writes up. The Linux nfsd has never done synchronous writes since the Linux file system implementation does not lend itself to this, but on non-Linux servers you can increase the performance this way with this in your exports file:
Last edited by sand_man (2010-03-03 00:23:23)