NFSv4: Server returns NFS4ERR_GRACE on open

Question

I've problems reading files from a NFSv4 exported ZFS dataset.
Server: Solaris 10 x86 u5 (5/08)
Clients: Solaris 10 x86 u5 (5/08)
I created a new zfs dataset:
server# zfs create mypool/export/test
server# zfs set sharenfs=ro mypool/export/test
server# zfs share mypool/export/testThat seems to work. I'm able to mount the export on the client:
client# mount server:/export/test /mntA 'ls /mnt' returns correct list of files, but if I try to read a file no data is transferred and the reading process just hangs (e.g. cp /mnt/file1.tar /var/tmp/.)
If I use NFSv3, it just works fine (mount -o vers=3).
I used snoop to dump the traffic on the network interface:
Client -> Server Call: readdir
Server -> Client Reply: readdir, Status: NFS4_OK
Client -> Server Call: getattr
Server -> Client Reply: getattr, Status: NFS4_OK
Client -> Server Call: access
Server -> client Reply: access, Status: NFS4_OK
Client -> Server Call: open
Server -> client Reply: open, Status: NFS4ERR_GRACE
[10 seconds]
Client -> Server Call: open
Server -> client Reply: open, Status: NFS4ERR_GRACE
[10 seconds]
Client -> Server Call: open
Server -> client Reply: open, Status: NFS4ERR_GRACE
[10 seconds]
and so on...
Why does the server reply with status NFS4ERR_GRACE?
Both machines are test boxes without any other services running, no other NFS exports.
I had the same issue with a Linux client (Debian etch, kernel 2.6.18).

Answer 1

You don't specifically mention whether this is running under Solaris Cluster or not, but given you've posted to the clustering group, I'll assume you are. If som you cannot share ZFS like that. You need to switch this off see http://docs.sun.com/app/docs/doc/820-2565/z4000275997776?l=en&a=view&q=ZFS.
If this is a plain Solaris ZFS question, you'd probably be better off posting to the Solaris list.
Tim
---