Disks do not auto-failover; must metaset take first

Question

Hello,
In Solaris 10, Cluster 3.2, two node:local zone cluster.
The SAN disks do not auto-failover when I do a switch on the resource group. Instead, to fail over, I must:
- clrg online fails
- manually mount the SAN disks to the node
- clrg online success
To fail over to other node, must
- clrg switch [2nd node] fails with messages:
# Dec 10 11:03:35 pluto SC[SUNW.HAStoragePlus:6,ftp4-com-rg,ftp4-data-has,hastorageplus_prenet_start]: Mount of /zones/ftp_zone/root/ftp_user_data failed: (33) mount: /zones/ftp_zone/root/ftp_user_data: Device busy
Dec 10 11:03:35 pluto SC[SUNW.HAStoragePlus:6,ftp4-com-rg,ftp4-data-has,hastorageplus_prenet_start]: Failed to mount: /zones/ftp_zone/root/ftp_user_data in global or local zone
- so now the rg is offine
- [on 2nd node] metaset -s [setname] -t so 2nd node becomes owner
- [on 2nd node] manually mount the disk
- clrg switch to 2nd node is successful
So appears the rg is not failing over b/c metaset is owned by one node and doesn't release them so the other node can take ownership. Do I need to do a metaset -r [release] or metaset -A enable ??
vfstab has been updated:
/dev/md/ftpbin/dsk/d110 /dev/md/ftpbin/rdsk/d110 /zones/ftp_zone/root/mysql_data ufs 1 no logging
/dev/md/ftpdata/dsk/d111 /dev/md/ftpdata/rdsk/d111 /zones/ftp_zone/root/ftp_user_data ufs 1 no logging
jupiter /etc # metaset
Set name = ftpbin, Set number = 1
Host Owner
jupiter Yes
pluto
Driv Dbase
d2 Yes
Set name = ftpdata, Set number = 2
Host Owner
jupiter Yes
pluto
Driv Dbase
d3 Yes
Edited by: onetree on Dec 10, 2008 10:02 PM

Answer 1

Ok, I found the cause. Need help finding solution.
I verified I have patch 124918-02, which is obsoleted by 120011-14
http://sunsolve.sun.com/search/document.do?assetkey=1-26-102546-1
The cause is in device groups and/or metasets. Need to get rid of ftpbin group, which points to the same disks as ftpabin.
pluto:/dev/md# ls -l
total 14
lrwxrwxrwx 1 root root 31 Jul 10 13:05 admin -> ../../devices/pseudo/md@0:admin
drwxr-xr-x 2 root root 512 Aug 18 11:13 dsk
lrwxrwxrwx 1 root root 8 Dec 10 16:13 ftpabin -> shared/3
lrwxrwxrwx 1 root root 8 Dec 10 16:13 ftpbin -> shared/1
lrwxrwxrwx 1 root root 8 Dec 10 16:13 ftpdata -> shared/2
drwxr-xr-x 2 root root 512 Aug 18 11:13 rdsk
lrwxrwxrwx 1 root root 42 Sep 25 18:17 shared -> ../../global/.devices/node@2/dev/md/shared
pluto:/dev/md/shared# find . -ls
7 1 drwxrwxr-x 5 root root 512 Dec 10 15:42 .
4291 1 drwxr-xr-x 4 root root 512 Dec 3 14:01 ./1
4292 1 drwxr-xr-x 2 root root 512 Dec 10 16:13 ./1/dsk
4300 1 lrwxrwxrwx 1 root root 44 Dec 10 16:13 *./1/dsk/d110* -> ../../../../../devices/pseudo/md@0:1,110,blk
4295 1 drwxr-xr-x 2 root root 512 Dec 10 16:13 ./1/rdsk
4304 1 lrwxrwxrwx 1 root root 44 Dec 10 16:13 *./1/rdsk/d110* -> ../../../../../devices/pseudo/md@0:1,110,raw
4298 1 drwxr-xr-x 4 root root 512 Dec 3 14:22 ./2
4299 1 drwxr-xr-x 2 root root 512 Dec 10 16:13 ./2/dsk
4296 1 lrwxrwxrwx 1 root root 44 Dec 10 16:13 ./2/dsk/d111 -> ../../../../../devices/pseudo/md@0:2,111,blk
4302 1 drwxr-xr-x 2 root root 512 Dec 10 16:13 ./2/rdsk
4310 1 lrwxrwxrwx 1 root root 44 Dec 10 16:13 ./2/rdsk/d111 -> ../../../../../devices/pseudo/md@0:2,111,raw
4305 1 drwxr-xr-x 4 root root 512 Dec 10 15:42 ./3
4306 1 drwxr-xr-x 2 root root 512 Dec 10 16:13 ./3/dsk
4307 1 lrwxrwxrwx 1 root root 44 Dec 10 16:13 *./3/dsk/d110* -> ../../../../../devices/pseudo/md@0:3,110,blk
4309 1 drwxr-xr-x 2 root root 512 Dec 10 16:13 ./3/rdsk
4311 1 lrwxrwxrwx 1 root root 44 Dec 10 16:13 *./3/rdsk/d110* -> ../../../../../devices/pseudo/md@0:3,110,raw
# cldg status
Device Group Name Primary Secondary Status
ftpbin - - Offline
ftpdata jupiter pluto Online
ftpabin jupiter pluto Online
# cldg show
Device Group Name: ftpbin
Type: SVM
failback: false
Node List: pluto
preferenced: false
numsecondaries: 1
diskset name: ftpbin
pluto:/dev/md# cldg delete ftpbin
cldg: (C795573) This operation is not allowed for Solaris Volume Manager device groups.
cldg: (C747142) Use the metaset(1M) or metassist(1M) commands to modify or delete Solaris Volume Manager device groups.
cldg: (C178871) Deletion of device group "ftpbin" failed.
pluto:/dev/md# metaclear -s ftpbin
metaclear: pluto: setname "ftpbin": no such set
pluto:/dev/md# metaset
Set name = ftpdata, Set number = 2
Host Owner
jupiter
pluto
Driv Dbase
d3 Yes
Set name = ftpabin, Set number = 3
Host Owner
jupiter
pluto
Driv Dbase
d2 Yes
pluto:/dev/md# metaset -s ftpbin
metaset: pluto: setname "ftpbin": no such set
pluto:/dev/md/shared# metaset -s ftpbin -C purge
metaset: pluto: setname "ftpbin": no such set
Also tried devfsadm -C and boot -r
Ran 'metaclusterbinddevs' to recreate the device links, but no diff.
# /usr/cluster/lib/sc/metaclusterbinddevs