Warning: offline or reservation conflict

Question

Hello!
Since last week I am trying to rebuild a previously working cluster setup. Two servers with two HBAs each are cross-connected to one Sun 6140. After enabling the multipath-software, mpathadm shows me two working paths for each server.
Then I installed the cluster software, formed the cluster and patched it. Now I get those messages on the console while booting:
WARNING: /scsi_vhci/ssd@g600a0b80002ae69c00000df547b189dc (ssd5):
        offline or reservation conflict
WARNING: /scsi_vhci/ssd@g600a0b800029e43800000d5b47b18b43 (ssd4):
        offline or reservation conflictIf there are any zpools on the devices above, I get an error for their malfunction as well. Also quorum does not work:
WARNING: CMM: Reading reservation keys from quorum device /dev/did/rdsk/d4s2 failed.
NOTICE: CMM: Cluster doesn't have operational quorum yet; waiting for quorum.Does anyone know what causes this? Can I trust mpathadm if it says a path is "OK"?
TIA
Stephan

Answer 1

When the node boots, you will always see the offline messages until the node joins the cluster and is given the keys to access the shared disks.
Just to cover the basics:
1) The hosts are setup as solaris mpxio (explicit failover)
2) Each host has a connection to each controller (i.e., there are no cabling issues where one host goes to one controller and the other host goes to the other controller.)
You should see (via luxadm) one primary and one secondary path, only 1 of which is active like this:
root[alinghi:/root]# luxadm display /dev/rdsk/c6t600A0B800029A2780000077446CC68BCd0s2
DEVICE PROPERTIES for disk: /dev/rdsk/c6t600A0B800029A2780000077446CC68BCd0s2
Vendor: SUN
Product ID: CSM200_R
Revision: 0619
Serial Num: SG11111111
Unformatted capacity: 204800.000 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x300
Maximum prefetch: 0x300
Device Type: Disk device
Path(s):
/dev/rdsk/c6t600A0B800029A2780000077446CC68BCd0s2
/devices/scsi_vhci/disk@g600a0b800029a2780000077446cc68bc:c,raw
Controller /dev/cfg/c4
Device Address 201600a0b829a278,1
Host controller port WWN 210000e08b85ce7c
Class primary
State ONLINE
Controller /dev/cfg/c5
Device Address 202700a0b829a278,1
Host controller port WWN 210100e08ba5ce7c
Class secondary
State STANDBY
Having two online paths would be a bad thing in your case and result in the issues you are seeing.
You will also want to verify that the 2nd node is up. At that point the system will have 2 out of 3 votes and thus operational quorum.
Of course, if the cables are correct and both nodes running, then somebody with better troubleshooting skills than me will have to step in :-)