Placing Vote Disk on diferents diskgroups - 11gR2

Hello, Oracle experts.
I have the above problem.
I finished the Rac instalation on Oracle 11gR2.
I would like to place 3 Vote Disk and 3 OCR, on ASM Diskgroup.
[oracle@srv15040 ~]$ crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [11.2.0.2.0]
[root@srv15040 bin]# ./crsctl query css votedisk
## STATE File Universal Id File Name Disk group
1. ONLINE 42a1aebf2ed84fd4bf7fba68b0af974e (/dev/oracleasm/disks/CRSVOL1) [OCRVT1]
[oracle@srv15040 ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2480
Available space (kbytes) : 259640
ID : 1528729410
Device/File Name : +OCRVT1
Device/File integrity check succeeded
Device/File Name : +OCRVT2
Device/File integrity check succeeded
Device/File Name : +OCRVT3
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user
How to place the Vote Disk on OCRVT2 and OCRVT3 ?

Hi;
I suggest move your issue on Forum Home » Grid Computing » Automatic Storage Management which you can get more quick response
Regard
Helios

Similar Messages

Adding ocrmirror and voting disk on another diskgroup 11gr2 RAC

Hi Gurus,
Can i add ocrmirror on another diskgroup(external redundancy) and voting disk(normal) on anotherdiskgroup (replace) in 11gr2 RAC while databases are up and running?

Hi
in Oracle 11gr2 voting disk are backed up automatically in the ocr part of any configuration change
Voting disk is automatically restored to any added voting disk
You can migrate voting fıska from non asm to asm without
Taking download cluster
To add a voting disk to asm
Crsctl replace votedisk +diskgroup
Ocrconfig -add +datagroup2
Oracle manage up to five redundant ocr locations
Hope this helps
Zekeriya Besiroglu
Http://zekeriyabesiroglu.blogspot.com

Restore Vote Disk in 11gr2

Hi,
In 11gR2, Vote Disk and OCR can reside in ASM . A New Feature in 11gR2, Vote Disk auto backup with OCR in one backup file. But if my vote disks failed, how can i restore it. Please guide me how to do ?
Thanks

Restoring Voting Disks
If all of the voting disks are corrupted, then you can restore them, as follows:
http://docs.oracle.com/cd/E11882_01/rac.112/e16794/votocr.htm#BGBCHCBI
Please close the thread if you feel you have the answer and keep the Oracle forum clean.
https://forums.oracle.com/forums/ann.jspa?annID=885

Move OCR and Vote Disk to a new SAN

Hi,
Can you guys help me with some links/documents for moving OCR and Vote disk files from old SAN to new SAN.
Both SAN will have ocfs2 formatted 3 partitions on which ocr and votedisk will be placed (normal redundancy).
Oracle version 11g R2,
OS linux 5.5
RAC nodes : 2
Datafiles on ASM
OCR/Votedisk on ocfs2
Regards,
Ajay

user7443016 wrote:
Thanks for the reply, but all the examples were showing either ocfs2 to ASM or raw to ocfs2. could you just point out the high level steps for moving 3 ocr & votedisk files from one san to another(ocfs to ocfs).
Hi Ajay,
Steps will be same . It doesnt matter whether it is raw to ocfs2 or ocfs to ocfs.
There will be change only how do you assign storage for OCR/voting disk.
I have another doubt that, i already have 3 ocr files, can i add more ocr files pointing to another mount point? the same goes with votedisk.
You can use replace option so there is no need to add 4th OCR disk.
i found steps in one of the website: i have few doubts in it, could any one clarify it...All the steps seems to be fine.
Moving OCR
1) create mount point(new san - /u05) in all nodes
2) precreate ocr files in new mount points (i am not sure how to do it? is it just touch and filename, its a ocfs2 file system)on ocfs you need to create file as below :
dd if=/dev/zero of=/ocfs/OCR/ocrnew.dbf bs=1M count=128
Also I would suggest to have a look at :
'ocrconfig -replace ocr' Fails With PROT-16 [ID 444757.1]
RAC Assurance Support Team: RAC Starter Kit and Best Practices (Linux) [ID 811306.1]
Regards
Rajesh

Recover OCR and VOTE disk after complete corruption of ASM disk groups.

Hi Gurus,
I am simulating a recovery situation to perform recover of OCR and Vote files after complete corruption of ASM related disks and diskgroups. I have setup my environment as follows:\
Environment: RAC
OS: OEL 5.5 32-bit
GI Version: 11.2.0.2.0
ASM Disk groups: +OCR, +DATA
OCR, Vote Files location: +OCR
ASM Redundancy: External
ASM Disks: /dev/asm-disk1, /dev/asm-disk2
/dev/asm-disk1 - mapped on +OCR
/dev/asm-disk2 - mapped on +DATA
With the above configuration in place I have manually corrupted +OCR, +DATA diskgroups with dd command. I used this command to completely corrupt +OCR disk group.
dd if=/dev/zero of=/dev/asm-disk1. I have manual backups as well as automatic backups of OCR and Vote disk. I am not using ASMLib.
I followed this link:
http://docs.oracle.com/cd/E11882_01/rac.112/e17264/adminoc.htm#TDPRC237
When I tried to recover OCR file, I could not do so as there is no such diskgroup which ASM can restore the OCR, Voting disk to. I could not Re-create OCR and DATA diskgroups as I cannot connect to ASM instance. If you have a solution or workaround for my situation please describe it. That will be greatly appreciated.
Thanks and Regards,
Suresh.

Please go through the following document which have the detailed steps to restore the OCR
How to restore ASM based OCR after complete loss of the CRS diskgroup on Linux/Unix systems [ID 1062983.1]

OCR/Votting disk change location

Hi,
I need to move my 2 nodes RAC (Solaris 10 / Oracle 10g (10.2.0.3)) to a new data center.
My OCR and Votting disks are on a SAN (x) and on the new datacenter I want to put them on a new SAN (y).
How can I perform this server movement without losing the current RAC/Clusterware configuration?
I was planning to do the following:
- Backup current databases
- Backup spfile
- Backup Diskgroups/ASM configuration
- Delete databases
- Delete Diskgroups/ASM
- Shutdown CRS (crsctl stop crs)
- Backup OCR disks (ocrconfig -export export_filename)
- Backup Votting disks (dd if=voting_disk_name of=backup_file_name)
- Turn off the servers
- Move them to the new Datacenter
- Attch the new SAN
- Restore Votting disks (dd if=backup_file_name of=voting_disk_name)
- Restore OCR disks ( ocrconfig -import export_filename)
- Then recreate Diskgroups/ASM
- Restore databases
Is this the right procedure/commands to accomplish it without losing the RAC configuration?
I am trying to avoid rebuilding the whole RAC from scratch.
Thanks,
AShum

I forgot to mention that there is no connection between the old and new datacenter. If there would, I would just present the new SAN space from the new datacenter to the old one, so I can just add the new OCR/Votting disk to the current ones (mirror) and then remove the old disks.
Same for the ASM and databases, but unfortunally, there is no connection between this two datacenter.
It is like moving the servers from one state to another.
Thanks,

About OCR and Vote Disk size

Hallo,
I have a simple question about OCR and Vote Disk in a 10g/11g RAC setup:
official documentation speaks about sizing OCR and Vote Disk about 280 MB, but do they mean each OCR and Vote disk LUN ?
In other words, for 2 OCR and 3 Vote disk I need 280*5 MB ( in multiple o single LUN)
Is it so?
thx in advance

ottocolori wrote:
Hallo,
I have a simple question about OCR and Vote Disk in a 10g/11g RAC setup:
official documentation speaks about sizing OCR and Vote Disk about 280 MB, but do they mean each OCR and Vote disk LUN ?
In other words, for 2 OCR and 3 Vote disk I need 280*5 MB ( in multiple o single LUN)
Is it so?
thx in advanceYes. Also see Re: Reduce size of OCR file

OCR and vote disk Allocation for shared raw storage with Solaris 10 questio

Hi all,
Current environment is Solaris 10 SPARC 64 bit OS with Hitachi SAN for shared storage and Sun E6900 servers.
For Oracle 10g RAC (10.2) and ASM, I am setting up the vote disk and OCR files on shared raw storage area network.
Assume that I have a 35Gb LUN carved out on raw device for the vote disk and OCR for a 2-node Oracle 10g RAC cluster and since the vote disk and OCR only require 120MB of storage, is there a way to use only a 120MB slice from the LUN or do I need to allocate the entire LUN/raw device to the vote disk?
I am looking for a way to avoid wasting space for the OCR and vote disk with my 10g RAC cluster and we are using raw devices and Oracle 10g RAC Clusterware with Veritas filesystem for the disk management. Can I have several slices on Solaris 10 with raw shared storage (Hitachi) to triple mirror my OCR and vote disk files?
It seems odd to have to use an entire 35GB LUN just for 2 small files: vote disk and OCR. Is there a way to partition a 120MB slice on one of the devices and allocate the rest to ASM?

Hi
In our RAC env.
we have 1GB LUN for OCR and Voting Disk and 100GB LUN for ASM disk. we have 4 Node RAC env on AIX with 2 SAN storage on Hitachi system for our Application.
we keept 1st Voting diks on SAN-1 2nd Voting Disk on SAN-2 and third voting on SAN-3.
OCR disk on SAN1 and OCR_MIRROR Disk on SAN-2.
we have ASM DIsk with Normal Redundancy . Failure Group 1 on SAN-1 and
Failure Group 2 on SAN-2
Regards
Bharat

Is it possible to move a disk from a diskgroup to another one ?

Hi,
I did not find the answer so here is my question :
My environment is Oracle 11g (11.1.0.6.0) database on OEL5U3.
The system contains 2 diskgroups (DATA and RECOVERY) which respectively contains 1 disk (DATA contains VOL1 and RECOVERY contains VOL2). VOL1 and VOL2 are the ASM disk names assigned by oracleasm to /dev/dpdb1 and /dev/dpdc1
SQL> select name,GROUP_NUMBER,TYPE FROM V$ASM_DISKGROUP;
NAME GROUP_NUMBER TYPE
DATA 1 EXTERN
RECOVERY 2 EXTERN
SQL> SELECT NAME,PATH FROM V$ASM_DISK;
NAME PATH
DATA_0000 /dev/oracleasm/disks/VOL1
RECOVERY_0000 /dev/oracleasm/disks/VOL2
My ultimate goal would be to use a snapshot-copy of /dev/dpdb1 (VOL1) to create a new diskgroup on the same server.
Let's say that my snapshot-copy of LUN /dev/dpdb is seen as /dev/dpdh:
# /etc/init.d/oracleasm querydisk /dev/dpdh1
Device "/dev/dpdh1" is marked an ASM disk with the label "VOL1" <-- the label on the disk is of course VOL1
# /usr/sbin/oracleasm renamedisk -f /dev/dpdh1 VOL3 <-- update of the label
Writing disk header: done
Instantiating disk "VOL3": done
# /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3 <-- the new disk is now listed
Then I try to create the new diskgroup:
SQL> create diskgroup TIMEVIEW external redundancy disk '/dev/oracleasm/disks/VOL3';
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15033: disk '/dev/oracleasm/disks/VOL3' belongs to diskgroup "DATA"
The same command with force option:
SQL> create diskgroup TIMEVIEW external redundancy disk '/dev/oracleasm/disks/VOL3' force;
Diskgroup created.
The issue here is that the newly created diskgroup is empty ! it does not contains original files. Did I miss something ?
Edited by: user11239258 on 21 oct. 2009 01:06
Edited by: user11239258 on 21 oct. 2009 01:07

The md_backup and md_restore are finally not helpful in my case. it doesn't change the fact that ASM metadata are written in the disk header.
I could try to update the header with dd and hexadecimal editor but I'm looking for a decent option here.
The only solution I've found is to shut down ASM instance, move the block special file /dev/oracleasm/disks/VOL1 pointing to the original disk to a safe location, create a new block special file using mknod to point on my disk (the snapshot), restart ASM instance and use asmcmd cp command to retrieve needed files.
Conclusion : snapshot backup is fast but recovery based on snapshot is slow dealing with ASM.
RFE : A command similar to vgimport would be nice to have with ASM disks, it would allow the creation of new diskgroup without losing the content of an ASM disk or a group of disk.

Error replacing vote disk

Hi all,
I've created an Oracle 11g R2 RAC with two nodes on OEL 5.6 (VMWare ESX).
So, I want to replace the vote disk (into ASM) to other device.
We've added five shared disks, and I've created five partitions by using fdisk
to the following:
/dev/sdg1
/dev/sdh1
/dev/sdi1
/dev/sdj1
/dev/sdk1So, the result of ls -ltr /dev/sd* is:
brw-r----- 1 root   disk 8,   0 Mar 18 15:51 /dev/sda
brw-r----- 1 root   disk 8,   2 Mar 18 15:51 /dev/sda2
brw-r----- 1 root   disk 8, 16 Mar 18 15:51 /dev/sdb
brw-r----- 1 root   disk 8, 32 Mar 18 15:51 /dev/sdc
brw-r----- 1 root   disk 8, 48 Mar 18 15:51 /dev/sdd
brw-r----- 1 root   disk 8, 64 Mar 18 15:51 /dev/sde
brw-r----- 1 root   disk 8, 80 Mar 18 15:51 /dev/sdf
brw-r----- 1 root   disk 8,   3 Mar 18 15:52 /dev/sda3
brw-r----- 1 root   disk 8,   1 Mar 18 15:52 /dev/sda1
brw-r----- 1 root   disk 8, 17 Mar 18 15:52 /dev/sdb1
brw-r----- 1 root   disk 8, 65 Mar 18 15:52 /dev/sde1
brw-r----- 1 root   disk 8, 33 Mar 18 15:52 /dev/sdc1
brw-r----- 1 root   disk 8, 81 Mar 18 15:52 /dev/sdf1
brw-r----- 1 root   disk 8, 49 Mar 18 15:52 /dev/sdd1
brw-r----- 1 root   disk 8, 96 Mar 18 16:18 /dev/sdg
brw-r----- 1 root   disk 8, 112 Mar 18 16:19 /dev/sdh
brw-r----- 1 root   disk 8, 128 Mar 18 16:19 /dev/sdi
brw-r----- 1 root   disk 8, 144 Mar 18 16:20 /dev/sdj
brw-r----- 1 root   disk 8, 160 Mar 18 16:20 /dev/sdk
brw-r----- 1 oracle root 8, 161 Mar 18 17:17 /dev/sdk1
brw-r----- 1 oracle root 8, 145 Mar 18 17:17 /dev/sdj1
brw-r----- 1 oracle root 8, 129 Mar 18 17:17 /dev/sdi1
brw------- 1 root   root 8, 113 Mar 18 17:17 /dev/sdh1
brw------- 1 root   root 8, 97 Mar 18 17:17 /dev/sdg1but, using the root user, when I try to replace the vote disk,
It return an error:
[root@rac1 ~]# /u01/app/11.2.0/grid/bin/crsctl replace votedisk /dev/sdi1 /dev/sdj1 /dev/sdk1
Now formatting voting disk: /dev/sdi1.
Now formatting voting disk: /dev/sdj1.
Now formatting voting disk: /dev/sdk1.
CRS-4256: Updating the profile
CRS-4602: Failed 8 to add voting file 984aee18b36a4fc4bf2652f36cc56ee8.
CRS-4602: Failed 8 to add voting file 994466304f9d4f41bfd34cb65a5c171b.
CRS-4602: Failed 8 to add voting file 3049f5bd91954f14bf62144995279d5a.
Failure 8 with Cluster Synchronization Services while deleting voting disk.
CRS-4000: Command Replace failed, or completed with errors.Somebody can help with this error?
thank you very much!!!!

Hi all,
Well, in the alert file, I can see the following:
[cssd(5437)]CRS-1638:Unable to locate voting file with ID 08537e4d-3ae74fa7-bf6f6fc5-1bd94653 that is being added to the
list of configured voting files; details at (:CSSNM00022:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.logAnd in the ocssd.log:
2011-03-21 10:10:29.414: [    CSSD][1129027904]clssnmReadDiscoveryProfile: voting file discovery string(+asm,/dev/sdi1,/dev/sdj1,/dev/sdk1)
2011-03-21 10:10:29.414: [    CSSD][1129027904]clssnmvDDiscThread: using discovery string +asm,/dev/sdi1,/dev/sdj1,/dev/sdk1 for voting file add
2011-03-21 10:10:29.414: [   SKGFD][1129027904]Discovery with str:/dev/sdi1,/dev/sdj1,/dev/sdk1:
2011-03-21 10:10:29.414: [   SKGFD][1129027904]UFS discovery with :/dev/sdi1:
2011-03-21 10:10:29.416: [   SKGFD][1129027904]Fetching UFS disk :/dev/sdi1:
2011-03-21 10:10:29.416: [   SKGFD][1129027904]OSS discovery with :/dev/sdi1:
2011-03-21 10:10:29.416: [   SKGFD][1129027904]Discovery with asmlib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: str :/dev/sdi1:
2011-03-21 10:10:29.416: [   SKGFD][1129027904]Discovery advancing to nxt string :/dev/sdj1:
2011-03-21 10:10:29.416: [   SKGFD][1129027904]UFS discovery with :/dev/sdj1:
2011-03-21 10:10:29.418: [   SKGFD][1129027904]Fetching UFS disk :/dev/sdj1:
2011-03-21 10:10:29.418: [   SKGFD][1129027904]OSS discovery with :/dev/sdj1:
2011-03-21 10:10:29.418: [   SKGFD][1129027904]Discovery with asmlib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: str :/dev/sdj1:
2011-03-21 10:10:29.418: [   SKGFD][1129027904]Discovery advancing to nxt string :/dev/sdk1:
2011-03-21 10:10:29.418: [   SKGFD][1129027904]UFS discovery with :/dev/sdk1:
2011-03-21 10:10:29.419: [   SKGFD][1129027904]Fetching UFS disk :/dev/sdk1:
2011-03-21 10:10:29.419: [   SKGFD][1129027904]OSS discovery with :/dev/sdk1:
2011-03-21 10:10:29.419: [   SKGFD][1129027904]Discovery with asmlib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: str :/dev/sdk1:
2011-03-21 10:10:29.419: [   SKGFD][1129027904]Handle 0x148b230 from lib :UFS:: for disk :/dev/sdi1:
2011-03-21 10:10:29.419: [   SKGFD][1129027904]Handle 0x148b650 from lib :UFS:: for disk :/dev/sdj1:
2011-03-21 10:10:29.419: [   SKGFD][1129027904]Handle 0x144b8e0 from lib :UFS:: for disk :/dev/sdk1:
2011-03-21 10:10:29.420: [    CLSF][1129027904]Read header of dev:/dev/sdi1:vfile:
2011-03-21 10:10:29.420: [   SKGFD][1129027904]Lib :UFS:: closing handle 0x148b230 for disk :/dev/sdi1:
2011-03-21 10:10:29.420: [    CLSF][1129027904]Read header of dev:/dev/sdj1:vfile:
2011-03-21 10:10:29.420: [   SKGFD][1129027904]Lib :UFS:: closing handle 0x148b650 for disk :/dev/sdj1:
2011-03-21 10:10:29.420: [    CLSF][1129027904]Read header of dev:/dev/sdk1:vfile:
2011-03-21 10:10:29.420: [   SKGFD][1129027904]Lib :UFS:: closing handle 0x144b8e0 for disk :/dev/sdk1:
2011-03-21 10:10:29.420: [    CSSD][1129027904]clssnmvDiskVerify: discovered a potential voting file
2011-03-21 10:10:29.420: [   SKGFD][1129027904]Handle 0x148b670 from lib :UFS:: for disk :/dev/sdi1:
2011-03-21 10:10:29.420: [    CLSF][1129027904]Opened hdl:0x1417b30 for dev:/dev/sdi1:
2011-03-21 10:10:29.423: [    CSSD][1129027904]clssnmvDiskVerify: Successful discovery for disk /dev/sdi1, UID 50fa5bb3-359a4f28-bf00a2ec-a0078929, Pending CIN 0:0:1, Committed CIN 0:0:1
2011-03-21 10:10:29.423: [    CLSF][1129027904]Closing handle:0x1417b30
2011-03-21 10:10:29.423: [   SKGFD][1129027904]Lib :UFS:: closing handle 0x148b670 for disk :/dev/sdi1:
2011-03-21 10:10:29.423: [    CSSD][1129027904]clssnmvDiskVerify: discovered a potential voting file
2011-03-21 10:10:29.423: [   SKGFD][1129027904]Handle 0x148b670 from lib :UFS:: for disk :/dev/sdj1:
2011-03-21 10:10:29.423: [    CLSF][1129027904]Opened hdl:0x1405c80 for dev:/dev/sdj1:
2011-03-21 10:10:29.425: [    CSSD][1129027904]clssnmvDiskVerify: Successful discovery for disk /dev/sdj1, UID 78d8c731-19194fc2-bf4d7e0c-e87dfa5c, Pending CIN 0:0:1, Committed CIN 0:0:1
2011-03-21 10:10:29.425: [    CLSF][1129027904]Closing handle:0x1405c80
2011-03-21 10:10:29.426: [   SKGFD][1129027904]Lib :UFS:: closing handle 0x148b670 for disk :/dev/sdj1:
2011-03-21 10:10:29.426: [    CSSD][1129027904]clssnmvDiskVerify: discovered a potential voting file
2011-03-21 10:10:29.426: [   SKGFD][1129027904]Handle 0x148b670 from lib :UFS:: for disk :/dev/sdk1:
2011-03-21 10:10:29.426: [    CLSF][1129027904]Opened hdl:0x1417b30 for dev:/dev/sdk1:
2011-03-21 10:10:29.429: [    CSSD][1129027904]clssnmvDiskVerify: Successful discovery for disk /dev/sdk1, UID 5577a7c8-17134f07-bfbc3812-2f215f6b, Pending CIN 0:0:1, Committed CIN 0:0:1
2011-03-21 10:10:29.429: [    CLSF][1129027904]Closing handle:0x1417b30
2011-03-21 10:10:29.429: [   SKGFD][1129027904]Lib :UFS:: closing handle 0x148b670 for disk :/dev/sdk1:
2011-03-21 10:10:29.429: [    CSSD][1129027904]clssnmvDiskVerify: Successful discovery of 3 disks
2011-03-21 10:10:29.430: [   SKGFD][1129027904]Discovery with str::
2011-03-21 10:10:29.430: [   SKGFD][1129027904]UFS discovery with ::
2011-03-21 10:10:29.430: [   SKGFD][1129027904]OSS discovery with ::
2011-03-21 10:10:29.430: [   SKGFD][1129027904]Discovery with asmlib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: str ::
2011-03-21 10:10:29.430: [   SKGFD][1129027904]Fetching asmlib disk :ORCL:DISK1:
2011-03-21 10:10:29.431: [   SKGFD][1129027904]Fetching asmlib disk :ORCL:DISK2:
2011-03-21 10:10:29.431: [   SKGFD][1129027904]Fetching asmlib disk :ORCL:DISK3:
2011-03-21 10:10:29.431: [   SKGFD][1129027904]Fetching asmlib disk :ORCL:DISK4:
2011-03-21 10:10:29.431: [   SKGFD][1129027904]Fetching asmlib disk :ORCL:DISK5:
2011-03-21 10:10:29.431: [   SKGFD][1129027904]Handle 0x13faf80 from lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: for disk :ORCL:DISK1:
2011-03-21 10:10:29.431: [   SKGFD][1129027904]Handle 0x13fb3a0 from lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: for disk :ORCL:DISK2:
2011-03-21 10:10:29.431: [   SKGFD][1129027904]Handle 0x13fb7c0 from lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: for disk :ORCL:DISK3:
2011-03-21 10:10:29.431: [   SKGFD][1129027904]Handle 0x14406f0 from lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: for disk :ORCL:DISK4:
2011-03-21 10:10:29.431: [   SKGFD][1129027904]Handle 0x1551320 from lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: for disk :ORCL:DISK5:
2011-03-21 10:10:29.432: [    CLSF][1129027904]Read ASM header off dev:ORCL:DISK1:96:128
2011-03-21 10:10:29.432: [   SKGFD][1129027904]Lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: closing handle 0x13faf80 for disk :ORCL:DISK1:
2011-03-21 10:10:29.434: [    CLSF][1129027904]Read ASM header off dev:ORCL:DISK2:0:0
2011-03-21 10:10:29.434: [   SKGFD][1129027904]Lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: closing handle 0x13fb3a0 for disk :ORCL:DISK2:
2011-03-21 10:10:29.435: [    CLSF][1129027904]Read ASM header off dev:ORCL:DISK3:0:0
2011-03-21 10:10:29.435: [   SKGFD][1129027904]Lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: closing handle 0x13fb7c0 for disk :ORCL:DISK3:
2011-03-21 10:10:29.435: [    CLSF][1129027904]Read ASM header off dev:ORCL:DISK4:0:0
2011-03-21 10:10:29.435: [   SKGFD][1129027904]Lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: closing handle 0x14406f0 for disk :ORCL:DISK4:
2011-03-21 10:10:29.435: [    CLSF][1129027904]Read ASM header off dev:ORCL:DISK5:0:0
2011-03-21 10:10:29.435: [   SKGFD][1129027904]Lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: closing handle 0x1551320 for disk :ORCL:DISK5:
2011-03-21 10:10:29.435: [    CSSD][1129027904]clssnmvDiskVerify: discovered a potential voting file
2011-03-21 10:10:29.435: [   SKGFD][1129027904]Handle 0x148b230 from lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: for disk :ORCL:DISK1:
2011-03-21 10:10:29.435: [    CLSF][1129027904]Opened hdl:0x1419910 for dev:ORCL:DISK1:
2011-03-21 10:10:29.437: [    CSSD][1129027904]clssnmFindVF: found VF by vdin in the configured queue
2011-03-21 10:10:29.437: [    CSSD][1129027904]clssnmFindVF: Duplicate voting file found in the queue of previously configured disks queued(ORCL:DISK1|[3d2e004e-73584ff6-bfa6666f-6ce172f9]), found(|[3d2e004e-73584ff6-bfa6666f-6ce172f9])
2011-03-21 10:10:29.437: [    CSSD][1129027904]clssnmvDiskDestroy: removing the voting disk
2011-03-21 10:10:29.437: [    CLSF][1129027904]Closing handle:0x1419910
2011-03-21 10:10:29.437: [   SKGFD][1129027904]Lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: closing handle 0x148b230 for disk :ORCL:DISK1:
2011-03-21 10:10:29.437: [    CSSD][1129027904]clssnmvDiskVerify: file is not a voting file, cannot recognize on-disk signature for a voting
2011-03-21 10:10:29.437: [    CSSD][1129027904]clssnmvDiskVerify: file is not a voting file, cannot recognize on-disk signature for a voting
2011-03-21 10:10:29.437: [    CSSD][1129027904]clssnmvDiskVerify: file is not a voting file, cannot recognize on-disk signature for a voting
2011-03-21 10:10:29.437: [    CSSD][1129027904]clssnmvDiskVerify: file is not a voting file, cannot recognize on-disk signature for a voting
2011-03-21 10:10:29.437: [    CSSD][1129027904]clssnmvDiskVerify: Successful discovery of 0 disks

Accidently added 3 voting disk into a diskgroup

In a process to add a new diskgroup ,I added my voting disks as the disk in my diskgroup and as a result my RAC box on AIX got bounced .
Now when I am trying to start the CRS , getting the below error in crs log :
[cssd(663686)]CRS-1604:CSSD voting file is offline: /dev/rhdisk4. Details in /oracle/app/product/10gcrs/log/ccman_avalon/cssd
/ocssd.log.
2008-02-04 11:37:57.001
[cssd(663686)]CRS-1604:CSSD voting file is offline: /dev/rhdisk5. Details in /oracle/app/product/10gcrs/log/ccman_avalon/cssd
/ocssd.log.
2008-02-04 11:37:57.016
[cssd(663686)]CRS-1604:CSSD voting file is offline: /dev/rhdisk6. Details in /oracle/app/product/10gcrs/log/ccman_avalon/cssd
/ocssd.log.
Can you guide me how to get out of this situation . I am completely stuck and dont know where to go from here.
Thanks,
Anks

You may have to delete and add a new voting device (high level steps):
1) Shutdown CRS
2) Use:crsctl delete css votedisk <vote_disk_path> -force
3) Use:crsctl add css votedisk <vote_disk+path> -force
4) Check the voting disks status using: crsctl query css votedisk
HTH
Thanks
-Chandra

No of vote Vote Disk ?

Dear All,
Please let me Why oracle recommend odd number of disk. i.e. one.three,five etc.... ? Any specific reason behind this ?
Here in my scenario two vote disk are working fine without any problem.
Regard's
Awanish Kumar
Edited by: Awanish on Mar 26, 2009 1:09 AM

I think that the reason behind it is to avoid a tie between even number of voting disks, which is not possible in odd number of voting disks............

OCR and VOT disk, in the same asm disk or different disks ?! and how ?!

What Oracle recommends regarding the OCR and Voting disks in Rac 11gR2 on ASM ?!
Making them under the same disk or different disks ?!
and how ?!
Thanks a lot,
Regards,
Gehad.

Best practice is that OCR/Voting should be under ASM. Making them under the same disk or different disks depends upon the ASM diskgroup. If we want to keep it on seperate failover groups then we should create muliple failover group.
For external : 1 ; Normal : 3 and higher : 5 asm diskgroups.
I hope i would be clear ith my answers.
Regards,
Dheeraj Vaish

Please Help - When I try to add ASM Disk to ASM Diskgroup it crashes Server

We are using a Pillar SAN and have LUNS Created and are using the following multipath device: (I'm a DBA more then anything else... but I am rather familiar with linux .... SAN Hardware not so much)
Device Size Mount Point
/dev/dpda1 11G /u01
The Above device is working fine... Below are the ASM Disks being Created
Device Size Oracle ASM Disk Name
/dev/dpdb1 198G ORCL1
/dev/dpdc1 21G SIRE1
/dev/dpdd1 21G CART1
/dev/dpde1 21G SRTS1
/dev/dpdf1 21G CRTT1
I try to create to the first ASM Disk
/etc/init.d/oracleasm createdisk ORCL1 /dev/dpdb1
Marking disk "ORCL1" as an ASM disk: [FAILED]
So I check the oracleasm log:
#cat /var/log/oracleasm
Device "/dev/dpdb1" is not a partition
I did some research and found that this is a common problem with multipath devices and to work around it you have to use asmtool
# /usr/sbin/asmtool -C -l /dev/oracleasm -n ORCL1 -s /dev/dpdb1 -a force=yes
asmtool: Device "/dev/dpdb1" is not a partition
asmtool: Continuing anyway
now I scan and list the disks
# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks: [ OK ]
# /etc/init.d/oracleasm listdisks
ORCL1
Here is whats going on in /var/log/messages when I run the oracleasm scandisks command
# date
Fri Aug 14 13:51:58 MST 2009
# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks: [ OK ]
cat /var/log/messages | grep "Aug 14 13:5"
Aug 14 13:52:06 seer kernel: dpdb: dpdb1
Aug 14 13:52:06 seer kernel: dpdc: dpdc1
Aug 14 13:52:06 seer kernel: dpdd: dpdd1
Aug 14 13:52:06 seer kernel: dpde: dpde1
Aug 14 13:52:06 seer kernel: dpdf: dpdf1
Aug 14 13:52:06 seer kernel: dpdg: dpdg1
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: printk: 30 messages suppressed.
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: sda : READ CAPACITY failed.
Aug 14 13:52:06 seer kernel: sda : status=1, message=00, host=0, driver=08
Aug 14 13:52:06 seer kernel: sd: Current: sense key: Illegal Request
Aug 14 13:52:06 seer kernel: Add. Sense: Logical unit not supported
Aug 14 13:52:06 seer kernel:
Aug 14 13:52:06 seer kernel: sda: test WP failed, assume Write Enabled
Aug 14 13:52:06 seer kernel: sda: asking for cache data failed
Aug 14 13:52:06 seer kernel: sda: assuming drive cache: write through
Aug 14 13:52:06 seer kernel: sda:end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: Dev sda: unable to read RDB block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: unable to read partition table
Aug 14 13:52:06 seer kernel: SCSI device sdb: 21502464 512-byte hdwr sectors (11009 MB)
Aug 14 13:52:06 seer kernel: sdb: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdb: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdb: sdb1
Aug 14 13:52:06 seer kernel: SCSI device sdc: 421476864 512-byte hdwr sectors (215796 MB)
Aug 14 13:52:06 seer kernel: sdc: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdc: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdc: sdc1
Aug 14 13:52:06 seer kernel: SCSI device sdd: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdd: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdd: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdd: sdd1
Aug 14 13:52:06 seer kernel: SCSI device sde: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sde: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sde: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sde: sde1
Aug 14 13:52:06 seer kernel: SCSI device sdf: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdf: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdf: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdf: sdf1
Aug 14 13:52:06 seer kernel: SCSI device sdg: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdg: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdg: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdg: sdg1
Aug 14 13:52:06 seer kernel: SCSI device sdh: 2107390464 512-byte hdwr sectors (1078984 MB)
Aug 14 13:52:06 seer kernel: sdh: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdh: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdh: sdh1
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sdi, logical block 0
Aug 14 13:52:06 seer kernel: sdi : READ CAPACITY failed.
Aug 14 13:52:06 seer kernel: sdi : status=1, message=00, host=0, driver=08
Aug 14 13:52:06 seer kernel: sd: Current: sense key: Illegal Request
Aug 14 13:52:06 seer kernel: Add. Sense: Logical unit not supported
Aug 14 13:52:06 seer kernel:
Aug 14 13:52:06 seer kernel: sdi: test WP failed, assume Write Enabled
Aug 14 13:52:06 seer kernel: sdi: asking for cache data failed
Aug 14 13:52:06 seer kernel: sdi: assuming drive cache: write through
Aug 14 13:52:06 seer kernel: sdi:end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:06 seer last message repeated 4 times
Aug 14 13:52:06 seer kernel: Dev sdi: unable to read RDB block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:06 seer kernel: unable to read partition table
Aug 14 13:52:06 seer kernel: SCSI device sdj: 21502464 512-byte hdwr sectors (11009 MB)
Aug 14 13:52:06 seer kernel: sdj: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdj: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdj: sdj1
Aug 14 13:52:06 seer kernel: SCSI device sdk: 421476864 512-byte hdwr sectors (215796 MB)
Aug 14 13:52:06 seer kernel: sdk: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdk: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdk: sdk1
Aug 14 13:52:06 seer kernel: SCSI device sdl: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdl: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdl: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdl: sdl1
Aug 14 13:52:06 seer kernel: SCSI device sdm: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdm: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdm: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdm: sdm1
Aug 14 13:52:06 seer kernel: SCSI device sdn: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdn: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdn: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdn: sdn1
Aug 14 13:52:06 seer kernel: SCSI device sdo: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdo: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdo: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdo: sdo1
Aug 14 13:52:06 seer kernel: SCSI device sdp: 2107390464 512-byte hdwr sectors (1078984 MB)
Aug 14 13:52:06 seer kernel: sdp: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdp: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdp: sdp1
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:06 seer kernel: sdq : READ CAPACITY failed.
Aug 14 13:52:06 seer kernel: sdq : status=1, message=00, host=0, driver=08
Aug 14 13:52:06 seer kernel: sd: Current: sense key: Illegal Request
Aug 14 13:52:06 seer kernel: Add. Sense: Logical unit not supported
Aug 14 13:52:06 seer kernel:
Aug 14 13:52:06 seer kernel: sdq: test WP failed, assume Write Enabled
Aug 14 13:52:06 seer kernel: sdq: asking for cache data failed
Aug 14 13:52:06 seer kernel: sdq: assuming drive cache: write through
Aug 14 13:52:06 seer kernel: sdq:end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:06 seer last message repeated 5 times
Aug 14 13:52:06 seer kernel: Dev sdq: unable to read RDB block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:06 seer kernel: unable to read partition table
Aug 14 13:52:06 seer kernel: SCSI device sdr: 21502464 512-byte hdwr sectors (11009 MB)
Aug 14 13:52:06 seer kernel: sdr: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdr: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdr: sdr1
Aug 14 13:52:06 seer kernel: SCSI device sds: 421476864 512-byte hdwr sectors (215796 MB)
Aug 14 13:52:06 seer kernel: sds: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sds: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sds: sds1
Aug 14 13:52:06 seer kernel: SCSI device sdt: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdt: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdt: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdt: sdt1
Aug 14 13:52:06 seer kernel: SCSI device sdu: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdu: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdu: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdu: sdu1
Aug 14 13:52:06 seer kernel: SCSI device sdv: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdv: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdv: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdv: sdv1
Aug 14 13:52:06 seer kernel: SCSI device sdw: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdw: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdw: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdw: sdw1
Aug 14 13:52:06 seer kernel: SCSI device sdx: 2107390464 512-byte hdwr sectors (1078984 MB)
Aug 14 13:52:06 seer kernel: sdx: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdx: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdx: sdx1
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdy, sector 0
Aug 14 13:52:06 seer kernel: sdy : READ CAPACITY failed.
Aug 14 13:52:06 seer kernel: sdy : status=1, message=00, host=0, driver=08
Aug 14 13:52:06 seer kernel: sd: Current: sense key: Illegal Request
Aug 14 13:52:06 seer kernel: Add. Sense: Logical unit not supported
Aug 14 13:52:06 seer kernel:
Aug 14 13:52:06 seer kernel: sdy: test WP failed, assume Write Enabled
Aug 14 13:52:06 seer kernel: sdy: asking for cache data failed
Aug 14 13:52:06 seer kernel: sdy: assuming drive cache: write through
Aug 14 13:52:06 seer kernel: sdy:end_request: I/O error, dev sdy, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdy, sector 0
Aug 14 13:52:06 seer last message repeated 5 times
Aug 14 13:52:06 seer kernel: Dev sdy: unable to read RDB block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdy, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdy, sector 0
Aug 14 13:52:06 seer kernel: unable to read partition table
Aug 14 13:52:06 seer kernel: SCSI device sdz: 21502464 512-byte hdwr sectors (11009 MB)
Aug 14 13:52:06 seer kernel: sdz: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdz: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdz: sdz1
Aug 14 13:52:06 seer kernel: SCSI device sdaa: 421476864 512-byte hdwr sectors (215796 MB)
Aug 14 13:52:06 seer kernel: sdaa: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdaa: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdaa: sdaa1
Aug 14 13:52:06 seer kernel: SCSI device sdab: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdab: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdab: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdab: sdab1
Aug 14 13:52:06 seer kernel: SCSI device sdac: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdac: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdac: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdac: sdac1
Aug 14 13:52:06 seer kernel: SCSI device sdad: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdad: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdad: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdad: sdad1
Aug 14 13:52:06 seer kernel: SCSI device sdae: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdae: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdae: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdae: sdae1
Aug 14 13:52:06 seer kernel: SCSI device sdaf: 2107390464 512-byte hdwr sectors (1078984 MB)
Aug 14 13:52:06 seer kernel: sdaf: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdaf: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdaf: sdaf1
Aug 14 13:52:06 seer kernel: scsi_wr_disk: unknown partition table
Aug 14 13:52:07 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:07 seer kernel: end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:07 seer kernel: end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:07 seer kernel: end_request: I/O error, dev sdy, sector 0
Here's some extra info:
# /sbin/blkid | grep asm
/dev/sdc1: LABEL="ORCL1" TYPE="oracleasm"
/dev/sdk1: LABEL="ORCL1" TYPE="oracleasm"
/dev/sds1: LABEL="ORCL1" TYPE="oracleasm"
/dev/sdaa1: LABEL="ORCL1" TYPE="oracleasm"
/dev/dpdb1: LABEL="ORCL1" TYPE="oracleasm"
I have learned that by excluding devices in the oracleasm configuration file I eliminate those I/O errors in /var/log/messages
# cat /etc/sysconfig/oracleasm
# This is a configuration file for automatic loading of the Oracle
# Automatic Storage Management library kernel driver. It is generated
# By running /etc/init.d/oracleasm configure. Please use that method
# to modify this file
# ORACLEASM_ENABELED: 'true' means to load the driver on boot.
ORACLEASM_ENABLED=true
# ORACLEASM_UID: Default user owning the /dev/oracleasm mount point.
ORACLEASM_UID=oracle
# ORACLEASM_GID: Default group owning the /dev/oracleasm mount point.
ORACLEASM_GID=oinstall
# ORACLEASM_SCANBOOT: 'true' means scan for ASM disks on boot.
ORACLEASM_SCANBOOT=true
# ORACLEASM_SCANORDER: Matching patterns to order disk scanning
ORACLEASM_SCANORDER="dp sd"
# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan
ORACLEASM_SCANEXCLUDE="sdc sdk sds sdaa sda"
# ls -la /dev/oracleasm/disks/
total 0
drwxr-xr-x 1 root root 0 Aug 14 10:47 .
drwxr-xr-x 4 root root 0 Aug 13 15:32 ..
brw-rw---- 1 oracle oinstall 251, 33 Aug 14 13:46 ORCL1
Now I can go into dbca to create the ASM instance, which starts up fine... create a new diskgroup, I see ORCL1 as a provision ASM disk I select it ... Click OK
CRASH!!! Box hangs have to reboot it....
I have gotten myself to exactly the same point right before clicking OK and here is what is in the ASM alertlog so far
Fri Aug 14 14:42:02 2009
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as /u01/app/oracle/product/11.1.0/db_1/dbs/arch
Autotune of undo retention is turned on.
IMODE=BR
ILAT =0
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up ORACLE RDBMS Version: 11.1.0.6.0.
Using parameter settings in server-side spfile /u01/app/oracle/product/11.1.0/db_1/dbs/spfile+ASM.ora
System parameters with non-default values:
large_pool_size = 12M
instance_type = "asm"
diagnostic_dest = "/u01/app/oracle"
Fri Aug 14 14:42:04 2009
PMON started with pid=2, OS id=3300
Fri Aug 14 14:42:04 2009
VKTM started with pid=3, OS id=3302 at elevated priority
VKTM running at (20)ms precision
Fri Aug 14 14:42:04 2009
DIAG started with pid=4, OS id=3306
Fri Aug 14 14:42:04 2009
PSP0 started with pid=5, OS id=3308
Fri Aug 14 14:42:04 2009
DSKM started with pid=6, OS id=3310
Fri Aug 14 14:42:04 2009
DIA0 started with pid=7, OS id=3312
Fri Aug 14 14:42:04 2009
MMAN started with pid=8, OS id=3314
Fri Aug 14 14:42:04 2009
DBW0 started with pid=9, OS id=3316
Fri Aug 14 14:42:04 2009
LGWR started with pid=6, OS id=3318
Fri Aug 14 14:42:04 2009
CKPT started with pid=10, OS id=3320
Fri Aug 14 14:42:04 2009
SMON started with pid=11, OS id=3322
Fri Aug 14 14:42:04 2009
RBAL started with pid=12, OS id=3324
Fri Aug 14 14:42:04 2009
GMON started with pid=13, OS id=3326
ORACLE_BASE from environment = /u01/app/oracle
Fri Aug 14 14:42:04 2009
SQL> ALTER DISKGROUP ALL MOUNT
Fri Aug 14 14:42:41 2009
At this point I don't want to click the OK until I am sure someone is in the office to reboot the machine manually if I do hang it again.... I hung it twice yesterday, however I did not have the devices excluded in the oracleasm configuration file as i do now
Edited by: user10193377 on Aug 14, 2009 3:23 PM
Well Clicking OK hun it again and I am waiting to get back into it, to see what new information might be gleened
Does anyone have any ideas on what to check or where to look????? Will update more once I can log back in

Hi Mark,
It looks like something is not correct with your raw device partition based on the error messages:
Aug 14 13:52:06 seer kernel: Add. Sense: Logical unit not supported
Aug 14 13:52:06 seer kernel:
Aug 14 13:52:06 seer kernel: sda: test WP failed, assume Write Enabled
Aug 14 13:52:06 seer kernel: sda: asking for cache data failed
Aug 14 13:52:06 seer kernel: sda: assuming drive cache: write through
Aug 14 13:52:06 seer kernel: sda:end_request: I/O error, dev sda, sector 0
It could be a number of things. I would check with your vendor and Oracle support to see if the multipath software drive is supported and if there is a potential workaround for ASM. Sorry this is not quite the solution, but its what jumps to mind based on issues with multipath software and storage vendors for ASM with Linux and Oracle. Have you checked the validation matrix available on Metalink?
Cheers,
Ben

Move disk devices between diskgroup in ASM SAN

Hi Experts,
We have a 4 nodes oracle 11.1 RAC with 10.2.04 database at red hat 5.1 Linux
there are two diskgroup to support database. data1 and data2.
data1 is for data files as 500G and data2 is for FRA as 1.7T.
in the back, 4 candidate disks to support data1 group and other 8 support data2 group
At the present, manager request to one devices to data 1 from data2.
How can we user EM ( or other way) to move one device to data1 from data2 diskgroup without damage database.
Please help me. this is emergency resuest due to data1 diskgroup without space and SA can not add a space in there right now.
Thanks very much
Jim

No, you won't loose any data assuming you have enough space on other disks to move the data from the disk that is being dropped. Once run the drop command, asm automatically start rebalance process and it will start moving data from the disk that is being dropped to other available disks. I would give few mins to ASM to finish the rebalance process. You can check the reblance process by running the follwonig query. If the query results to nothing or zero records that means ASM is not doing any rebalances.
select * from v$asm_operation
you can also check the following query to check the disks
desc v$asm_disk
select * from asm_disk (or pick the fields you want to view such as name, status, header_status, mount_status, etc)

Placing Vote Disk on diferents diskgroups - 11gR2

Similar Messages

Maybe you are looking for