Add a disk to RAC

Hello,
I have a RAC database with 2 nodes and a cluster.
In my Cluster I have 3 disks with RAID5. I need to add 2 disks RAID0 where I will put my redolog files because I have a big waiting time in "log file parallel write"
I' m new in RAC and cluster can I add disksto a cluster with different RAID?

You could mix any kind of RAID (of course not recommended). Indeed, redologs with their high write activity should not be put on RAID5.
Werner

Similar Messages

Add ASM disk in RAC

Hello everyone,
We are on 10gR2 ASM.
The system admin has provided one disk as follows:
On node-1 (tphsmsd1)
/dev/rdisk/disk83 CANDIDATE
On node-2 (tphsmsd2)
/dev/rdisk/disk77 CANDIDATE
I've read that paths of ASM disks in RAC may be different.
When I give "ls -l /dev/rdisk/disk83" gives no such files and vice-versa for disk on node2.
I don't understand how ASM handles this ? If I add disk (disk83) on node 1 how it's going to get to that disk on node 2 ?
thanks
Jitu Keshwani

Jitu Keshwani wrote:
I've read that paths of ASM disks in RAC may be different.
When I give "ls -l /dev/rdisk/disk83" gives no such files and vice-versa for disk on node2.
I don't understand how ASM handles this ? If I add disk (disk83) on node 1 how it's going to get to that disk on node 2 ?The header label of the disk, identifies the ASM disk name and diskgroup it belongs to. You can hexdump or octaldump the 1st 128 bytes of the device as ascii chars to view the label. The string "+ORCLDISK+" in the header identifies it as an ASM disk. This is then followed by the ASM diskname and then the AM diskgroup it belongs to.
But there's no reason for not having a consistent and static device name layer across all cluster nodes. Each scsi device has a WWID (World Wide ID)- a unique identifier. This means that the kernel (and kernel drivers) can uniquely identify a device.
On Linux, Multipath is used to map a logical device name to a WWID - and using the same +/etc/multipath.conf+ configuration file on all cluster nodes, ensures that the same device names are used across the cluster.
But from the the device file entry you listed, you're likely not running Linux? In that case, depending on the Unix flavour used and the type of cluster storage, there can be similar options to Linux's Multipath. One such option is EMC's PowerPath - but that of course is specific to EMC SANs and requires additional licensing fees.

Want to move datafiles, controlfiles, redolog on new ASM Disks (11gR2 RAC)

Hi Guys,
Setup: Two Node 11gR2 (11.2.0.1) RAC on RHEL 5.4
Existing disks are from Old SAN & New Disks are from New SAN.
Can I move all datafiles (+DATA), controlfiles (+CTRL), redolog (+REDO) on new ASM Disks by adding disks in is same Diskgroup & dropping older disks from existing Diskgroup taking advantage of ASM Re-balancing Feature.
1) add required disks in the DATA Diskgroups,
ALTER DISKGROUP DATA ADD DISK
'/dev/oracleasm/disks/NEWDATA3' NAME NEWDATA_0003,
'/dev/oracleasm/disks/NEWDATA4' NAME NEWDATA_0004,
'/dev/oracleasm/disks/NEWDATA5' NAME NEWDATA_0005
REBALANCE POWER 11;
Check rebalance status from v$ASM_OPERATION.
2) When rebalance completes, drop the old disks.
ALTER DISKGROUP DATA DROP DISK
NEWDATA_0000,
NEWDATA_0001
REBALANCE POWER 11;
Check rebalance status from v$ASM_OPERATION.
3) Do it same for Redo log groups & Controlfile Diskgroups.
I hope, I could do this Activity, even if database is Up. is there possibility of Database block Corruption ??? (or is it necessary to perform above steps when database is down)
Would be appreciated, your quick responses on the same.
It's an urgent requirement. Thanks.
Regards,
Manish

Manish Nashikkar wrote:
Hi Guys,
Setup: Two Node 11gR2 (11.2.0.1) RAC on RHEL 5.4
Existing disks are from Old SAN & New Disks are from New SAN.
Can I move all datafiles (+DATA), controlfiles (+CTRL), redolog (+REDO) on new ASM Disks by adding disks in is same Diskgroup & dropping older disks from existing Diskgroup taking advantage of ASM Re-balancing Feature.
1) add required disks in the DATA Diskgroups,
ALTER DISKGROUP DATA ADD DISK
'/dev/oracleasm/disks/NEWDATA3' NAME NEWDATA_0003,
'/dev/oracleasm/disks/NEWDATA4' NAME NEWDATA_0004,
'/dev/oracleasm/disks/NEWDATA5' NAME NEWDATA_0005
REBALANCE POWER 11;
Check rebalance status from v$ASM_OPERATION.
2) When rebalance completes, drop the old disks.
ALTER DISKGROUP DATA DROP DISK
NEWDATA_0000,
NEWDATA_0001
REBALANCE POWER 11;
Check rebalance status from v$ASM_OPERATION.
3) Do it same for Redo log groups & Controlfile Diskgroups.
I hope, I could do this Activity, even if database is Up. is there possibility of Database block Corruption ??? (or is it necessary to perform above steps when database is down)
Would be appreciated, your quick responses on the same.
It's an urgent requirement. Thanks.
Regards,
Manish
Hi Manish,
Yes you can do that by adding new disk to existing diskgroup and delete old diskgroup. The good thing is this can be done online however you need to make sure the rebalance power is meet your business time, higher rebalance power is faster to rebalance to complete however it also will consume more resources
Cheers

Add new disk(space) into exist ASM

Hi Experts,
We have a oracle 11.1 4 nodes RAC at red hat 5 kinux. we database build as 10.2.04 at ASM with two disk group.
the original DBA put all space into two diskgroup data+ 300G and FSDISK 1500G.
Now my boss ask to create second database in this server. But I found there are no canada disk is availuable to build two new diskgroup to support new second database.
Existing database is running over 2 years. How can we release 500G diskspace as canada disk for ASM from FSDISK group?
or
How can we add new disk into ASM?
We do not have special storage person in small sys/dba team.
I am looking for help!!
JIm

Hi nvengurl ,
It is nice to get your advice. some sr.DBA told me as" Two instances in the same database can share the disk-group"
other DBA told me as
Two different DATABASES cannot (remember, RAC uses a database with multiple instances). So technically you can share a diskgroup between instances, as long as they're in the same database."
some DBA said as Two separate instances cannot share a diskgroup"
I am not sure. please oracle expert and oracle company staff to confirm this point.
If I create two database with exist diskgroup. any performance or maintenance issue
I am looking for your help!!!
JIm

OVM disks for RAC implementation

Dear All
is there any guide available on how can you create the disks for RAC ASM in OVM 3.3.1 using a fiber channel block level storage?
Thanks
George

You are right, you can't use virtual disks for RAC configuration. Have a look here (especially page 18):
http://www.oracle.com/technetwork/products/clustering/oracle-rac-in-oracle-vm-environment-131948.pdf
Using physical disks means that you create LUNs on your storage array connected to Oracle VM Servers by fabric channel. You map these LUNs to all servers in the pool or all standalone servers where you are going to install your virtual machines being Clusterware nodes. Then you rediscover storage in Oracle VM Manager, mark these LUNs as "shared" in OVMM and add them to your virtual machines as "Physical disks" (by editing guest properties in OVMM).
Alternatively you can directly map iSCSI or NFS storage to your guests. By "directly" I mean you use IP addresses and software in your guests as iSCSI initiator or NFS client - without engaging Oracle VM in the middle.
Regards,
Michal

Add shared disks without shutdown vm

Hi all,
we have a Pc with VMWare 2 and two VMS with
Oracle 10g on RAC and CentOS 4.7 . There are two nodes (or vms): rac1 and rac2.
Each VM has shared SCSI disks.
Is there any way to add shared disks without shutdown the VMs?
thank you.

Well,
Look the link:
http://www.vmware.com/support/ws5/doc/ws_disk_add_virtual.html
You must shutdown the instance.
but in your case, I dont know, maybe shutdown one VM, add the disk, and restart.
So, shutdown other vm, add the same disk, and restart. After, you can add the disk to disk group
if you are using ASM.
Please, correct me if I'm wrong.
Cheers

How to add more disk space into /   root file system

Hi All,
Linux 2.6.18-128
can anyone please let us know how to add more disk space into "/" root file system.
i have added new hard disk with space of 20GB,
[root@rac2 shm]# df -h
Filesystem            Size Used Avail Use% Mounted on
/dev/hda1             965M 767M 149M 84% /
/dev/hda7             1.9G 234M 1.6G 13% /var
/dev/hda6             2.9G   69M 2.7G   3% /tmp
/dev/hda3             7.6G 4.2G 3.0G 59% /usr
/dev/hda2              18G   12G 4.8G 71% /u01
LABLE=/               2.0G     0 2.0G   0% /dev/shm
/dev/hdb2             8.9G 149M 8.3G   2% /vm
[root@rac2 shm]#

Dude! wrote:
I would actually question whether or not more disks increase the risk of a disk failure. One disk can break as likely as one of two of more disks.
Simple stats. Buying 2 lottery tickets instead of one, gives you 2 chances to win the lottery prize. Not 1. Even though the odds of winning per ticket remains unchanged.
2 disks buy you 2 tickets in The-Drive-Failure lottery.
Back in the 90's, BT (British Telecom) had a 80+ node OPS cluster build with Pyramid MPP hardware. They had a dedicated store of scsi disks for replacing failed disks - as there were disk failure fairly often due to the number of disks. (a Pryamid MPP chassis looked like a Xmas tree with all the scsi drive LEDs, and BT had several)
In my experience - one should rather expect a drive failure sooner, than later. And have some kind of contingency plan in place to recover from the failure.
The use of symbolic links instead of striping the filesystem protects from the complete loss of the enchilada if a volume member fails, but it does not reduce the risk of loosing data.
I would rather buy a single ticket for the drive failure lottery for a root drive, than 2 tickets in this case. And using symbolic links to "offload" non-critical files to the 2nd drive means that its lottery ticket prize is not a non-bootable server due to a toasted root drive.

How to add additional disks on vmware OEL4 and use it for Oracle 10gR2?

I created a virtual machine on vmware workstation 6 and installed OEL4.
during first install I created 20 GB disk but now I want to add more disks.
from vmware documentation I tried to add more 8 gb disk to the virtual host.
under devices I see two lines;
Hard Disk (SCSI 0:0) 20.0 GB
Hard Disk (SCSI 0:2) 8.0 GB
but I must be missing some step since I can not see 20 + 8 gb at df;
[root@antuhost ~]# df -h
Filesystem            Size Used Avail Use% Mounted on
/dev/sda1              13G 9.7G 2.3G 82% /
none                  506M     0 506M   0% /dev/shm
/dev/sda2             4.9G 851M 3.8G 19% /homeThank you.

Oh the check the answer from Re: How to add additional disks on vmware OEL4 and use it for Oracle 10gR2?

I am trying to add a hard disk but keep getting an error message to add null disk

I am trying to add a hard disk but keep getting an error message to add null disk. How do I fix this problem?

When installing a new OSX, it is not unusual that some applications will not work. You have to check with the developer to see if they have upgraded to the new OSX or find a substitute application.
In regards to your new HDD, most of the time they come formatted for Windows, not Mac. Connect the HDD to your MBP and open Disk Utility>Erase. To make it compatible, select 'Mac OS Extended (Journaled)' as the format. Click on the Erase button. You then should be able to read and write to it.
Ciao.

How Prepare Disks for RAC installation

How configure disks for RAC instalation for Windows Enterprise Server 2003

Best resource for RAC on Windows:
www.jobcestbon.com/oracle/RacOnWindows.pdf
HTH
Thanks
-Chandra PabbaI read it, but i unable to configure Shared Memory accourding to this link. Please guide me how i can configure shared Memory for RAC in Windows Server Enterprise 2003.
Thanks

Shared Disks For RAC

Hi,
I plan to use shared disks to create Oracle RAC using ASM. What options do I have? OCFS2? or any other option?
Can some one lead me to a documnet on how can I use the shared disks for RAC?
Thanks.

javed555 wrote:
I plan to use shared disks to create Oracle RAC using ASM. What options do I have? You have two options:
1. Create shared virtual, i.e. file-backed disks. These files will be stored in /OVS/sharedDisk/ and made available to each guest
2. Expose physical devices directly to each guest, e.g. an LVM partition or a multipath LUN.
With both options, the disks show up as devices in the guests and you would then provision them with ASM, exactly the same way as if your RAC nodes were physical.
OCFS2 or NFS are required to create shared storage for Oracle VM Servers. This is to ensure the /OVS mount point is shared between multiple Oracle VM Servers.

Need to add Hard Disk for backup purposes

Hi all,
I'm new here and really new to Netware. I'm just beginning to get involved with Netware Servers (V 6.5) and am trying to add hard disks to a server for backup purposes. I have two SCSI hard disks to add to this server and wish to name them backup_1 and backup_2. Using the Backup Exec software we have a site license. I want to be able to do an incremental backup Monday thru Thursday and a full back up on Friday on each disk alternating out the disks weekly with one of the disks being off site while the other is in service.
I can add the new drive in a Hot manner and then must go through configuring the disk.....during the configuration I'm asked if I want the disk to be a part of a pool or something to that effect. What do I want to do here?
Actually I'd like to know just what is a pool technically? I've got several books on Netware and none of them has painted me a clear picture of just what a pool is.
I'm kind of excited to be learning about Netware I hope you all can bear with me as I stumble through this.
Thanks so much!!!

A pool is a structure between a volume and a partition. It allows, for instance, you to have multiple volumes in a pool, each of which can dynamically grow to use unallocated space within the pool on demand.
Novell Documentation
In your situation they are of no extra use to you, what you do is create a partition on each of those new disks, and create a volume on each and let it (I assume you are using NSSMU on the console) automatically create an underlying pool for each of those volumes.

How to add ASM Disk to Disk Group that is shown as MEMBER?

HI,
We have a Production Oracle RAC on HP-UX. We have Two Disk Groups one for Arcive (ARC_DISK) and other for Database(DATA_DISK).
Today I wanted to add another 200 GB of Disk Space to the DATA_DISK Group.
I opened DBCA and could not find the new disk in the 'Show Candidates' option. But after the Unix admin changed the ownership and permission of the Disk, it was shown in 'Show Candidates' option. I selected this Disk and continued but got an error later as we didnt change the ownership and permission from the 2nd Unix server. After doing this, , when I open DBCA to add disk, nothing is shown in 'Show Candidates' option but when we click on 'Show All', the new disk is listed as MEMBER (header status) but not allocated to any Disk Group.
I would like to know how to allocate this Disk (which is already shown as MEMBER) to the DATA_DISK - ASM diskgroup. This is a production database system.
Rgds,
Thiru

user1983888 wrote:
Hi,
Due to huge difference in Disk Size which I am trying to add to the existing Disk Group which has 1024 GB of Disk, would it be a better idea to create a new Disk Group with 200g GB Disk? Will this command work as the header_status is already MEMBER?
SQL>CREATE DISKGROUP DATA_DISK1
EXTERNAL REDUNDANCY
DISK 'disk path';
Thanks.Yes, you can create new diskgroup (DATA_DISK1). Just make sure all datafile in DATA_DISK autoextend off, then create all your new datafile to the new diskgroup DATA_DISK1. The command will not work as the header_status is MEMBER. You'll receive ORA-15018 & ORA-15033 error. As I mentioned before, you need to use FORCE option or format the disk using dd* command before perform above command.
1st Option : (Format the raw disk using dd command creating new diskgroup. Header_status MEMBER -> CANDIDATE). E.g.
1. dd if=/dev/zero of=/dev/rdsk/c2t1d0 bs=8192 count=1000 //any rac nodes
2. # chown oracle:oinstall /dev/rdsk/c2t1d0 //perform on both node. This apply if using LVM or direct access to raw
3. SQL> create diskgroup data_disk1 external redundancy disk '/dev/rdsk/c2t1d0'; //any RAC node in ASM instance
4. SQL> alter diskgroup data_disk1 mount; //try to mount on the other node
2nd Option : (Using FORCE option when creating the new diskgroup. Header_Status = MEMBER)
1. # chown oracle:oinstall /dev/rdsk/c2t1d0 //perform on both node. This apply if using LVM or direct access to raw
2. SQL> create diskgroup data_disk1 external redundancy disk '/dev/rdsk/c2t1d0' force; //any RAC node in ASM instance
3. SQL> alter diskgroup data_disk1 mount; //try to mount on the other node
Just be careful when using FORCE option or DD command. If you perform this operation on existing ASM disk groups disk, it may destroy existing disk groups.

Doubts about shared disk for RAC

Hi All,
I am really new to RAC.Even after reading various documents,I still have many doubts regarding shared storage and file systems needed for RAC.
1.Clusterware has to be installed on a shared file system like OCFS2.Which type of hard drive is required to install OCFS2 so that it can be accessed from all nodes??
It has to be an external hard drive???Or we can use any simple hard disk for shared storage??
If we use external hard drive then does it need to be connected to a seperate server alltogether or can it be connected to any one of the nodes in the cluster???
Apart from this shared drives,approximately what size of hard disk is required for all nodes(for just a testing environment).
Sincerely appreciate a reply!!
Thanks in advance.

Clusterware has to be installed on shared storage. RAC also requires shared storage for the database.
Shared storage can be managed via many methods.
1. Some sites using Linux or UNIX-based OSes choose to use RAW disk devices. This method is not frequently used due to the unpleasant management overhead and long-term manageability for RAW devices.
2. Many sites use cluster filesystems. On Linux and Windows, Oracle offers OCFS2 as one (free) cluster filesystem. Other vendors also offer add-on products for some OSes that provide supported cluster filesystems (like GFS, GPFS, VxFS, and others). Supported cluster filesystems may be used for Clusterware files (OCR and voting disks) as well as database files. Check Metalink for a list of supported cluster filesystems.
3. ASM can be used to manage shared storage used for database files. Unfortunately, due to architecture decisions made by Oracle, ASM cannot currently be used for Clusterware files (OCR and voting disks). It is relatively common to see ASM used for DB files and either RAW or a cluster filesystem used for Clusterware files. In other words, ASM and cluster filesystems and RAW are not mutually exclusive.
As for hardware--I have not seen any hardware capable of easily connecting multiple servers to internal storage. So, shared storage is always (in my experience) housed externally. You can find some articles on OTN and other sites (search Google for them) that use firewire drives or a third computer running openfiler to provide the shared storage in test environments. In production environments, SAN devices are commonly employed to provide concurrent access to storage from multiple servers.
Hope this helps!
Message was edited by:
Dan_Norris

Please Help - When I try to add ASM Disk to ASM Diskgroup it crashes Server

We are using a Pillar SAN and have LUNS Created and are using the following multipath device: (I'm a DBA more then anything else... but I am rather familiar with linux .... SAN Hardware not so much)
Device Size Mount Point
/dev/dpda1 11G /u01
The Above device is working fine... Below are the ASM Disks being Created
Device Size Oracle ASM Disk Name
/dev/dpdb1 198G ORCL1
/dev/dpdc1 21G SIRE1
/dev/dpdd1 21G CART1
/dev/dpde1 21G SRTS1
/dev/dpdf1 21G CRTT1
I try to create to the first ASM Disk
/etc/init.d/oracleasm createdisk ORCL1 /dev/dpdb1
Marking disk "ORCL1" as an ASM disk: [FAILED]
So I check the oracleasm log:
#cat /var/log/oracleasm
Device "/dev/dpdb1" is not a partition
I did some research and found that this is a common problem with multipath devices and to work around it you have to use asmtool
# /usr/sbin/asmtool -C -l /dev/oracleasm -n ORCL1 -s /dev/dpdb1 -a force=yes
asmtool: Device "/dev/dpdb1" is not a partition
asmtool: Continuing anyway
now I scan and list the disks
# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks: [ OK ]
# /etc/init.d/oracleasm listdisks
ORCL1
Here is whats going on in /var/log/messages when I run the oracleasm scandisks command
# date
Fri Aug 14 13:51:58 MST 2009
# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks: [ OK ]
cat /var/log/messages | grep "Aug 14 13:5"
Aug 14 13:52:06 seer kernel: dpdb: dpdb1
Aug 14 13:52:06 seer kernel: dpdc: dpdc1
Aug 14 13:52:06 seer kernel: dpdd: dpdd1
Aug 14 13:52:06 seer kernel: dpde: dpde1
Aug 14 13:52:06 seer kernel: dpdf: dpdf1
Aug 14 13:52:06 seer kernel: dpdg: dpdg1
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: printk: 30 messages suppressed.
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: sda : READ CAPACITY failed.
Aug 14 13:52:06 seer kernel: sda : status=1, message=00, host=0, driver=08
Aug 14 13:52:06 seer kernel: sd: Current: sense key: Illegal Request
Aug 14 13:52:06 seer kernel: Add. Sense: Logical unit not supported
Aug 14 13:52:06 seer kernel:
Aug 14 13:52:06 seer kernel: sda: test WP failed, assume Write Enabled
Aug 14 13:52:06 seer kernel: sda: asking for cache data failed
Aug 14 13:52:06 seer kernel: sda: assuming drive cache: write through
Aug 14 13:52:06 seer kernel: sda:end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: Dev sda: unable to read RDB block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sda, logical block 0
Aug 14 13:52:06 seer kernel: unable to read partition table
Aug 14 13:52:06 seer kernel: SCSI device sdb: 21502464 512-byte hdwr sectors (11009 MB)
Aug 14 13:52:06 seer kernel: sdb: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdb: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdb: sdb1
Aug 14 13:52:06 seer kernel: SCSI device sdc: 421476864 512-byte hdwr sectors (215796 MB)
Aug 14 13:52:06 seer kernel: sdc: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdc: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdc: sdc1
Aug 14 13:52:06 seer kernel: SCSI device sdd: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdd: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdd: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdd: sdd1
Aug 14 13:52:06 seer kernel: SCSI device sde: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sde: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sde: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sde: sde1
Aug 14 13:52:06 seer kernel: SCSI device sdf: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdf: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdf: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdf: sdf1
Aug 14 13:52:06 seer kernel: SCSI device sdg: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdg: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdg: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdg: sdg1
Aug 14 13:52:06 seer kernel: SCSI device sdh: 2107390464 512-byte hdwr sectors (1078984 MB)
Aug 14 13:52:06 seer kernel: sdh: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdh: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdh: sdh1
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:06 seer kernel: Buffer I/O error on device sdi, logical block 0
Aug 14 13:52:06 seer kernel: sdi : READ CAPACITY failed.
Aug 14 13:52:06 seer kernel: sdi : status=1, message=00, host=0, driver=08
Aug 14 13:52:06 seer kernel: sd: Current: sense key: Illegal Request
Aug 14 13:52:06 seer kernel: Add. Sense: Logical unit not supported
Aug 14 13:52:06 seer kernel:
Aug 14 13:52:06 seer kernel: sdi: test WP failed, assume Write Enabled
Aug 14 13:52:06 seer kernel: sdi: asking for cache data failed
Aug 14 13:52:06 seer kernel: sdi: assuming drive cache: write through
Aug 14 13:52:06 seer kernel: sdi:end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:06 seer last message repeated 4 times
Aug 14 13:52:06 seer kernel: Dev sdi: unable to read RDB block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:06 seer kernel: unable to read partition table
Aug 14 13:52:06 seer kernel: SCSI device sdj: 21502464 512-byte hdwr sectors (11009 MB)
Aug 14 13:52:06 seer kernel: sdj: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdj: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdj: sdj1
Aug 14 13:52:06 seer kernel: SCSI device sdk: 421476864 512-byte hdwr sectors (215796 MB)
Aug 14 13:52:06 seer kernel: sdk: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdk: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdk: sdk1
Aug 14 13:52:06 seer kernel: SCSI device sdl: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdl: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdl: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdl: sdl1
Aug 14 13:52:06 seer kernel: SCSI device sdm: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdm: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdm: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdm: sdm1
Aug 14 13:52:06 seer kernel: SCSI device sdn: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdn: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdn: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdn: sdn1
Aug 14 13:52:06 seer kernel: SCSI device sdo: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdo: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdo: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdo: sdo1
Aug 14 13:52:06 seer kernel: SCSI device sdp: 2107390464 512-byte hdwr sectors (1078984 MB)
Aug 14 13:52:06 seer kernel: sdp: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdp: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdp: sdp1
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:06 seer kernel: sdq : READ CAPACITY failed.
Aug 14 13:52:06 seer kernel: sdq : status=1, message=00, host=0, driver=08
Aug 14 13:52:06 seer kernel: sd: Current: sense key: Illegal Request
Aug 14 13:52:06 seer kernel: Add. Sense: Logical unit not supported
Aug 14 13:52:06 seer kernel:
Aug 14 13:52:06 seer kernel: sdq: test WP failed, assume Write Enabled
Aug 14 13:52:06 seer kernel: sdq: asking for cache data failed
Aug 14 13:52:06 seer kernel: sdq: assuming drive cache: write through
Aug 14 13:52:06 seer kernel: sdq:end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:06 seer last message repeated 5 times
Aug 14 13:52:06 seer kernel: Dev sdq: unable to read RDB block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:06 seer kernel: unable to read partition table
Aug 14 13:52:06 seer kernel: SCSI device sdr: 21502464 512-byte hdwr sectors (11009 MB)
Aug 14 13:52:06 seer kernel: sdr: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdr: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdr: sdr1
Aug 14 13:52:06 seer kernel: SCSI device sds: 421476864 512-byte hdwr sectors (215796 MB)
Aug 14 13:52:06 seer kernel: sds: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sds: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sds: sds1
Aug 14 13:52:06 seer kernel: SCSI device sdt: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdt: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdt: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdt: sdt1
Aug 14 13:52:06 seer kernel: SCSI device sdu: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdu: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdu: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdu: sdu1
Aug 14 13:52:06 seer kernel: SCSI device sdv: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdv: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdv: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdv: sdv1
Aug 14 13:52:06 seer kernel: SCSI device sdw: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdw: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdw: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdw: sdw1
Aug 14 13:52:06 seer kernel: SCSI device sdx: 2107390464 512-byte hdwr sectors (1078984 MB)
Aug 14 13:52:06 seer kernel: sdx: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdx: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdx: sdx1
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdy, sector 0
Aug 14 13:52:06 seer kernel: sdy : READ CAPACITY failed.
Aug 14 13:52:06 seer kernel: sdy : status=1, message=00, host=0, driver=08
Aug 14 13:52:06 seer kernel: sd: Current: sense key: Illegal Request
Aug 14 13:52:06 seer kernel: Add. Sense: Logical unit not supported
Aug 14 13:52:06 seer kernel:
Aug 14 13:52:06 seer kernel: sdy: test WP failed, assume Write Enabled
Aug 14 13:52:06 seer kernel: sdy: asking for cache data failed
Aug 14 13:52:06 seer kernel: sdy: assuming drive cache: write through
Aug 14 13:52:06 seer kernel: sdy:end_request: I/O error, dev sdy, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdy, sector 0
Aug 14 13:52:06 seer last message repeated 5 times
Aug 14 13:52:06 seer kernel: Dev sdy: unable to read RDB block 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdy, sector 0
Aug 14 13:52:06 seer kernel: end_request: I/O error, dev sdy, sector 0
Aug 14 13:52:06 seer kernel: unable to read partition table
Aug 14 13:52:06 seer kernel: SCSI device sdz: 21502464 512-byte hdwr sectors (11009 MB)
Aug 14 13:52:06 seer kernel: sdz: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdz: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdz: sdz1
Aug 14 13:52:06 seer kernel: SCSI device sdaa: 421476864 512-byte hdwr sectors (215796 MB)
Aug 14 13:52:06 seer kernel: sdaa: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdaa: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdaa: sdaa1
Aug 14 13:52:06 seer kernel: SCSI device sdab: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdab: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdab: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdab: sdab1
Aug 14 13:52:06 seer kernel: SCSI device sdac: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdac: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdac: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdac: sdac1
Aug 14 13:52:06 seer kernel: SCSI device sdad: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdad: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdad: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdad: sdad1
Aug 14 13:52:06 seer kernel: SCSI device sdae: 43006464 512-byte hdwr sectors (22019 MB)
Aug 14 13:52:06 seer kernel: sdae: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdae: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdae: sdae1
Aug 14 13:52:06 seer kernel: SCSI device sdaf: 2107390464 512-byte hdwr sectors (1078984 MB)
Aug 14 13:52:06 seer kernel: sdaf: Write Protect is off
Aug 14 13:52:06 seer kernel: SCSI device sdaf: drive cache: write through w/ FUA
Aug 14 13:52:06 seer kernel: sdaf: sdaf1
Aug 14 13:52:06 seer kernel: scsi_wr_disk: unknown partition table
Aug 14 13:52:07 seer kernel: end_request: I/O error, dev sda, sector 0
Aug 14 13:52:07 seer kernel: end_request: I/O error, dev sdi, sector 0
Aug 14 13:52:07 seer kernel: end_request: I/O error, dev sdq, sector 0
Aug 14 13:52:07 seer kernel: end_request: I/O error, dev sdy, sector 0
Here's some extra info:
# /sbin/blkid | grep asm
/dev/sdc1: LABEL="ORCL1" TYPE="oracleasm"
/dev/sdk1: LABEL="ORCL1" TYPE="oracleasm"
/dev/sds1: LABEL="ORCL1" TYPE="oracleasm"
/dev/sdaa1: LABEL="ORCL1" TYPE="oracleasm"
/dev/dpdb1: LABEL="ORCL1" TYPE="oracleasm"
I have learned that by excluding devices in the oracleasm configuration file I eliminate those I/O errors in /var/log/messages
# cat /etc/sysconfig/oracleasm
# This is a configuration file for automatic loading of the Oracle
# Automatic Storage Management library kernel driver. It is generated
# By running /etc/init.d/oracleasm configure. Please use that method
# to modify this file
# ORACLEASM_ENABELED: 'true' means to load the driver on boot.
ORACLEASM_ENABLED=true
# ORACLEASM_UID: Default user owning the /dev/oracleasm mount point.
ORACLEASM_UID=oracle
# ORACLEASM_GID: Default group owning the /dev/oracleasm mount point.
ORACLEASM_GID=oinstall
# ORACLEASM_SCANBOOT: 'true' means scan for ASM disks on boot.
ORACLEASM_SCANBOOT=true
# ORACLEASM_SCANORDER: Matching patterns to order disk scanning
ORACLEASM_SCANORDER="dp sd"
# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan
ORACLEASM_SCANEXCLUDE="sdc sdk sds sdaa sda"
# ls -la /dev/oracleasm/disks/
total 0
drwxr-xr-x 1 root root 0 Aug 14 10:47 .
drwxr-xr-x 4 root root 0 Aug 13 15:32 ..
brw-rw---- 1 oracle oinstall 251, 33 Aug 14 13:46 ORCL1
Now I can go into dbca to create the ASM instance, which starts up fine... create a new diskgroup, I see ORCL1 as a provision ASM disk I select it ... Click OK
CRASH!!! Box hangs have to reboot it....
I have gotten myself to exactly the same point right before clicking OK and here is what is in the ASM alertlog so far
Fri Aug 14 14:42:02 2009
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as /u01/app/oracle/product/11.1.0/db_1/dbs/arch
Autotune of undo retention is turned on.
IMODE=BR
ILAT =0
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up ORACLE RDBMS Version: 11.1.0.6.0.
Using parameter settings in server-side spfile /u01/app/oracle/product/11.1.0/db_1/dbs/spfile+ASM.ora
System parameters with non-default values:
large_pool_size = 12M
instance_type = "asm"
diagnostic_dest = "/u01/app/oracle"
Fri Aug 14 14:42:04 2009
PMON started with pid=2, OS id=3300
Fri Aug 14 14:42:04 2009
VKTM started with pid=3, OS id=3302 at elevated priority
VKTM running at (20)ms precision
Fri Aug 14 14:42:04 2009
DIAG started with pid=4, OS id=3306
Fri Aug 14 14:42:04 2009
PSP0 started with pid=5, OS id=3308
Fri Aug 14 14:42:04 2009
DSKM started with pid=6, OS id=3310
Fri Aug 14 14:42:04 2009
DIA0 started with pid=7, OS id=3312
Fri Aug 14 14:42:04 2009
MMAN started with pid=8, OS id=3314
Fri Aug 14 14:42:04 2009
DBW0 started with pid=9, OS id=3316
Fri Aug 14 14:42:04 2009
LGWR started with pid=6, OS id=3318
Fri Aug 14 14:42:04 2009
CKPT started with pid=10, OS id=3320
Fri Aug 14 14:42:04 2009
SMON started with pid=11, OS id=3322
Fri Aug 14 14:42:04 2009
RBAL started with pid=12, OS id=3324
Fri Aug 14 14:42:04 2009
GMON started with pid=13, OS id=3326
ORACLE_BASE from environment = /u01/app/oracle
Fri Aug 14 14:42:04 2009
SQL> ALTER DISKGROUP ALL MOUNT
Fri Aug 14 14:42:41 2009
At this point I don't want to click the OK until I am sure someone is in the office to reboot the machine manually if I do hang it again.... I hung it twice yesterday, however I did not have the devices excluded in the oracleasm configuration file as i do now
Edited by: user10193377 on Aug 14, 2009 3:23 PM
Well Clicking OK hun it again and I am waiting to get back into it, to see what new information might be gleened
Does anyone have any ideas on what to check or where to look????? Will update more once I can log back in

Hi Mark,
It looks like something is not correct with your raw device partition based on the error messages:
Aug 14 13:52:06 seer kernel: Add. Sense: Logical unit not supported
Aug 14 13:52:06 seer kernel:
Aug 14 13:52:06 seer kernel: sda: test WP failed, assume Write Enabled
Aug 14 13:52:06 seer kernel: sda: asking for cache data failed
Aug 14 13:52:06 seer kernel: sda: assuming drive cache: write through
Aug 14 13:52:06 seer kernel: sda:end_request: I/O error, dev sda, sector 0
It could be a number of things. I would check with your vendor and Oracle support to see if the multipath software drive is supported and if there is a potential workaround for ASM. Sorry this is not quite the solution, but its what jumps to mind based on issues with multipath software and storage vendors for ASM with Linux and Oracle. Have you checked the validation matrix available on Metalink?
Cheers,
Ben

Add a disk to RAC

Similar Messages

Maybe you are looking for