ASM on RAW or OCFS2

We have a 2-node RAC cluster using ASM that has a couple diskgroups (DATA and FRA) on RAW devices. With our current backup methodology, we use RMAN to backup simultaneously to FRA and /u02/backup (cooked filesystem on node 1 for backups) from where netbackup picks it up and tapes them. The team is a bit concerned with the learning curve involved with RAW and also the maintenance complexities involved in db cloning etc (eg. recently we were asked to clone this RAC database to a non-RAC database on a different host).
One thought inside the team is to do away with RAW and put ASM on OCFS2 filesystem (in which case we won't have to maintain a separate /u02/backup at all plus no learning curve to manage RAW involved). However we do acknowledge that by doing so, we won't be able to reap the benefits of RAW long-term (when the usage of our RAC instances goes up). Also, I believe Oracle suggest ASM on RAW (could be wrong but that is what I see generally people talking about).
Any suggestions/advices for or against having ASM created on OCFS2 (or even NFS etc)?
In case that helps, the servers are Dell PE with RHEL4 and Oracle 10.2.0.3. Our duties are well defined between the storage group, Linux group and DBAs.
Thank you,
- Ravi

Dan,
There are some things about ASM that make it easier than a FS, but there are others that are more difficult; there is definitely a tradeoff. For the DBA who is coming from a background that is light on hardware, the things that ASM does best are "black box", tasks that a sysadmin or an EMC junkie normally do. The "simple" things a normal DBA would do (copy files, list files, check sizes) are now taken through another layer (whether you go asmcmd or a query against the ASM instance, or RMAN). Kirk McGowan briefly talked about how the job role of the DBA has changed with the new technology:
http://blogs.oracle.com/kmcgowan/2007/06/27#a12
Let's look at two "simple" things I have come across so far that I would like to see improved. First is resolving archivelog gaps:
Easiest way to fill gap sequence in standby archivelog with RAW ASM
Yes, we all know dataguard can do this. But this is not a thread about dataguard (I am more than willing to talk about it in another thread or privately). With ASM on Raw (from now on, I will just say ASM and assume Raw), you have to use RMAN. I have no problem saying that all of us should become better at RMAN (truly), but it bothers me that I cannot login to my primary host and scp a group of logs from the archive destination to the archived destination on my standby host. Unless of course you put your archive destination on a cooked FS. But then we go back to the beginning of this thread.
Another "simple" tasks is monitoring space usage. ASM has a gimped version of 'du' that could stand a lot of improvement. Of course, there is sqlplus and just run a nice hierarchy query against one of the v$asm views. But 'du -sk /u0?/oradata/*' is so much simpler than either approach.
Which leads me to ask myself whether or not we are approaching disk monitoring from a completely wrong angle. What does the 'A' stand for in ASM? grin
There is a lot that ASM can do. And I have no doubt that, due to my lack of experience with ASM, I am simply "not getting it" in some cases.
"While it may seem painful in the midst of it, the best way to overcome that learning curve is to diagnose problems in a very hands-on manner." - Kirk McGowan

Similar Messages

OCFS2 , ASM and Raw Device for Voting Disk and OCR

Dear all ,
My manager assigned me to establish a 11gRAC , 3 node on OEL. He likes to locate ASM with datafiles , at the same time, 2 OCR vol on OCFS2 and RawDevice respectively , 2 voting disks on OCFS2 and RawDevice respectively also.
This really annoys me .
Oh all experts, could anybody give me some advices ? Can those stuff be co-existed ?

Billy Verreynne wrote:
MccLok wrote:
My manager assigned me to establish a 11gRAC , 3 node on OEL. He likes to locate ASM with datafiles , at the same time, 2 OCR vol on OCFS2 and RawDevice respectively , 2 voting disks on OCFS2 and RawDevice respectively also. Not a great idea to introduce additional s/w layers between the CRS software and its OCR and voting disks. If ocfs2 is for example used as a shared file system for these devices (files), then the ocfs2 cluster s/w needs to be up and running, the shared devices mounted, before CRS can successfully start. Thus any problem with the ocfs2 stack, will impact on CRS. Why? What does ocfs2 buy you in this regard? What are the benefits that justifies the increased complexity and the dependency of CRS on another s/w layer?
Fewer moving parts means less complexity, less stuff that needs to be configured and can go wrong.In fact, some users had perception that " Pay mroe so that Get more".

How to check space on asm and raw device?

Hi All,
I want to know how to check the space on asm device.
we are using asm device which is on raw partition.
Our archive gets full.I need to check the space on asm archve partiotion.
how can i check the archive destination space?
we are using raw device partition for crs and voting device.
how can we check the space on raw device and how can we know which device is in use and which are not in use
Thanks in advance

4 - 5 raw partitions...
What oracle version?
Install OCR + VOTE in raw device better than OCFS2. if you use 10g to 11gR1.
If use 11gR2,keep OCR + VOTE in ASM Disk Group.
you can check which raw device OCR:
$ ocrcheck
VOTEDISK:
$ crsctl query css votedisk
On 10g if use raw device for OCR + votedisk: recommend 1 file = 250M
11g recommend 1 file = 512M
Good Luck

Using ASM without raw disks?

I realize the whole point of employing ASM in an enterprise environment would be to alleviate the need for third-party LVMs and data management at the OS level.
Still, I wonder if there is a way around the raw disk requirement for using ASM-- not in a production environment, but for the purpose of getting aquainted with its benefits and intricacies, and running through its procedures without impacting a live enviromnent (and to save the time and expense of additional hardware installation.
I have read about an "undocumented" ASM parameter called ASM_ALLOW_ONLY_RAW_DISKS, which, when set to FALSE, would allow me to do what I need to do without additional disks:
http://www.orafaq.com/parms/parm50.htm
http://www.dbazine.com/OLC/still.shtml
Unfortunately (but not surprisingly), this hasn't worked for me thus far. Has anyone had any luck with this parameter in a WinXP environment?
Any ideas on what might work toward reaching this end would be greatly appreciated.
Thank you.
John

Hi John,
I was able to get ASM working without raw disks on Windows 2000 as follows:
create init+ASM.ora for example:
*.background_dump_dest='F:\oracle\product\10.1.0\admin\+ASM\bdump'
*.core_dump_dest='F:\oracle\product\10.1.0\admin\+ASM\cdump'
instance_type=asm
*.large_pool_size=12M
*.remote_login_passwordfile='SHARED'
*.user_dump_dest='F:\oracle\product\10.1.0\admin\+ASM\udump'
*._ASM_ALLOW_ONLY_RAW_DISKS=false
*.asm_diskstring='f:\asm\*'
The setting of asm_diskstring is very important, this should be set to the directory where you are going to create the "ASM files" that will pretend to be disks.
The asmtool which is on the Oracle 10g CRS CD, can be used to create empty files, for example:
ASMTOOL -create \\.\F:\ASM\ASMDISK1.ASM 100
ASMTOOL -create \\.\F:\ASM\ASMDISK2.ASM 100
ASMTOOL -create \\.\F:\ASM\ASMDISK3.ASM 100
This will create 3 100Mb files in f:\asm\ which with should be picked up in v$asm_disk once you asm instance is restarted.
obviously you will need to change the drive letter above, as required.
Also looks like you need the OracleCSService running, which I think can be started using ocssd.
cheers,
John

ASM on raw or block devices?

What is Oracle's recommendation for Linux platform considering only performance:
a) ASM implementation on raw-bound devices (/dev/raw/raw*)
or
b) ASM implementation on block devices (/dev/hda*, /dev/hdb*...) with ASMLib?
N.J.

Starting with RH4 block devices can do direct non-buffered IO so either raw bindings or block devices would perform equally well. Using block devices has the added advantage that has less management. And, as mentioned before, raw bindings will be deprecated in some future release of Linux.
ASMLIB will not provide better performance necessarily and is more of a manageability tool. It solves the device name and permission persistence problem you may have in between reboots and provides more efficient IO management.

Move OCR and Vote Disk to a new SAN

Hi,
Can you guys help me with some links/documents for moving OCR and Vote disk files from old SAN to new SAN.
Both SAN will have ocfs2 formatted 3 partitions on which ocr and votedisk will be placed (normal redundancy).
Oracle version 11g R2,
OS linux 5.5
RAC nodes : 2
Datafiles on ASM
OCR/Votedisk on ocfs2
Regards,
Ajay

user7443016 wrote:
Thanks for the reply, but all the examples were showing either ocfs2 to ASM or raw to ocfs2. could you just point out the high level steps for moving 3 ocr & votedisk files from one san to another(ocfs to ocfs).
Hi Ajay,
Steps will be same . It doesnt matter whether it is raw to ocfs2 or ocfs to ocfs.
There will be change only how do you assign storage for OCR/voting disk.
I have another doubt that, i already have 3 ocr files, can i add more ocr files pointing to another mount point? the same goes with votedisk.
You can use replace option so there is no need to add 4th OCR disk.
i found steps in one of the website: i have few doubts in it, could any one clarify it...All the steps seems to be fine.
Moving OCR
1) create mount point(new san - /u05) in all nodes
2) precreate ocr files in new mount points (i am not sure how to do it? is it just touch and filename, its a ocfs2 file system)on ocfs you need to create file as below :
dd if=/dev/zero of=/ocfs/OCR/ocrnew.dbf bs=1M count=128
Also I would suggest to have a look at :
'ocrconfig -replace ocr' Fails With PROT-16 [ID 444757.1]
RAC Assurance Support Team: RAC Starter Kit and Best Practices (Linux) [ID 811306.1]
Regards
Rajesh

10g ASM on Logical Volumes vs. Raw devices and SAN Virtualization

We are looking at setting up our standards for Oracle 10g non-rac systems. We are looking at the value of Oracle ASM in our environment.
As per the official Oracle documentation, raw devices are preferred to using Logical Volumes when using ASM.
From here: http://download.oracle.com/docs/cd/B19306_01/server.102/b15658/appa_aix.htm#sthr
ef723
"Note: Do not add logical volumes to Automatic Storage Management disk groups. Automatic Storage Management works best when you add raw disk devices to disk groups. If you are using Automatic Storage Management, then do not use LVM for striping. Automatic Storage Management implements striping and mirroring."
Also, as per Metalink note 452924.1:
"10) Avoid using a Logical Volume Manager (LVM) because an LVM would be redundant."
The issue is: if we use raw disk devices presented to ASM, the disks don't show up as used in the unix/AIX system tools (i.e. smit, lspv, etc.). Hence, when looking for raw devices on the system to add to filesystems/volume groups/etc., it's highly possible that a UNIX admin will grab a raw device that is already in use by Oracle ASM.
Additionally, we are using a an IBM DS8300 SAN with IBM SAN Volume Controller (SVC) in front of it. Hence, we already have storage virtualization and I/O balancing at the SAN/hardware level.
I'm looking for a little clarification to the following questions, as my understanding of their responses seem to confict:
QUESTION #1: Can anyone clarify/provide additional detail as to why Logical volumes are not preferred when using Oracle ASM? Does the argument still hold in a SAN Virtualized environment?
QUESTION #2: Does virtualization at the software level (ASM) make sense in our environment? As we already have I/O balancing provided at the hardware level via our SVC, what do we gain by adding yet another level of I/O balancing at the ASM level? Or as in the
arguments the Oracle documentation makes against using Lvm, is this an unnecessary redundant striping (double-striped or in our case triple-striped/plaid)?
QUESTION #3: So does SAN Virtualization conflict or compliment the virtualization provided by ASM?

After more research/discussions/SR's, I've come to the following conclusion.
Basically, in an intelligent storage environment (i.e. SVC), you're not getting a 100% bang for the buck by using ASM. Which is the cat's meow in a commodity hardware/unintelligent storage environment.
Using ASM in a SVC environment potentially wastes CPU cycles having ASM balance i/o that is already balanced on the backend (sure if you shuffle a deck of cards that are already shuffled you're not doing any harm, but if they're already shuffled - then why are you shuffling them again??).
That being said, there may still be some value for using ASM from the standpoint of storage management for multiple instances on a server. For example, one could better minimize space wastage by being able to share a "pool" of storage between mulitiple instances, rather than having to manage space on an instance-by-instance (or filesystem by filesystem) level.
Also, in the case of having a unfriendly OS where one is unable to dynamically grow a filesystem (i.e. database outage required), there would be a definite benefit provided by ASM in being able to dynamically allocate disks to the "pool". Of course, with most higher-end end systems, dynamic filesystem growth is pretty much a given.
In the case of RAC, regardless of the backend, ASM with raw is a no-brainer.
In the case of a standalone instance, it's a judgement call. My vote in the case of intelligent storage where one could dynamically grow filesystems, would be to keep ASM out of the picture.
Your vote may be different....just make sure you're putting in a solution to a problem and not a solution that's looking for a problem(s).
And there's the whole culture of IT thing as well (i.e. do your storage guys know what you're doing and vice versa).....which can destroy any technological solution, regardless of how great it is.

ASM + RAW

We have following MICCP configuration:
* Two IBM AIX machines with 10gR2-RAC
* ASM with raw devices
* We are backing up using RMAN to ASM disk group
Concerns with this configuration:
** We didn't implement tape backup
** With our limited knowledge, what we know is, we need to convert the database to COOKED filesystem to back this to tape.
We want oracle, to give us some best practices with ASM+RAW for
* BACKUPS to tape
* Cloning
* Shadow backups
* DR
* Storage migration path ( Faster moving from one storage to ohter )
Reason:
* As of now, we are not sure of using ASM+RAW for future projects.

With our limited knowledge, what we know is, we need to convert the database to COOKED You don't need to convert you DB to any filesystem in order to take tape backups. If your MML layer can be integrated with RMAN means you have to use oracle tape backup agent provided by your Media Manager vendor, then you can use RMAN to take backup of DB sitting on RAW + ASM to tape by allocating SBT channels. There are several tape backup solutions are availabe in market and now you can also use Oracle's own OSB (Oracle Secure Backup) for this. Read Backup & Recovery guide for more on RMAN and SBT:
http://download-east.oracle.com/docs/cd/B19306_01/backup.102/b14191/toc.htm
Daljit Singh

Oracle ASM installation and raw device options

I am looking to install Oracle ASM on Linux Red Hat AS 4 (ia64) using Oracle 10.2.0.3.
A general question I have regarding prerequisites for ASM using raw devices. It appears there are 2 options for dealing with RAW devices.
1. Use the ASM library driver
or
2. simply bind the devices to raw devices (and then install a new $OH with an ASM instance)
Is there a recommendation from Oracle, industry best practices, expert opinions on which way to go?
If I understand correctly it sounds like option 2 will require an ASM $OH and instance. Is that right or do both option require this?
Can someone provide guidance as to which option I should use, an Oracle whitepaper, a recommendation, instructions or something.
Any help or insight is provided as to which option I should use.
Thank you.
echardrd

OK. Perhaps I am a little condused then. From what I read Oracle ASM recommends the use of RAW devices for ASM storage devices. What I have is a new LUN presented to my database server from the SAN. I assumed that the LUN needs to be mounted in some fashion on the server, but my SA tells me in order to mount it, would require the use of a filesystem. If we put a filesystem(NTFS, ext, etc) on the LUN prior to mounting it, will ASM be able to use it? If ASM can use a device that has a filesystem on it will this add unneccessary overhead (I/O)?
I guess my question is for ASM should I
1. use a RAW partition or a filesystem? Does it matter?
2. Will RAW have better I/O performance?
3. How do i make the LUN presentation from the SAN a usable RAW device to my database server? Any steps or guidance would be helpful since my SA and I can not seem to bridge this gap.
Thanks.

11510 to RAC ASM RAW Conversion

Hi,
We are using 11510 in multinode env, Now customer asked me to convert into RAC ASM with RAW device. My present architecture is Database is running HP-Itanium 10gR1 and Application running on HP-Tru64 11510. Please tell me any good note to implement the solutions. i have rman coldbackup backup for my database.
Thanks in Advance,
Panneer.

Metalink Note 220970.1 could be a starting point, section: "What is the optimal migration path to be used while migrating the E-Business suite to RAC?"
C.

ASM RAID 6

We might have to break the current OCFS2 implementation (on SAN) and replace it with ASM on raw devices. What is the difference when it comes to storage consumption? Do we gain or lose disk space, either case - how much?
SLES9
10.1.0.5
Appreciate your thoughts

-Did external redundancy for ASM exist in 10.1?Yes, it exists in 10.1 also.
-ASM vs. OCFS2 from a file system perspective - apart
from the manageability issue, why would ASM be
recommended over OCFS2?You can say that there are many new features which are being added into ASM but there has not been much development in OCFS.
-Amit
http://askoracledba.wordpress.com/

RAC with ASM and shared disks?

Hi all,
Can someone clarify this little point please. If I use ASM as my storage with a RAC database, I have to configure these nodes to shared disks. At least this is what the UG says ...
When you create a disk group for a cluster or add new disks to an existing clustered disk group, you only need to prepare the underlying physical storage on shared disks. The shared disk requirement is the only substantial difference between using ASM in a RAC database compared to using it in a single-instance Oracle database. ASM automatically re-balances the storage load after you add or delete a disk or disk group.
With my 9i databases, I used HCAMP to allow for concurrent VG access among the nodes. My questions are ...
1) How can I share this storage as stated above without using HACMP? My understanding is with 10g I no longer have to use it.
2) Can Oracle's clusterware be used to share storage? I have not seen any indication that it does.
3) Does this mean I still have to use HCAMP with 10g crs to allow shared storage?
Thank you

"...meaning visible to all the participating nodes, which you don't need HACMP..."
This is one step forward, but still not clear. On unix, storage is presented to ASM as raw volumes. As such, how can these volumes be visible on all nodes without using HCAMP (or whatever 3rd party clusterware you are using). Presenting raw volumes on several nodes is something that is not done at OS level without using some clusterware functionality.
I do understand that storage or LUNs can be shared at the SAN fabric level. But then, these LUNs are carved in bug chunks and I would like to be able to allocate storage at much granular level using raw partitions.
So all in all, here are my questions ...
1) On unix platforms, can ASM disks be LUNs, raw volumes, or may be both?
2) If raw volumes, how are these shared (or made visible) without using 3rd party clusterware? Having managed 9i RAC, it was the function of HACMP to make these volumes visible on all nodes, otherwise, we had to imp/exp VGs on all nodes to make them visible.
Thank you

Multiple ASM instances on a single node

Can i have multiple ASM instances on a single node? This is to have each instance supporting different environment dev,stage etc
Thanks
Sannidhi

I had been discussing the same issue with someone from Oracle. I asked for multiple ASMs on a server so that we could have seperate ASMs running for, say, 10g and 11g.
He explained that we should think of ASM as being the same as Veritas FileSystem. We don't run multiple instances of Veritas on our servers. On single Veritas "instance" (set of drivers) provides all the Veritas Fx mountpoints on the server. Similarly, there should be only 1 ASM instance on a server.
(substitute UFS or ZFS or CFS for Veritas in the above example and you still have only 1 filesystem manager on a server providing one type of filesystem, although you could have, say , UFS and Veritas co-exist ; just as you could have Veritas, ASM and Raw co-exist on a server).
Hemant K Chitale
Edited by: Hemant K Chitale on Sep 29, 2009 11:50 AM

ASM Configuration on oracle 10.2 RAC

hi all
os : sun sparc solaris 10
db : 10.2 rac database
clusterware configured using raw devices.
we have no other option than configuring asm on raw devices.
could u tell me narrows steps how to do it manually as well as using dbca

Hi
I've do mine last week but it's in HPUX not sun and not with raw device but i think it's the same
Steps:
1- Launch dbca
2- Choose configure ASm
3- Select all the nodes
4- Choose Create Initiialization parameter file and click on ASM parameters and in the asm_diskstring parameter specify the disk you want to use in asm group example
vg01/vol1,vg01/vol2, ... Don't specify first the disk group
5- Create new disk group
6- choose the disk candidate for the group
7- Finished
regards raitsarevo

Deleting raw device content

Hello,
After one installation of cluster software (with problem) I try to install one more time, but the last installation of cluster software leaves some data on my raw devices (for OCR and Voting Disk I use direct raw devices without OCFS, for data files ASM on raw devices).
How can I really delete the content of raw devices? When I delete partitions and create new, it does not delete the content.
Thanx,
Jacek

I don't know your OS, this is for Linux:
dd if=/dev/zero of=<raw_device>
This command writes binary 0's to the destination.
Be careful when specifying the 'of' argument, this command is used also to clean a whole harddisk (when you plan to sell it).
Werner

ASM on RAW or OCFS2

Similar Messages

Maybe you are looking for