ASM: cannot create asm disks - permission denied

Question

We cannot create ASM disks using the command:
# /etc/init.d/oracleasm creatdisk ASMDISK1 /dev/mapper/mpathdp1
Marking disk "ASMDISK1" as an ASM disk: [FAILED]
The log (/var/log/oracleasm) has the following:
Clearing disk header: done
Disk "ASMDISK1" does not exist or is not instantiated
Writing disk header: done
Instantiating disk: oracleasm-instantiate-disk: Unable to create ASM disk "ASMDISK1": Permission denied
failed
Clearing disk header: done
The command is being executed by the root user. Our environment is OEL6:
[oracle@uxdbidbd1 mapper]$ uname -rm
2.6.32-100.28.11.el6.x86_64 x86_64
To install ASMLib we did the following:
Installed the three packages:
1.oracleasm-support-2.1.4-1.el5.x86_64.rpm
2.oracleasmlib-2.0.4-1.el5.x86_64.rpm
3.oracleasm-2.6.18-238.9.1.el5-debuginfo-2.0.5-1.el5.x86_64.rpm
[root@uxdbidbd1 ~]# modprobe oracleasm
[root@uxdbidbd1 ~]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: oracle
Default group to own the driver interface []: oinstall
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [  OK  ]
Scanning the system for Oracle ASMLib disks: [  OK  ]
[root@uxdbidbd1 ~]#
[root@uxdbidbd1 ~]# /etc/init.d/oracleasm status
Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes
Our SA partitioned the disk using fdisk:
[root@uxdbidbd1 ~]# fdisk -l /dev/mapper/mpathd
Disk /dev/mapper/mpathd: 52.4 GB, 52428800000 bytes
255 heads, 63 sectors/track, 6374 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x4849251a
Device Boot Start End Blocks Id System
/dev/mapper/mpathdp1 1 6374 51199123+ 83 Linux
Here are the owner/permissions:
[oracle@uxdbidbd1 mapper]$ ls -al /dev/mapper/
lrwxrwxrwx. 1 root root 7 Apr 22 14:58 mpathd -> ../dm-5
brw-rw-r--. 1 oracle oinstall 253, 6 Apr 25 10:56 mpathdp1
[oracle@uxdbidbd1 dev]$ ls -al /dev
brw-rw----. 1 root disk 253, 5 Apr 22 15:45 dm-5
drwxr-xr-x. 4 root root 0 Apr 22 14:58 oracleasm
[oracle@uxdbidbd1 oracleasm]$ ls -al /dev/oracleasm
total 0
drwxr-xr-x. 4 root root 0 Apr 22 14:58 .
drwxr-xr-x. 23 root root 7200 Apr 24 03:41 ..
-rw-rw----. 1 oracle oinstall 0 Apr 22 14:58 .check_iid
drwxr-xr-x. 1 root root 0 Apr 22 14:58 disks
-rw-rw----. 1 oracle oinstall 0 Apr 22 14:58 .get_iid
drwxrwx---. 1 oracle oinstall 0 Apr 22 14:58 iid
-rw-rw----. 1 oracle oinstall 0 Apr 22 14:58 .query_disk
-rw-rw----. 1 oracle oinstall 0 Apr 22 14:58 .query_version
We ran an strace on the createdisk command that is failing and I noticed this in the output file:
28830 mknod("/dev/oracleasm/disks/ASMDISK1", S_IFBLK|0600, makedev(253, 6)) = -1 EACCES (Permission denied)
28830 write(2, "oracleasm-instantiate-disk: ", 28) = 28
28830 write(2, "Unable to create ASM disk \"ASMDI"..., 56) = 56
We changed /dev/oracleasm/disks to 777 in order to test but we got the same result. Do we need to adjust the permissions to all of the parent directories in the path (/dev/oracleasm/disks)? I've been looking around for anything related to permissions on /dev/oracleasm but the docs don't mention anything. Any help would be greatly appreciated.

Answer 1

From what I understand, you should not set device permissions of /dev/oracleasm since oracleasm takes care of it according to /etc/init.d/oracleasm configure.
The problem in your case seems to be that ASM refuses to initialize and write metadata to your device. Was the device previously used by ASM? ASM will fail to initialize a device that contains previous ASM metadata. To erase the device you will need to use the dd command, e.g. dd if=/dev/zero of=/dev/sdb. Both ASMLib and raw devices require the candidate disks to be partitioned before they can be accessed. To create a partition on /dev/sdb, for instance, type fdisk /dev/sdb and create a standard partition using the defaults. This will create /dev/sdb1 and you can try oracleasm createdisk again.
You may try the above, but anyway, I was not sure how much you can trust your installation. The problem could also be that you installed the oracleasm kernel driver and asmlib software for OEL 5. Oracle Linux 6 includes an oracleasm kernel driver and as far as I remember you must not install another oracleasm kernel driver. Using modprob oracleasm and installing oracleasm support package using yum should do the trick. You may also need to download and install the asmlib package for the Oracle ASM grid install to see the ASM disks.