Poor SSD disk IO speed in Oracle Linux 6.3 (Windows migration)

Question

Hello,
I am trying to migrate from Windows to Oracle Linux, but I'm seeing very poor disk IO speeds. It's probably a tuning thing, but I'm relatively new to Oracle Linux and could use some detailed advice.
I took one physical server and migrated it from Windows 2008R2 to Oracle Linux 6.3 while maintaing the same Oracle version (11.2.0.3 Enterprise with ASM) and the same hardware (quad CPU 48 core HP DL585 G7 with 128GB RAM, 7 LSI 9200-8e HBAs, 28 Samsung SSD Drives). Disk IO performance, as measured using Oracle IO Calibration, was ~7,800MB/Second and 440K IOPS under windows but fell to ~2,400MB/Second and 250K IOPS under Linux.
Oracle Linux and the DB were installed using default values. The Oracle tools seem to have done a great job setting all of the obvious IO tuning parameters like the scheduler, but I figure that there are other important IO-related OS or DB parameters and that I have failed to configure the system properly.
My goal for the migration is sequential read IO speed and I would have bet money that Linux would provide better performance than Windows. I still think that it should. What basic IO tuning should I do for Oracle Linux using ASM and SSD drives?
Thank you!
Some details:
Oracle DB 11.2.0.3 enterprise installed via the GUI with the "Data Warehousing" template
ASM - single disk group, 28 SSD disks, AU=4MB
Oracle memory: Automatic memory management, 64GB allocated
Non-default Oracle params: filesystemio_options=setall, disk_asynch_io=true
Edited by: 975524 on Dec 7, 2012 8:56 AM

Answer 1

Thanks "dude" for the advice. Unfortunately, I am still seeing low IO speeds.
The default scheduler for OEL 6.3 with the DB pre-install package is deadline, which seemed like a far better choice than CFS. Based on your advice, I tried noop this morning and got the same results. I also tested with and without hugepages and saw only a small difference - at least in IO speed - I did not test overall DB performance. Lastly, I understand the /dev/shm issue, but even with the default configuration I'm getting 64MB allocated to Oracle, which is far more than is needed to test sequential IO - in fact I can get better results by using less RAM.
To answer your questions, I am testing using Oracle IO Calibration, which is an IO testing feature of the Oracle DB that is similar to the standalone Oracle Orion tool. I also performed a few tests using IOMeter, but found that the Linux version of that product was not giving me consistent data. The overall trend was the same however - IO on the Linux version far lower than the same hardware running Windows. The system is functioning very well, so I assume that everything has been installed correctly, but I do not think that it was installed optimally - thus my cry for help.
I am so surprised that Linux is showing slow IO!
Edited by: 975524 on Dec 7, 2012 9:22 AM