Major performance trouble Oracle 11g RHELx64, HP 8100 storage

Question

Hi hope some one can help me with this.
We have a datawarehouse with this configuratuion
Database
RHEL5 u5 on 2* intel 5700 cpu, 36gb ram, emulex 4gb dual port hba, HP EVA 8100 san.
Oracle 11g 11.1.7.
Database 8 kblock
Disks are
/data 2.0TB 72 disks formated with ext3 4k block.
/log 100GB 72 disks formated with ext3 4k block.
ETL
Informatica Power Center 8.6 on similar machine.
My problem is vhen we try to load the database with data it takes forever so we tested this stuff
iops script run
SET SERVEROUTPUT ON
DECLARE
lat INTEGER;
iops INTEGER;
mbps INTEGER;
BEGIN
-- DBMS_RESOURCE_MANAGER.CALIBRATE_IO (<DISKS>, <MAX_LATENCY>, iops, mbps, lat);
DBMS_RESOURCE_MANAGER.CALIBRATE_IO (72, 10, iops, mbps, lat);
DBMS_OUTPUT.PUT_LINE ('max_iops = ' || iops);
DBMS_OUTPUT.PUT_LINE ('latency = ' || lat);
dbms_output.put_line('max_mbps = ' || mbps);
end;
Results
Warehouse database about 600GB data in it
max_iops = 82
latency = 11
max_mbps = 204
LOW!!!
I installed a new database with just the exaple schemas on same disk /data and got theese results
max_iops = 25665
latency = 8
max_mbps = 517
Any one have some ideas what i should test or change?
/Jens Bäckström (Noob)Dba

Answer 1

The second database IOTEST is a small test db.
This is how i did the test
i stopped the DWRPD01 wich is the dw database.
i start the IOTEST DATABSE and run the iops script.
after the run i stoped IOTEST and started DWPRD02 and run the same script.
i am sure the database files was on the same san disk group . but ofc i can not besure that the data be on the same physical disk in the SAN cabinett.
Ill check ur other questions as soon as i have some ansvers
and thanks for helping me out.
Below are my initfiles from the DB's
/Jens
IOTEST Database
IOTEST.__db_cache_size=16106127360
IOTEST.__java_pool_size=268435456
IOTEST.__large_pool_size=268435456
IOTEST.__oracle_base='/oracle_prod01'#ORACLE_BASE set from environment
IOTEST.__pga_aggregate_target=6442450944
IOTEST.__sga_target=21474836480
IOTEST.__shared_io_pool_size=0
IOTEST.__shared_pool_size=4294967296
IOTEST.__streams_pool_size=268435456
*.archive_lag_target=1800
*.audit_file_dest='/oracle_prod01/admin/IOTEST/adump'
*.audit_sys_operations=true
*.audit_trail='db'
*.compatible='11.1.0.0.0'
*.control_file_record_keep_time=16
*.control_files='/iotest/IOTEST/control01.ctl','/iotest/IOTEST/control02.ctl','/iotest/IOTEST/control03.ctl'
*.db_block_checking='LOW'
*.db_block_checksum='TYPICAL'
*.db_block_size=8192
*.db_create_file_dest=''
*.db_domain=''
*.db_file_multiblock_read_count=32
*.db_files=200
*.db_name='IOTEST'
*.db_recovery_file_dest=''
*.db_writer_processes=3
*.diagnostic_dest='/oracle_prod01'
*.dispatchers='(PROTOCOL=TCP) (SERVICE=IOTESTXDB)'
*.filesystemio_options='setall'
*.log_checkpoints_to_alert=true
*.nls_language='AMERICAN'
*.open_cursors=512
*.optimizer_index_cost_adj=50
*.parallel_io_cap_enabled=true
*.parallel_max_servers=10
*.parallel_min_servers=2
*.pga_aggregate_target=6442450944
*.processes=512
*.recyclebin='off'
*.remote_login_passwordfile='EXCLUSIVE'
*.sga_target=21474836480DWPRD01.__db_cache_size=16106127360
*.star_transformation_enabled='TRUE'
*.undo_tablespace='UNDOTBS1'
DWPRD01 Database
DWPRD01.__java_pool_size=268435456
DWPRD01.__large_pool_size=268435456
DWPRD01.__oracle_base='/oracle_prod01'#ORACLE_BASE set from environment
DWPRD01.__pga_aggregate_target=6442450944
DWPRD01.__sga_target=21474836480
DWPRD01.__shared_io_pool_size=0
DWPRD01.__shared_pool_size=4294967296
DWPRD01.__streams_pool_size=268435456
*.archive_lag_target=1800
*.audit_file_dest='/oracle_prod01/admin/DWPRD01/adump'
*.audit_sys_operations=true
*.audit_trail='db'
*.compatible='11.1.0.0.0'
*.control_file_record_keep_time=16
*.control_files='/dbffast_prod01/control01.ctl','/oracle_prod01/control02.ctl','/log_prod01/control03.ctl'
*.db_block_checking='LOW'
*.db_block_checksum='TYPICAL'
*.db_block_size=8192
*.db_create_file_dest=''
*.db_domain=''
*.db_file_multiblock_read_count=32
*.db_files=200
*.db_name='DWPRD01'
*.db_recovery_file_dest=''
*.db_writer_processes=3
*.diagnostic_dest='/oracle_prod01'
*.dispatchers='(PROTOCOL=TCP) (SERVICE=DWPRD01XDB)'
*.filesystemio_options='setall'
*.log_archive_dest_1='LOCATION=/arch_prod01'
*.log_archive_format='DWPRD01_%t_%s_%r.log'
*.log_checkpoints_to_alert=true
*.nls_language='AMERICAN'
*.open_cursors=512
*.optimizer_index_cost_adj=50
*.parallel_io_cap_enabled=true
*.parallel_max_servers=10
*.parallel_min_servers=2
*.pga_aggregate_target=6442450944
*.processes=512
*.recyclebin='off'
*.remote_login_passwordfile='EXCLUSIVE'
*.sga_target=21474836480
*.star_transformation_enabled='TRUE'
*.undo_tablespace='UNDOTBS1'