Seeking a solid reccomendation concerning 10g RAC implementation.

Question

I need a solid recommendation regarding what hardware I should use for my planned Oracle 10g RAC environment.
Currently, I am looking for both node and SAN recommendations.
I also created a list of 11 questions that I need to have answered.
Here is how this will be configured:
3 Node Oracle 10g RAC
Each Node will consist of:
32Gb of RAM (minimum)
Windows 2003 64-bit Enterprise (corporate standard, but I can go with Windows 2008 if I have a valid justification)
74Gb RAID 1 (minimum)
Dual 10Gb Ethernet NICs
The SAN will consist of:
750Gb-3Tb dedicated to Oracle
10Gb Ethernet NICs
Here are some of my general questions:
1.) Are there any benchmarks or case studies that you are aware of that might help in the decision process regarding hardware?
2.) Where should my backups reside (SAN or elsewhere)?
3.) What margin should be added for SAN provisioning and waste (even if thin provisioning is implemented)?
4.) My CIO has decided to direct me to go with 10GbE copper instead of fiber channel, what unplanned hiccups can I expect?
5.) What are the caveats of running dual redundant SAN systems?
6.) We are currently using DELL hardware for all other systems, however I do have the freedom to change the vendor for this particular project if I can state a valid business justification. Any other vendors out there that make a better SAN or Node solution than Dell/EMC?
7.) How much local storage do I need on each node?
8.) Can I utilize 1 node to run ETL during production hours without drastically affecting database performance?
9.) Regarding the maintenance of this setup, are their any particular classes or training you would recommend? I am a seasoned Oracle DBA, but could use any specific training on the specifics of RAC and/or general performance.
10.) Are there specific advantages in Oracle 11g RAC vs. 10g that would be worth the cost of upgrading our 1000+ 10g user licenses?
11.) Are there any studies on the performance advantages/disadvantages of running Oracle 10g on Windows vs. Solaris (or any Unix)?
I will consider all input of high value, so please let me know what you would recommend.
Thank you for your time.

Answer 1

please find answers that i know for some questions.
1.) Are there any benchmarks or case studies that you are aware of that might help in the decision process regarding hardware?
2.) Where should my backups reside (SAN or elsewhere)?
=> what is the backup method you will be using? certainly recommend RMAN (in addition to OS backups that you are planning to use) for block recovery, should there a block corruption.
i would keep some copies on the SAN local and mirror to DR site too.
3.) What margin should be added for SAN provisioning and waste (even if thin provisioning is implemented)?
=> do not suggest thin provisioning for production.
how big the volumes are going to be? reason for asking the Q is, if the volumes are going to be in TB, percentages can be lower. please plan for 2 years atleast.
4.) My CIO has decided to direct me to go with 10GbE copper instead of fiber channel, what unplanned hiccups can I expect?
5.) What are the caveats of running dual redundant SAN systems?
6.) We are currently using DELL hardware for all other systems, however I do have the freedom to change the vendor for this particular project if I can state a valid business justification. Any other vendors out there that make a better SAN or Node solution than Dell/EMC?
7.) How much local storage do I need on each node?
=> have beetween 75GB to 100GB. hope you are going to have only ORACLE RDBMS and CRS software in here.
please make sure that you rotate your listener logs and alert logs and regular removal of old .trc files older than certain number of days old.
8.) Can I utilize 1 node to run ETL during production hours without drastically affecting database performance?
=> node usage will help from CPU standpoint only. what about the disk writes it is going to perform to the data files?
9.) Regarding the maintenance of this setup, are their any particular classes or training you would recommend? I am a seasoned Oracle DBA, but could use any specific training on the specifics of RAC and/or general performance.
=> attend a 5 day course from Oracle. install RAC on own and play with it.
10.) Are there specific advantages in Oracle 11g RAC vs. 10g that would be worth the cost of upgrading our 1000+ 10g user licenses?
=> did not compare, though installed 11g.
11.) Are there any studies on the performance advantages/disadvantages of running Oracle 10g on Windows vs. Solaris (or any Unix)?
=> running on windows? please do not. used on linux a lot. pretty stable and solid. recommend to go with 10.2.0.4 or higher.
hope the above will help.