Disk Layout for a RAC deployment on a shoe string

Question

Hi, not sure if this is better placed under RAC or Linux group.
I am planning a demo RAC system using 2 linux servers and due to stingy managers, the smallest budget ever imagined !
I will have the 2 servers networked but am looking at a cheapo disk setup ( excluding virtualisation and SAN/NAS ). I currently have
Server A and Server B
Each has an Internal disk of 250 Gb – I am going to place the linux OS on here, the 11gR2 Oracle Grid Infrastructure Home and the 11gR2 Enterprise Edition Oracle Home
Q1. Since the local server is where the Oracle Instance runs – would I be correct in assuming this is where the SPFILE for the instance goes as well i.e. Server A and B will both hold an SPFILE in the 11gR2 ORACLE_HOME ?
Q2. What else should be stored on the local server – all the usual logs etc ( eg the instance alert log etc ) ? Is it normal to put a copy of the database control file on each local server ?
My plan was to carve up the 250 Gb disk into a root partition ( for Linux ) and a /u01 partition for the Oracle Grid Infrastructure Home and the Database Oracle Home
I was then going to use the remainder of the 250 Gb Disk in Server A as the shared disk ( for RAC ) between the Servers.  I am going to attempt to NFS mount the remainder of the 250 Gb disk on Server A ( disgusted I have no budget for NAS ! ), so Server B will have access to it.
What I was not sure about is –
Q3. in regards to ASM Disk Groups, I wasn’t sure if ASM can discover disk space for use, once it has been mounted as a standard file system ( or in my case as an NFS mount ) ?
Q4. How does ASM interact with standard file system mounts ( if at all ) ?
Q5. If a physical disk has some standard filesystems mounted on it, can ASM use / discover any remaining part of a disk that has not been configured as a filesystem ?
Q6. I am assuming the FRA is better placed on the Shared Disk ( so both nodes have access to it ) ?
Any guidance would be appreciated,
Jim

Answer 1

This won't be a RAC, Jim.
What are you going to use as Interconnect? Do these machines have a 2nd physical NIC? You realise that crossover Ethernet cables are not supported. Is this Gigabit Ethernet? RAC Interconnect over a 10 or even 100 Mb/s  Ethernet, is significantly below par.
Where are the OCR and Voting Disks going to reside?
ASM is in my view, mandatory. It makes no sense not using ASM for RAC. ASM needs raw devices to use. It does not run on cooked file systems. NFS shares the latter.
If NFS is used, over what IP layer is it going to run? The same as the Interconnect? Which would be  an irony as the the RAC db cache is now using the same transport layer as it does for physical I/O - both are over slow Ethernet vying for the same bandwidth.
RAC is only as sound and as robust as the h/w it is build on. Stuff up with the hardware and you WILL have an inferior and unstable RAC. Accompanied with utter frustration as it will not work the way it should.
IMO - the ONLY way one can build a 2 server RAC without a SAN or NAS, is with Firewire. Using a shared Firewire 800 disk that supports 2 separate servers at the same time. And then use this Firewire disk as shared storage. This is not supported by Oracle for production use - but past support notes did say that this is a config that can be considered for a dev type RAC. 
If the 2 servers do not support Firewire 800, then 2 x Firewire 800 PCI cards (and cables) are needed, in addition to the Firewire drive.