Storage rules for an editing rig. Some basics.

Question

How do you set up your editing machine in terms of disks for maximum performance and reliability? (SSD's are left out here.)
This is a question that often arises and all too often one sees that initial settings are really suboptimal. These rules are intended to help you decide how to setup your disks to get the best response times. Of course the only disks in an editing machine must be 7200 RPM types or faster. No GREEN disks at all.
Rule 1: NEVER partition a disk. You may ask why? First of all, it does not increase disk space, it just allocates the space differently. However, the major drawback is that for a partitioned disk the OS must first access a partition table at the beginning of the disk for all accesses to the disk, thus requiring the heads to move to the beginning of the disk, then when it has gotten the partition info move to the designated area on the disk and perform the requested action. This means much more wear-and-tear on the mechanics of the disk, slower speeds and more overhead for the OS, all reducing efficiency.
Rule 2: Avoid using USB drives, since they are the slowest on the market. Do not be tricked by the alleged bandwidth of USB 2.0 advertisements, because is just is not true and remember that the alleged bandwidth is shared by all USB devices, so if you have a USB mouse, keyboard, printer, card reader or whatever, they all share the bandwidth. Stick to SCSI or SATA disks or e-SATA. If needed, you can use Firewire-800 or even Firewire-400 disks, but they are really more suited for backups than for editing.
Rule 3: Use at least 3 different physical disks on an editing machine, one for OS/programs, one for media and one for pagefile/scratch/renders. Even on a notebook with one internal drive it is easy to accomplish this by using a dual e-SATA to Express card connector. That gives you an additional two e-SATA connections for external disks.
Rule 4: Spread disk access across as many disks as you have. If you have OS & programs on disk C:, set your pagefile on another disk. Also set your pagefile to a fixed size, preferably somewhere around 1.5 times your physical memory.
Rule 5: Turn off index search and compression. Both will cause severe performance hits if you leave them on.
Rule 6: If the fill rate on any of your SATA disks goes over 60-70% it is time to get a larger or an additional disk.
Rule 7: Perform regular defrags on all of your disks. For instance, you can schedule this daily during your lunch break.
Rule 8: Keep your disks cool by using adequate airflow by means of additional fans if needed. You can use SMART to monitor disk temperatures, which should be under 35 degrees C at all times and normally hover around 20-24 C, at least in a properly cooled system.
Rule 9: If people want raid, the cheapest way is to use the on-board IHCR or Marvell chip, but it places a relatively high burden on the CPU. The best way is a hardware controller card, preferably based on the IOP348 chip. Areca ARC and ADAPTEC come to mind. 3Ware uses it's own chipset and though not bad, they are not in the same league as the other two. Promise and the like in the budget range are no good and a complete waste of money. Expect to spend around $ 800 plus for a good controller with 12 connectors internally and 4 e-SATA connectors. Important to consider in a purchasing decision is whether the on-board cache memory can be expanded from the regular 256/512 MB to 2 or even 4 GB. Be aware that 2 GB cache can be relatively cheap, but the 4 GB version extremely costly ($ 30 versus $ 300). For safety reasons it is advisable to include a battery backup module (BBM).
Rule 10: If you can easily replace the data in case of disk failure (like rendered files), go ahead and use raid0, but if you want any protection against data loss, use raid 3/5/6/10/30/50. For further protection you can use hot spares, diminishing downtime and performance degradation.
In general when you get a new disk, pay close attention to any rattling noise, do perform regular disk checks, and in case of doubt about reliability, exchange the disk under guarantee. Often a new disk will fail in the first three months. If they survive that period, most of the disks will survive for the next couple of years. If you use a lot of internal disks like I do (17), set staggered spin-up to around 1 second to lessen the burden on the PSU and improve stability.
Hope this helps to answer some basic questions. If not, let me know. Further enhancements and suggestions are welcome.

Answer 1

...well, it is a northern D - they call us often "Fischköpfe" because we love to eat fish here in Hamburg!
I just have summarized a bit the storage configuration I am thinking of
RAID Type
Objective
System requirements
RAID level
Offline Storage
store a whole video project (1h of 4k material requires about 128 GB)
needs to be highly reliable (redundancy is a must);
doesn't need to be extremely fast;
discs can be cheap because they don't have a high burden (just upload & download) to the video RAID.
10
Video RAID
store material for a day work
fast and reliable
10
Installation RAID
just to install Windows XP with CS4 Master Collection
redundant but speed isn't critical here
1
Working RAID
for page-file/scratch/renders
as fast as possible
disc failure isn't a big problem
0
In order to realize this, I am thinking of the following configuration
RAID Type
number of discs
Type
GB/disc
tot. storage [GB]
usable storage [GB]
cost [€]
Offline Storage
8
SATA
1500
6000
4800
900
Video RAID
6
SCSI/SAS
300
900
720
2100
Installation RAID
2
SCSI/SAS
36
36
30
200
Working RAID
4
SCSI/SAS
147
580
470
1000
Here are my assumptions and constraints:
I only have 6 bays for the Installation - and working RAID;
For the video RAID I also would like to reuse an enclosure, which just has 6 bays;
I would need to buy a NAS enclosure - so here I am open minded and just assumed 8 bays;
the usable storrage I estimated as 80% of the total storage;
discs, which are used heavily should be SCSI or SAS - I am thinking of the Cheetah 15K
Looking into the cost associated, I hit 4000€ easily just for discs. Ok, I can reuse some discs and enclosures, which I have here - but since I need to purchase the NAS enclosure (with 8 bays), which will also cost 1000€ additional,  I will use 25% of my foreseen budget for storage.