RAID 5 and filesystems

Question

I have a SunFire server with 3 disks. My primary goal is to have the metadb spread across all three disks in order to achieve the required quorum, in the event a disk goes bad and there is a need to reboot. Additionally I would like to use RAID 5.
The Sun documentation specified the following regarding RAID 5:
--- Cut Here ---
Like a mirror, a RAID 5 volume increases data availability, but with a minimum of cost in terms of hardware and only a moderate penalty for write operations. However, you cannot use a RAID 5 volume for root (/), /usr, and swap, or for existing file systems
--- Cut Here ---
Does this mean that I would have to use some other scheme (Such as RAID 1) for root(/), /usr and swap?
The metainit command is required for RAID 5 and expects as arguments whole disks and not slices. If anybody has implemented RAID 5 or some unique combination of RAID 5 to circumvent the afformentioned FS limitations, I would certainly like your feedback.
Many Thanks,
soul_survivin

Answer 1

I use LVM-on-RAID, but I think of the layers as:
RAID - do fancy stuff like mirroring/striping/etc. for performance + security
LVM - make my life easier when it comes to allocating diskspace
That is, I usually put the LVM on top of RAID, giving all of the RAID device to LVM as a physical volume and volume group, and then allocate only what I currently need for my system into logical volumes (for /, /home, etc.). I might not originally have had a separate /var/ partition, but then realize it'd be cool to try it out on another filesystem to improve pacman performance, at which point I can create a new logical volume and mount that as /var/. Then, when I realize that there's an even *better* filesystem out there (as there inevitably is), I can simply create another logical volume with the new filesystem (assuming I still have space in my volume group), sync the stuff over from my old var, and mount the new one as /var/. Once I'm satisfied that the new /var/ is good and performs better, I can delete the old logical volume and reclaim the space. So, LVM to me is more about flexibility with my filesystems/mountpoints and not having to worry about fiddling the partition table and blowing stuff up.
If you're mainly concerned with growing RAID arrays, you can probably skip LVM; the way I use LVM, you'd have to grow the RAID array under LVM first, anyway.