Creating a 100TB+ Storage Volume.

Question

Hi,
I have to create a 100TB+ Storage center for several Mac Pro connected into a fibre network. I want to use some Promise Vtrak products.
Is it possible to use 4x Promise Vtrak 32GB to create one unique raid 5 100TB+ volume? How can I interconnect the 4 peripherals? With a special raid card? or an XServe with fibre channel card?
I am beginner with this situation but I have no choice to perform a configuration like that.
Thanks for your help.
Best regards

Answer 1

Not an answer follows.
At that scale of storage, you may well hit the inevitability of a secondary spindle failure during the overhead of the RAID-5 recovery from the primary spindle failure. Run the bit error rates per the CMU and Google papers of empirical MTBF for disks, and see where you get. (In general, I'd pick a different RAID, as RAID-5 was really good for dinky and expensive disks, but that recovery processing is a massive I/O hit and suffers an increasing risk of secondary failures as the volumes scale upwards.)
Various sites I deal with operating at this scale also have the RAID and all of the block-level processing entirely out at the controller, and not at the host. That includes either journaling, archival processing, and related. (And there exist storage controllers - not sufficiently familiar with the Promise stuff here to tell you if that product features sufficient virtualization - that present the host with virtualized storage, where the SAN doesn't need to allocate the storage to the virtual disk that the host sees until the host actually needs it. That allows the host OS or application to "think" that there's a petabyte or so, but to allow the disk spindles to be added as capacity requirements track.)
Some of the SAN controllers can also stripe that data across far more spindles than can RAID-5, which means you'll have both better use of your spindles, and you'll avoid the transfer limits of the spindles; the fastest you can push an n-volume RAID-5 is the aggregate of those n spindles, where a controller-based non-RAID-5 might be able to get some slice of most or all of your spindles, and you'll be able to use slices across more (or all) of your disk spindles.
Another limit you'll hit with SAN is how fast you can push the FC SAN connections, and the only way around that is more fibre, and spreading your spindles across controllers and fibres.
Why do I mention this? How long will it take to recover that 100 TB array from your archive, or how long to create a full archive of that volume if you need to quiesce that spindle to get a good backup (and do you have archival tools that read that disk array sequentially or can you "stripe" it out?). If you're running RAID-5 with all that, you have the further limit of how fast you can push those 'n' spindles behind that array.
And yes, you might get SSDs going silly-fast here, but run the bandwidth calculations to see if your outboard or FC HBA controllers can go that fast or if you'll hit a performance wall somewhere. But then I've been dealing with some 2 Gb FC HBAs, and they're, um, slow. SSDs work best in-board, and you need sufficiently silly-fast connections to those to make that approach technically viable. (You don't want to be the guy that paid for a metric boat-load of fast SSDs and then mounted them behind 1 Gb FC HBAs.)
Other approaches involve moving to CouchDB or such databases, but that depends on what your data access patterns look like; you're at the hardware level, and sometimes, um, "adjusting" the application design can help massively.