Recommended storage for Network-based accounts?

Question

Hello everyone. Sorry in advance for all the following information, but I want to thorough in hopes of allowing you to offer better input. I'm using an Xserve 2 x 2.8 Ghz. Quad Xeon (Mac OS X Server 10.6.6) with 22 GBs RAM and 6 GB NIC LACP bond to our backbone switch. This switch feeds three labs, each with their own gigabit switch, with a total of approximately 50 iMacs combined. All iMacs are connected via gigabit ethernet. All user accounts are network-based, bound to the Xserve via OD via AFP. I have WGM folder-redirects to keep the user caches folder and some of the Adobe stuff off the network for better performance. Primary software is the Adobe Creative Suite Premium CS5 throughout; with one lab using Final Cut Express & Pro and Adobe After Effects. This lab has local partitions for the high I/O requirements of video editing (so I'm not looking to sustain multiple HD streams over the network. etc.). I installed an 8TB OWC Mercury Rack Pro (external hardware RAID enclosure) with an Oxford 936 chipset this past summer, which is currently configured as RAID 5 and connected to the Xserve via a NewerTech 6GB-capable SATA host card (also provided by OWC). All of our network home directories are on the OWC Mercury Rack Pro. We also upgraded two of our three labs with brand new 27" Intel i5 iMacs this past summer. Lastly, I upgraded the Xserve to Snow Leopard also this past summer.
The Problem:
Since the upgrades this past summer (Snow Leopard Server, OWC Mercury Rack Pro, new iMacs), network account performance is notably more sluggish (log-in, opening apps, etc.) compared to before the upgrades (Xserve was running Server 10.5.8, labs had Mac Mini systems with gigabit running Mac OS X 10.5.8 and Adobe CS4). My network accounts were on an eSATA Rocstor ArticRoc RAID 5 unit previously, connected to the Xserve via an older Sonnet Tempo-X SATA card (which was PCI-X, not PCI-Express).
Turns out the new iMacs don't support jumbo frames (yikes!), but notwithstanding that issue, it appears like the new Mercury Rack Pro might not be performing well under load. I've done some testing using OWC's provided QuickBench software. I logged into 3 iMacs using a local admin account, mounted three separate home directories from the Xserve and started testing performance simultaneously (to simulate multiple user access). The iMacs were next to each other, so my tests were started about 1 second apart, but were otherwise running simultaneously. Here's the results for review:
The tests were performed without file caching enabled, to better gauge the raw storage performance. The results for each test file size are in MB/sec and the 4 result columns in order from left to right are Seq. Read, Seq. Write, Rand. Read, Rand. Write. The averages are totaled at the bottom. Hope this comes through in a readable fashion...
iMac-1 Test:
4 KB 10.499 0.502 10.645 0.087
8 KB 16.351 4.179 15.913 0.143
16 KB 29.35 11.213 28.878 0.213
32 KB 39.703 19.318 40.633 0.251
64 KB 55.519 27.347 51.766 0.335
128 KB 70.823 38.541 63.345 0.414
256 KB 78.946 46.074 70.803 0.383
512 KB 87.872 56.071 77.47 0.325
1024 KB 93.209 60.616 87.667 0.281
Average 53.586 29.318 49.68 0.27
iMac-2 Test:
4 KB 9.901 0.494 10.843 0.08
8 KB 14.208 5.116 15.942 0.142
16 KB 22.762 9.668 26.973 0.174
32 KB 30.357 16.301 42.276 0.183
64 KB 2.605 25.486 51.606 0.179
128 KB 4.831 28.404 18.495 0.308
256 KB 87.839 43.936 87.014 0.404
512 KB 96.93 28.836 95.64 0.335
1024 KB 99.789 40.661 71.096 0.318
Average 41.025 22.1 46.654 0.236
iMac-3 Test:
4 KB 4.689 0.79 10.348 0.065
8 KB 7.908 5.526 16.399 0.086
16 KB 6.848 8.783 27.967 0.056
32 KB 30.183 14.756 42.096 0.132
64 KB 46.42 13.255 53.114 0.277
128 KB 74.744 11.307 4.369 0.424
256 KB 80.955 25.521 26.432 0.484
512 KB 97.356 16.138 65.667 0.386
1024 KB 103.434 44.617 103.015 0.612
Average 50.282 15.632 38.823 0.28
It appears that small file performance is poor (historically a problem via AFP, I recall), but the Random Write performance is what scared me the most. It's very low across the spectrum. I'm going to provide these results to OWC for review, but wanted to get some additional perspective from the community. I'd appreciate any thoughts or ideas you might share.
Related Question:
We may have 35 users logged in at peak time. But given that I'm hosting network accounts for approximately 50 gigabit-equipped Macs, what would you recommend (or are you using) for storage based on my usage criteria mentioned earlier? I'm hoping there's a solution that's less expensive than the fibre-channel Promise RAID (or equivalent); as our budget unfortunately won't support that. Any storage solutions in the SATA realm that might be sufficient for hosting home directories where video capture isn't required?
Thanks for your patience and your advice!
Regards - Zeek

Answer 1

RE: options (b) or (c) with external firewire or USB drives -- if you go this route, are you thinking of afp- or smb-mounting (i.e., ⌘k in Finder) the mini or cube or G3? If so, you'll want to get ahold of SharePoints (unless you know how to create mount points in NetInfo), so you can create an additional mount point on the mini or cube or G3 for your external drive. As you know, when you afp-mount another Mac, the mount points that show up are the individual user accounts on that other Mac /Users/{shortUserNameGoesHere}. But an external firewire drive would not be visible at /Volumes/{extDriveNameGoesHere} on the mini or cube or G3 from these User mount points because /Volumes/{extDriveNameGoesHere} is not in the path of /Users/{shortUserNameGoesHere}. The only way that you could get there would be if you connected as admin on the cube or mini or G3, and then mounted the mini or cube or G3 at its root (/). But you might not want to let regular user accounts access the cube or mini or G3 as admin. But SharePoints will let you define that mount point, so when you ⌘k, it shows up just like the user names do. I am not familiar with smb mounts (for the benefit of your Wintel boxes) but SharePoints will let you define smb mount points, too. It has enabled me to create a "community (inbound) fax" afp mount point, for remote users to retrieve faxes from my computer, as well as permit a "central" afp-mount point for all my users' dropboxes, so you don't have to mount every user if you have multiple drops to make. (That required me to actually move all the drop boxes to a common folder, and make aliases to those locations back in the original users' drop box locations). So you might want to look into SharePoints if you decide to go this route.
(if you find that this solves your problem, or is actually helpful towards arriving at a solution to your problem, please consider clicking on either the "helpful" or "solved" buttons in the header of this post)