RAID 5 performance

I have 4 320gb disks. I have put them in RAID 5(Intel ICH10). I have used RAID 0 and RAID 1 before. This is the first time I am using RAID 5. Read performance is excellent, but write is very slow at 15MB/sec. Is it normal for RAID 5?Single disk write for my disks is around 70MB/sec.

Those write speeds are normal for a RAID5 array using an even number of disks. To get excellent write speeds on software RAID5 controllers like the ICH8R-ICH10R, you need an odd number of disks. The only options are a 3 or 5 disk RAID5 array since there are only a maximum of 6 ports.
See this thread to get an idea of what I'm talking about. The thread is about the nForce onboard SATA RAID, but the concepts also apply to ICHR chipsets as well:
http://forums.storagereview.net/index.php?showtopic=25786
To summarize the thread, when dealing with RAID0 or RAID5, to get optimal performance you need to use aligned partitions, ideally created at offsets of 1024KB x (number of usable drives in array). The overall best stripe size to use for the best read, write performance for all filesizes is 32KB. The best cluster size to use when formatting the partition with NTFS is 32KB when storing a variety of filesizes on the array. If you are only dealing with very large files on the array (256MB+), you can get the best performance by using a 128KB stripe size for the RAID array and a 64KB cluster size for the NTFS partition. I store a variety of files on my RAID array, so I use the 32KB stripe/32KB cluster option.
The next thing you need to do is create an aligned first partition on the array. If you use Windows Vista or later to create a partition it will create an aligned first partition by default. If you are using Windows XP, then you need to use a utility called diskpar.exe (not diskpart.exe since the XP version does not have partition alignment capability, but Windows Server 2003's diskpart.exe, and Vista and later's diskpart.exe do). You can gain slightly more performance by manually aligning the partition yourself using diskpar/diskpart. If you have a 5 disk RAID5 array, you would align the first partition on the array to 4096KB, or 1024KB for every non-parity (i.e. usable) drive in the array. For a 3 disk RAID5 array, you would align on 2048KB.
Yes, you can get awesome RAID5 write speeds on an Intel onboard RAID controller using the information above. My 5 disk RAID5 array with Samsung F1 500GB drives has a maximum read speed of 350MB/s and write speeds of nearly 300MB/s. They trail off linearly as you get further into the array just as any mechanical HDD's performance does when looking at HDtach/HDTune benchmarks. You're never going to get read and write speeds that match a RAID0 array with the same number of drives, but with proper stripe/cluster/alignment you can get close. For comparison, the same 5 drives in RAID0 have a max read speed of 450MB/s and write speeds over 400MB/s. On ICHR chipsets, RAID0 arrays are not severely hampered by non-aligned partitions, but alignment does help quite a bit. For RAID5, partition alignment is essential for good write performance.
Diskpart.exe usage (done on a clean drive/array with no partitions):
1) Open a command prompt window and type diskpart then hit Enter.
2.) Type: list disk then hit enter. Look for the disk number that corresponds to your RAID array
3.) Type: select disk 1 (if disk 1 is your RAID array)
4.) Type: create partition primary align 4096 (if you have a 5 disk RAID5 array, use 2048 if you have a 3 disk array)
That's it. Format the drive making sure to select the correct cluster size (at least 32KB). If you created a first partition that didn't fill the drive, any subsequent partitions you create on the drive will be aligned because the first one is aligned.

Similar Messages

  • Very poor RAID performance

    Hi all
    Sorry about this...I know this subject has come up a lot of times on the forum already but I've been unable to find anything that might explain the problem I'm currently having.
    The board is a KT3 Ultra ARU. I'm using BIOS 5.6, a fresh install of Win XP, RAID drivers supplied with the motherboard and 2 Maxtor D740X 40GB 2MB Cache ATA 133 Harddrives.
    First of all I connected each harddrive to it's own raid socket, made sure both were set to master, set up a RAID0 array and then followed the guide to RAID setup for this board that is posted by Mr. Steveo on the AMDMB forums. I installed the raid drivers when the prompt to press F6 came up for mass storage devices and then set them as a single 75GB NTFS. I then installed the O/S without any apparent problems. Once I'd installed all the other necessary drivers I installed SiSoft Sandra 2002 and benchmarked the new RAID0 setup. The new benchmark was 21089, almost 3500pts slower than a single D740X 40GB harddrive had benchmarked at (24460).
    I began hunting around for ideas. I tried installing the raid performance patch and the pci latency patch. A tiny bit better. I decided I must have doen something wrong in the original installation and so redid everything from scratch...deleted the array, reformatted, fresh install etc.
    After completing the 2nd fresh install and changing the memory usage to cache from programs I managed to get the benchmark up to just over 26000, still nowhere near what I was expecting to see. Is this all I can realistically expect to see in terms of performance gains or does anyone here think I've missed someting in the initial installation/setup. As you can see in my sig I'm currently running my cpu unlocked at 166MHz. Although this motherboard has a 1/5 divider which should ensure the pci, agp and ide slots all run at their correct speeds when the FSB is turned up to 166MHz I never the less took the precaution of setting the FSB to 133MHz before starting out.
    I'm at a loss as to what to do. I've got hold of a copy of the Kunibert modded BIOS for full raid plus the drivers for it but don't see any point in using it if there's something more serious at fault
    Hope someone can help

    Thanks for the advice. I've now tried setting the PCI latency to 64 and 96. When I set it to 64 the HDD benchmark rose to 28967, a 3000pt gain. When I set it to 96 the score dropped to 28003. I've not tried running at 128 yet, since I assumed the score would drop again.
    Are the scores shown as comparisons in SiSoft Sandra anywhere near achievable in "real life" situations. Any idea if there's a place I can check my HDD's against ones of the same spec? I tried running PC Mark 2002 and the HDD performance was rated at 1092 if I recall correctly.
    The tests were run on a HDD's that were formatted and had just a fresh install of Win XP, fuzzy logic and SiSoft Sandra on them.
    Whilst the new score is better than the score I was getting when using just 1 40GB HDD on the normal IDE channels it was actually scoring the same until I changed the PCI latency setting, which means the old single drive set up would probably still score the same as the "faster" RAID setup

  • Raid Performance and Rebuild Issues

    Rebuilding a Raid array
    What happens when you have a Raid array and one (or more) disk(s) fail?
    First let's consider the work-flow impact of using a Raid array or not. You may want to refresh your memory about Raids, by reading Adobe Forums: To RAID or not to RAID, that is the... again.
    Sustained transfer rates are a major factor in determining how 'snappy' your editing experience will be when editing multiple tracks. For single track editing most modern disks are fast enough, but when editing complex codecs  like AVCHD, DSLR, RED or EPIC, when using uncompressed or AVC-Intra 100 Mbps codecs, or using multi-cam or multiple tracks  the sustained transfer speed can quickly become a bottleneck and limit the 'snappy' feeling during editing.
    For that reason many use raid arrays to remove that bottleneck from their systems, but this also raises the question:
    What happens when one of more of my disks fail?
    Actually, it is simple. Single disks or single level striped arrays will lose all data. And that means that you have to replace the failed disk and then restore the lost data from a backup before you can continue your editing. This situation can become extremely bothersome if you consider the following scenario:
    At 09:00 you start editing and you finish editing by 17:00 and have a planned backup scheduled at 21:00, like you do every day. At 18:30 one of your disks fails, before your backup has been made. All your work from that day is lost, including your auto-save files, so a complete day of editing is irretrievably lost. You only have the backup from the previous day to restore your data, but that can not be done before you have installed a new disk.
    This kind of scenario is not unheard of and even worse, this usually happens at the most inconvenient time, like on Saturday afternoon before a long weekend and you can only buy a new disk on Tuesday...(sigh).
    That is the reason many opt for a mirrored or parity array, despite the much higher cost (dedicated raid controller, extra disks and lower performance than a striped array). They buy safety, peace-of-mind and a more efficient work-flow.
    Consider the same scenario as above and again one disk fails.  No worry, be happy!! No data lost at all and you could continue editing, making the last changes of the day. Your planned backup will proceed as scheduled and the next morning you can continue editing, after having the failed disk replaced. All your auto-save files are intact as well.
    The chances of two disks failing simultaneously are extremely slim, but if cost is no object and safety is everything, some consider using a raid6 array to cover that eventuality. See the article quoted at the top.
    Rebuilding data after a disk failure
    In the case of a single disk or striped arrays, you have to use your backup to rebuild your data. If the backup is not current, you lose everything you did after your last backup.
    In the case of a mirrored array, the raid controller will write all data on the mirror to the newly installed disk. Consider it a disk copy from the mirror to the new disk. This is a fast way to get back to full speed. No need to get out your (possibly older) backup and restore the data. Since the controller does this in the background, you can continue working on your time-line.
    In the case of parity raids (3/5/6) one has to make a distinction between distributed parity raids (5/6) and dedicated parity raid (3).
    Dedicated parity, raid3
    If a disk fails, the data can be rebuild by reading all remaining disks (all but the failed one) and writing the rebuilt data only to the newly replaced disk. So writing to a single disk is enough to rebuild the array. There are actually two possibilities that can impact the rebuild of a degraded array. If the dedicated parity drive failed, the rebuilding process is a matter of recalculating the parity info (relatively easy) by reading all remaining data and writing the parity to the new dedicated disk. If a data disk failed, then the data need to be rebuild, based on the remaining data and the parity and this is the most time-consuming part of rebuilding a degraded array.
    Distributed parity, raid5 or raid6
    If a disk fails, the data can be rebuild by reading all remaining disks (all but the failed one), rebuilding the data and recalculating the parity information and writing the data and parity information to the failed disk. This is always time-consuming.
    The impact of 'hot-spares' and other considerations
    When an array is protected by a hot spare, if a disk drive in that array fails the hot spare is automatically incorporated into the array and takes over for the failed drive. When an array is not protected by a hot spare, if a disk drive in that array fails, remove and replace the failed disk drive. The controller detects the new disk drive and begins to rebuild the array.
    If you have hot-swappable drive bays, you do not need to shut down the PC, you can simply slide out the failed drive and replace it with a new disk. Remember, when a drive has failed and the raid is running in 'degraded' mode, there is no further protection against data loss, so it is imperative that you replace the failed disk at the earliest moment and rebuild the array to a 'healthy' state.
    Rebuilding a 'degraded' array can be done automatically or manually, depending on the controller in use and often you can set the priority of the rebuilding process higher or lower, depending on the need to continue regular work versus the speed required to repair the array to its 'healthy' status.
    What are the performance gains to be expected from a raid and how long will a rebuild take?
    The  most important column in the table below is the sustained transfer  rate. It is indicative and no guarantee that your raid will achieve  exactly the same results. That depends on the controller, the on-board  cache and the disks in use. The more tracks you use in your editing, the higher the resolution you use, the more complex your codec, the more  you will need a high sustained transfer rate and that means more disks in the array.
    Sidebar: While testing a  new time-line for the PPBM6 benchmark, using a large variety of source  material, including RED and EPIC 4K, 4:2:2 MXF, XDCAM HD and the like,  the required sustained transfer rate for simple playback of a  pre-rendered time-line was already over 300 MB/s, even with 1/4  resolution playback, because of the 4 4 4 4 full quality deBayering of  the 4K material.
    Final thoughts
    With the increasing popularity of file based formats, the importance of backups of your media can not be stressed enough. In the past one always had the original tape if disaster stroke, but no longer. You need regular backups of your media and projects.  With single disks and (R)aid0 you take risks of complete data loss, because of the lack of redundancy.  Backups cost extra disks and extra time to create and restore in case of disk failure.
    The need for backups in case of mirrored raids is far less, since there is complete redundancy. Sure, mirrored raids require double the number of disks but you save on the number of backup disks and you save time to create and restore backups.
    In the case of parity raids, the need for backups is more than with mirrored arrays, but less than with single disks or striped arrays and in the case of 'hot-spares' the need for backups is further reduced. Initially, a parity array may look like a costly endeavor. The raid controller and the number of disks make it expensive, but if you consider what you get, more speed, more storage space, easier administration, less backups required, less time for those backups, continued working in case of a drive failure, even though somewhat sluggish, the cost is often worth more with the peace-of-mind it brings, than continuing with single disks or striped arrays.

    Raid3 is better suited for video editing work, because it is more efficient when using large files, as clips usually are. Raid5 is better suited in high I/O environments, where lots of small files need to be accessed all the time, like news sites, webshops and the like. Raid3 will usually have a better rebuild time than raid5.
    But, and there is always a but, raid3 requires an Areca controller. LSI and other controller brands do not support raid3. And Areca is not exactly cheap...
    Keep in mind that a single disk shows declining performance when the fill rate increases. See the example below:
    A Raid3 or Raid30 will not show that behavior. The performance remains nearly constant even if fill rates go up:
    Note that both charts were created with Samsung Spinpoint F1 disks, an older and slower generation of disks and with an older generation Areca ARC-1680iX-12.

  • RAID performance is poor on the K8N NEO NF4 SLi mobo...why?

    I reccently downloaded and purchased pcmark05 to keep tabs on my system performance, I have 2 wd800jb/se ide in raid'0', isn't there supposed to be a performance gain with using raid?, is it a hardware or software version of raid that is used with my motherboard?, reason I ask is because I took a 50% performance hit configuring these 2 hd in raid as opposed to just using them straight up without raid, is it the nvidia raid controller or raid/ide drivers screwing  up and making this so?, hell I don't know, I spent a couple of hours trying to find some specific information on this, I have to admit my knoweldge is very limited in the hd aspect, I don't really know the differences between sata, and ide other then they use different apetures, and sata is a newer more current technology, I know how to oc, set up ram or configure a system, but like I said I honestly don't know much about hard drives or what is fater or how to optimally set them up, I still manage to score 4400 on pcmark05 which is pretty good when the only lagging aspect is the hard drive performance, people who are scoring  half of what I'am still are getting twice as high marks in the hard drive test.
      In fact I failed to find one score out of a couple of hundred that I came anywhere close to their marks, it is a shame cause otherwise I have a very tight system, just so you know I'am being realistic here, I don't exspect to get better performane the a w740 raptor, does anyone have any ideas or suggestions.....if not where would be a good site on the subject...Thanks
    none raid bench result with one wd800jb   http://service.futuremark.com/compare?pcm05=198218
    raid'o' 2 wd800jb bench result http://service.futuremark.com/compare?pcm05=196491  note there is an 800 point difference, I hought raid was supposed to enhance performance somewhat

    Well, I need to update my sig, I have only one 7800gt, sli hasn't always lived up to it's potential, so I would think I'am within the power requirements now/then, 460 should be more then enough, as far as my overclock goes other then some random crashes when I was FIRST setting up the oc, which btw is normal when you are finding the acceptable limits of the cpu/memory and then backing down into a good stable oc, would a 45% overclock really cause this poor performance in RAID?, cause honestly, my system is rock solid stable at it's current settings, cpu is 33 at idle and 40-42 under load, the ram is beyond specs but like I said it has been put thru every stress test I could get my hands on here are some everest reports if it helps to give you an idea......         

  • Raid performance. What do you get?

    Hi All,
    I'm setting up an XSan using 4 14*500Gigs XRaids + 1 XRaid for the metadata. Before installing XSan I'm testing the speed of the system when it's striped as a raid 0. We're using atto dual 4Gigabyte cards in the computers, together with a QLogic switch. Using 4 Raid5 slices, and striping them togethere as raid 0, I get speeds of about 250 Mbyte/s read, 300 Mbyte/s write using Decklinks speeddisk utility. What speeds do you get?
    I've also posted this in the XSan forum, I guess it belongs in both.

    Hi Wout,
    I have a 180x14 xserve raid with 512mb of cache. I have a PCI-X card and I seem to max out at 185mb/s write and around 235mb's read. This is on a Dual 2ghz G5. BTW performance seems pretty flat going from PCI to PCI-X and upgrading the caches from 128mb. Seems like the speed of the disks and controllers is the determining factor.
    I'm experimenting with a 2nd xserve raid and I'll report the performance soon.
    Cheers,
    Sam

  • Raid performance deteriorates, looses drive, but no messages in event log

    We have a PATA XServer Raid running, which was fine until a few weeks ago. Suddenly performance tanked, writing a 70 MB file via a 100 Mbps network would take an hour.
    The Raid volume, a 6 drive 250 GB raid with one hotspare, had 100 GB left.
    First we thought it was the network, there was a loose DHCP server, but nothing was available that could really explain the performance deterioration. The only thing that was noticed was that with Helios Lan test the first write to the raid would take more time as the subsequent writes. So we decided to upgrade the firmware and do a volume test, with diskutility. That gave a interleaf problem.
    After upgrading the network (which was planned anyway) to 1 GBps, we rebooted the Xserve and the Raid. Now, the 6th drive had an orange status in Raid Admin, a warning, saying it was part of an Unknown Array. The 7th drive, the hot-swap, was now part of the Raid.
    The strange thing is that in the event log there are no messages what so ever about a failed drive, nor any other messages relating to this. All I get is Coprocessor offline/Online, Raid Controller Restarted and Riad Controller 1 Fibre LIP, and Firmwareupdate
    I also noticed that the time was off to local time, so I synchronized both to time.euro.apple.com.
    Shouldn't we be getting messages something is very wrong in the event log?
    XServe G5 Xserve Raid   Mac OS X (10.3.9)  

    We updated the firmware to 1.5.1. After that, the network switch was replaced by a 1Gbps switch, and the XServe and the XServe raid were rebooted. After that one of the people discovered that disk 6 had an orange led burning.
    XServe G5, XServe Raid   Mac OS X Server (10.3.9)   Raid Firmware 1.5.1

  • KT3ULTRA ARU RAID performance in PCmark2002

    I have configured a new RAID array : 2 Maxtor 30GB ATA133 disks in striping on IDE3 and IDE4. I also have a Seagate 20GB ATA-66 on IDE1. In PC Mark 2002 my harddisk score used to be 450 with my old array (2 Quantum lct10 10GB ATA-66 disks). Now I get exactly the same scores with my new disks.
    My guess is that PCmark2002 only tests the Seagate disk on IDE1. Is there a way to change the settings of PCMark2002 so that it actually tests the RAID array?
    Does anyone have a better benchmark to measure harddisk performance?
    Greetz,
    Z
    http://users.pandora.be/zwendel

    hi,
    u can try this guide.

  • Hows my raid performance?

    im using 2 wd800jb drives (80 gigs) in a raid 0
    im using the lite raid which only allows 64kb blocks
    a kt3 ultra2 board
    a 2000+ running at 166 and 10.5 multiplier
    512 corsair xms3200c2 ram (default hidden settings and 2-2-5-2-4-4 1t enabled normal)
    pci latency is at 64
    winxp sp1
    my sandra benchmarks are around 44000
    pcmark 2002 hd bench is : 1545
    should i use the modded bios and change the block size?
    i tried latency patches and all slowed down benches
    any other recommendations for improving performance?
    thanks

    Hi!
    I looked pretty much on that and 16 cluster and 16 stripe  sizemis the best fo your drives.....
    I don't know if you can choose that with the lite bios but because I have the modded bios and it works great...
    You'll have to reformat if you change stripe size....
    hope it helps!

  • K7N2 Delta SATA Raid Performance

    I just installed this motherboard and was wondering what kind of performance improvement I might see using sata converters on my almost new WD 80 GB ATA 100 HDDs? Taking advantage on board of the serial ata raid controler.  The converters are only $20 and 2 new sata HDDs would be $260.  Also if I use the Serial ata for my HDD wich IDE plug do I use for my optical drives?   Any Help would be apperciated.

    what they say as regards a single drive is true,but you would still see the benefits of running in raid and they are quite high,im still using my highpoint controlller on my wd drives,but i would not go back to single drives from raid 0 now ,it just makes the pc more snappy,some of the adaptors like the abit ones have got a bad reputation though,seems they are prone to falling out
    as i recall on a clean install my c in sandra was benching around 54000

  • RAID PERFORMANCE

    I'm running a 4x2.5GHz PowerPC G5, with 4GB DDR2 SDRAM. I have a Black Magic card in the back, and a 5.5TB Fibre Channel RAID attached. However despite the predicted speed and power of this combination I am getting dropped frames either about every minute or at every edit point or transition. This is making life very difficult, as I can't playback any edits in full for clients. Can anyone please advise.

    As mentioned above, make sure the 'Allow Host Cache Flushing' option is turned off for both RAIDs. Also there's a bug in the 1.5 firmwware, where this setting does not survive a re-boot, so each time you power cycle a RAID, you'll probably have to go back in and re-set it.
    I'm running a 4x2.5GHz PowerPC G5, with 4GB DDR2
    SDRAM. I have a Black Magic card in the back, and a
    5.5TB Fibre Channel RAID attached. However despite
    the predicted speed and power of this combination I
    am getting dropped frames either about every minute
    or at every edit point or transition. This is making
    life very difficult, as I can't playback any edits in
    full for clients. Can anyone please advise.
    MacBook Pro   Mac OS X (10.4.6)  

  • Dell PERC H710P and Samsung SSD 840 Pro RAID performance

    Hi. I have some questions about Dell PERC H710P.
    I have a Dell PERC H710P RAID controller and 8 Samsung SSD 840 Pros.
    I created virtual drive on 8x256GB Samsung SSD 840 Pro in RAID10.
    And tried DD test (bs=512kb, count=20000) in CentOS 6.3.
    But I had just around 370MB/sec.
    Is there any problem of my test?
    Please reply.
    Thanks.

    ive also had trouble with the Samsung 840 pro (256 GB) and my Dell Perc card. i have the smaller one the H310 though.
    sent the drive back to samsung (their tech sup is horrid) and got it back. still an issue.
    i think that it just a Dell mobo issue. we are waiting on ours to be replaced.

  • P35 Neo2/FR raid performance ?

    I'm trying to find some benchmarks of a 2 drive raid setup  vs a single drive on this motherboard.
    Can anybody point me in the right direction. Also does anybody know if it's actually worth doing raid on the board with 2 drives?
    Thanks

    The operating system has to switch the driver when you change the controllers operating mode.  The Generic Drivers that come with Vista are not the best.  The Intel ICH9R RAID Drivers cannot be installed until the controller is switched to RAID mode as until then there is nothing they can work with.
    The Best and cleanest way to solve this problem is to reinstall Vista when the controller is already operating in RAID mode and integrate the Intel ICH9R SATA RAID Drivers during the Vista Setup Routine.  You can also try a Vista Repair Installation. 

  • External SATA raid performance

    Can anybody tell me what a good average transfer/write speed should be from a Win 7 64 PC to an eSATA RAID0 device that shows 185 MB/S write speed in Blackmagic's speed test.
    When I transfer 130 GB of XDcam data across to a 1TB G-Raid device that has no previous data on it, the speed stars out at about 120 MB/s and gradually slows until it maintains 20-24 MB/s for the majority of the transfer.
    Should I be expecting higher transfer rates than that?
    Thanks.

    Jon, that is lousy, I do not know your RAID configuration, but one good internal drive would do much better than that 20-24 MB/sec!  Here are my Read and Write benchmarks on a single SATA3 Seagate 7200.14 2 TB drive with HD Tune Pro (all with my Areca 1880).  Sorry I do not have the Blackmagic test while a good end to end test it does not tell you if it is a read or write problem
    Here are four of them in SATA3 RAID 10

  • GE70 2OE SSD raid performance

    hi guys,
    since about a week my ssd's became horribly slow.
    also when i boot my system it will take about 2 mins. for all applications like nvidia GE are loaded.
    it only does this since a week, does anyone of you has a solution for this?
    greetings Thom

    Quote from: haarent on 25-December-14, 19:46:44
    i also noticed this sign at my toolbar:
    normally it shows a checkmark instead of a blue loading circle.
    That's Intel RST, Automatic (delayed start) - perhaps it is causing that slowdown on writes. It should turn to checkmark after a while. Or you could launch that service manually to see if the writes do speed up.

  • XServe RAID RAID5 performance?

    Hello,
    I would like to know how the XServe RAID performs in RAID5.
    We are using multiple FC and SCSI RAID-subsystems at work and we need a bigger one by the end of the year.
    TIA
    Best
    Nicolas

    The Xserve RAID is specifically tuned for RAID 5 performance -- RAID 5 is about 99% as fast as RAID 0. Unless you were hosting a database on the RAID, and doing lots of small I/Os, there is no reason to use any other RAID scheme on Xserve RAID
    There's a performance page here:
    http://www.apple.com/xserve/raid/performance.html
    With some info... not very detailed. If you can get ahold of an eval unit (Apple's direct sales force has a program for this), you can run Iozone benchmarks on one, and see how it might fit your usage pattern.

  • ZFS on HW RAID with poor performance. Ideas?

    Hi all,
    I'm trying to figure out why my RAID performs so poorly using ZFS compared to even UFS. Any thoughts on the following would be helpful.
    H/W Config:_
    - SunFire v245 w/ 4GB RAM and LSI1064 mirrored 73GB SAS
    - QLogic QLE2462
    - Fibre Channel 4Gb Hardware RAID w/ 1GB ECC cache
    - 16 x 1TB SATA II drives
    - Configured 1 volume RAID-6
    S/W Config:_
    - Solaris 08/07 SPARC - 5.10 Generic_120011-14
    - Latest 10_Recommended as of 1/08
    - zpool create fileserv /dev/dsk/c3t0d0s6
    - zfs set atime=off fileserv
    Test Raw:_
    To the "RAW" disk or volume I get descent numbers:
    # dd if=/dev/zero of=/dev/rdsk/c3t0d0s6 bs=1048576 count=16384
    # dd if=/dev/rdsk/c3t0d0s6 of=/dev/null bs=1048576 count=16384
    Both c3t0d0 and c3t0d0s6 result in similar numbers:
    5 x 16GB writes took 475.50 seconds, yielding an average of *172.28 MB/s*.
    5 x 16GB reads took 384.68 seconds, yielding an average of *212.97 MB/s*.
    Test ZFS:_
    Creating and mounting ZFS to the same volume, I modified my scripts to use dd created files from /dev/null instead:
    # dd if=/dev/zero of=./16G bs=16777216 count=4096
    Then ran:
    # dd if=/dev/zero of=./16G bs=1048576 count=16384
    # dd if=./16G of=/dev/null bs=1048576
    The result for ZFS on a 14.6TB volume is horrible:
    5 x 16GB writes took 1013.28 seconds, yielding an average of *80.84 MB/s*.
    5 x 16GB reads took 980.36 seconds, yielding an average of *83.56 MB/s*.
    What would cause a delta difference of more than half my capable speed from the unit just by running ZFS? Thinking it might just be a dd issue, I ran iozone using the same parameters as "milek's blog" at http://milek.blogspot.com and got the same crappy results:
    # iozone -ReM -b iozone-2G.wks -r 128k -s 2g -t 32 -F /test/*
    (snippet:)
    Output is in Kbytes/sec
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 1024 Kbytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.
    Throughput test with 32 processes
    Each process writes a 2097152 Kbyte file in 128 Kbyte records
    Children see throughput for 32 initial writers = 90584.21 KB/sec
    Parent sees throughput for 32 initial writers = 79823.98 KB/sec
    Min throughput per process = 2186.76 KB/sec
    Max throughput per process = 3225.63 KB/sec
    Avg throughput per process = 2830.76 KB/sec
    Min xfer = 1427456.00 KB
    Children see throughput for 32 rewriters = 88526.23 KB/sec
    Parent sees throughput for 32 rewriters = 88380.95 KB/sec
    Min throughput per process = 2264.00 KB/sec
    Max throughput per process = 3152.58 KB/sec
    Avg throughput per process = 2766.44 KB/sec
    Min xfer = 1510656.00 KB
    I can understand that ZFS wants to have the entire "disk" as in a JBOD, but that's just not a viable option in my Corporate America world. Does presenting a RAID Controller handled LUN Vs. a JBOD presented matrix of drives really make this much of a difference? I'm just at a loss here as to why a unit that is capable of 4Gbit per channel is working as it should via raw, but runs like a single disk in performance as a ZFS filesystem.
    I would appreciate some assistance in figuring this one out.
    Thanks to all in advance.
    --Tom                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       

    Robert,
    Thanks for the help. Unfortunately, using RAID-Z or Z2 is not an option here as I'd rather rely on my hardware RAID over using software. While doing testing, I set my RAID to pass-thru mode and allowed ZFS to inherit the RAW drives from the RAID Chassis, set up a pool, and did my testing. In all cases, write performance was not much better overall, CPU utilization with compression on was ~30% higher, and my RAM utilization shot up under load. I just don't see that as an option on a heavily hit CVS server where IOPS and CPU/RAM are important for compilations and the like.
    Things Hardware RAID gives me that ZFS can't do are:
    Ease of use - changing failed drives for example is just different. It's a complicated filesystem once you get into it.
    Pre-write buffer performance - This is ZFS's bottleneck afterall.
    Fastest I/O - RAID Controllers handle more drives better than ZFS and IOPS are much faster under a H/W RAIDThere's a whole line of reasons why I choose not to use ZFS RAID-Z/Z2. Why else would Sun continue selling both H/W RAID and JBOD? I will check into the Intent Logging though, that's a good idea. I appreciate the link to solarisinternals, I had not thought about using their wiki for this.
    Thanks again!
    --Tom                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       

Maybe you are looking for