Filesystems: Media storage

Question

Hey guys, I've got a new 1TB drive coming in the next few days, and I was planning on using it for storing large media files (mostly videos). In the past I've used NTFS for storing such files, but since I never use Windows for anything other than playing games nowadays, booting into Windows to periodically defrag the storage partition has become an unbearable chore.
So, in your opinion, what is the best Linux filesystem for storing large files? If you have your own dedicated storage partition, are there any specific mount options you use with it to help prevent against data corruption (caused by unclean shutdowns or whatever)? Is it worth using LVM on the disk, and maybe freeing up one of my other terabyte drives, putting LVM on that, and having a RAID0? Is there any significant processor hit when using RAID0?
Also, as a side topic, MBR or GPT as the partition table? (It's kinda irrelevant, but never mind).

Answer 1

I've had various arrays over the years, mostly using md + xfs.  It works pretty good, as long as the drives and cables are stable.  It does not, however, protect against bit rot or bad cables.  I've had several bad SATA cables that were tough to track down.
My first 8 x 2TB array had nothing but problems.  I would get it set up and transfer data to it, thinking it was going to be great.  It was months before I realized a high percentage of my files had become corrupted.  That's scary.
zfs-fuse solved these problems.  It allowed me to discover drives with bad sectors and the bad cables.  With zfs-fuse, it will even tell you what drive is experiencing the difficulty.  Pretty sweet.
I'm running WD20EARS drives.  I know the mantra is not to use them in an array but expensive drives would not have been any more reliable in the face of the bad cable problem and I can't imagine they would be much better on the bad sector issue, either.  The point is, zfs-fuse weathered these problems, told me what was wrong... right down to the serial number of the drive having issues, and repaired the data once the problem was corrected.
zfs-fuse is slow and not appropriate for boot drives.  I wouldn't run it on a root drive, either.  It's pretty much bullet proof on reliability though, so I really like it on arrays that hold bulk data.  For uses like ZoneMinder and MythTV recording, zfs-fuse is fast enough... barely.
By the way, it got about 50% faster in version 0.7.  I'm getting about 40 MBps on an older dual core 2.5 GHz athlon.
Some have reported 100 MBps.  That could well be possible with smoking fast drives in mirror mode but my fastest quad core server can't break 45 MBps on RAIDZ1 arrays.
... just my 2 cents. 
Last edited by TomB17 (2011-09-06 04:58:17)