Home server limitations

Question

Heyo,
tl;dr at bottom
I currently have a home server which has been running well however my storage requirements keep going up and have reached a point where a change is needed.
my system has a nonRAID HBA with connectivity for up to 16 drives.
I knew my storage requirements would grow but i have not anticipated the speed at which it has.
atm my system boots from an ssd, runs samba, ssh, sql, vpn, multiple simple nginx sites,
the storage setup atm is dm-raid with 7 drives in RAID5 with a drive soon to be added to provide RAID6.
I have ext4 on luks however i have come to a point where i cannot expand the array past 16TB, which i thought was no longer a problem with ext4
Google searches however showed me that it must be created above 16TB initially and that i cannot resize past 16TB.
i chose ext4 as i was expected to convert to btrfs once it became viable.
my questions now are.
where should i go from here. im currently stuck in a 15TB filesystem quickly running out of space.
its important for me to have a single volume of data, with data protection(hd failure resolution+encryption) and becoming a concern in the long term is data integrity.
With the choice to convert over to btrfs i have another question in my head regarding raid/encryption. as i understand btrfs has raid1/0 support and potentially raid5+ in the future,
i currently like the filesystem ontop of luks ontop of dmraid, it makes sense to me.
if i change to btrfs does this layering choice make sense?
can someone recommend a more appropriate choice?
in the not to distant future my storage requirements may go up to around 30TB+ easily.
as i see it i can do the following.
1. keep dm-raid+luks and replace ext4 with btrfs and grow volume(via convert filesystem) - (ok for meantime but an array of may disks like this doesnt sound ideal)
2. change to a different filesystem that supports multi-device large volumes with the features requested.
3. split up the storage into multiple smaller volumes(least preferred)
advice would be very much appreciated,
tl;dr HALP! many disk RAID array wont grow above 16TB(ext4) and need a solution for future.

Answer 1

I think that none of the setups both of you describe is really hacky. Layering several filesystems and whatnot on top of each other works fine and people do it all the time. I am, however, a bit critical of both approaches. (Particularly as a fan of ZFS, so consider me biased.)
1. If seiichiro0185 uses a limited number of 4 disks as raid-z inside a NAS, then this is a perfectly fine solution. But with that setup he won't ever experience that large raid-z arrays with the wrong number of disks (for more info see here: http://www.solarisinternals.com/wiki/in … ces_Guide) or even several raid-z arrays grouped together (even more info here: http://constantin.glez.de/blog/2010/06/ … rformance) will give you performance hits.
2. If smelly has 7 disks he wants to keep (and planning to add more over time, if I understood correctly), I would recommend what I wrote above: Creating a striped ZFS pool from sets of mirrors, similar to RAID-10, by using zpool add and zpool attach. This will give you less space than RAID-Z (granted) and maybe slower r/w speeds (don't know enough about that), but your CPU will have to do less work and you will be more flexible in adding/removing disks. Of course, you need an even number of disks for that, so 6 for now. creating mirrors of same-size (or similar size) disks and glueing them together to make one pool. You can add more mirrors to extend the pool over time, or, if one disk in a mirror fails you can replace it with a bigger one. As soon as you replace the other disk, ZFS will automatically grow the pool.
And keep in mind that setting up ZFS on top of anything else but physical disks (and most definitely placing ZFS anywhere above an mdadm layer) will practically disable ZFS's error correction magic!
I could go on and on about this but I'll stop here. Hope you don't mind the pamphlet.