Comment by __jal
6 years ago
Sounds like you turned on dedupe, or had an absurdly wide stripe size. You do need to match your array structure to your needs as well as tune ZFS.
On our backup servers (45 disks, 6-wide Z2 stripes) easily handle wire-speed 10G with 32G ARC.
And you're just wrong about snapshots and filesystem counts.
ZFS is no speed demon, but it performs just fine if you set it up correctly and tune it.
Stripe size could have been a problem, though we just went with the default there afair. Most of the first tries was just along the Sun docs, we later only changed things until performance was sufficient. Dedupe wasn't even implemented back then.
Maybe you also don't see as massive an impact because your hardware is a lot faster. X4200s were predominantly meant to be cheap, not fast. No cache, insufficient RAM, slow controllers, etc.
X4200s were the devil's work. Terrible BMC, raid controller, even the disk caddies were poorly designed.
The BMC controller couldn't speak to the disk controller so you had no out-of-band storage management.
I had to Run a fleet of 300 of them, truly an awful time.