← Back to context

Comment by zamadatix

1 hour ago

I just built a 4th gen Epyc build with 22x NVMe drives and a dual port 40G NIC. It was a FAR superior experience to trying to use prosumer parts, PCIe splitters, etc and didn't end up costing as much as I thought it would (though the DDR5 RAM has went up 2x in the last month since I bought haha).

Side note on RAID: Motherboard integrated garbage is still meh, but hardware options are also pretty meh. I just use software options like ZFS on Linux (or mdraid if I just need a fat RAID 0 with no protection) and get fantastic speeds, portability, no artificial drive topology restrictions, and no additional latency to accessing the drives over PCIe. On Windows the equivalents would be ReFS Storage Pools or whatever the software RAID 0 in the disk manager was called.

If you're looking for cheaper and "not the latest gen" the search tugm4770 on Ebay and filter to the CPU you want. They have MB bundles for standard EATX chassis or bundles with used supermicro servers+PSUs as well. I went the EATX route and put it in a standard PC case, it's crazy how quiet it runs (it's in my living room even)! I used to buy from this seller for the lab at my previous job, but decided to go all in at home this time. Never had trouble with them and it is by far the cheapest way to get these kinds of setups (just avoid the really old Epyc generations).

If the sky is truly the limit and you want the absolute best of the best, get a current gen high frequency optimized Epyc new (don't know the easiest place to do that). You can also do the Threadripper PRO, just depends.

Both of these Epyc options have a "how sky high?" option of getting a dual socket MB, which can give you up to 192 PCIe lanes instead of up to 128. You have to populate both sockets, and the physical footprint starts to become monstrous (especially if you use all the lanes).