Comment by Tsiklon
1 day ago
This looks like a very cool high end x86 workstation. CPU alone is $12000 retail.
Theres a few potential “problems” I see with it - main system RAM is about 30-40x slower than that of the GPU’s and you will have PCI-e as a bottleneck getting working data to or between the GPUs.
In practice is the 400-500GB/s main memory limiting some use cases, or is the PCI-E bus the main issue with these types of systems?
Realistically the models will always be resident in the GPU during processing - the CPU and it's memory would probably more be used for related development tasks (as this is a workstation not a cloud server).
If it's just running the same model all day that CPU and it's RAM would be pretty wasted.
lol $12k is basically the main problem