Comment by zozbot234
13 hours ago
Why are these sockets "ruled out"? Pipeline/layer parallelism doesn't need high bandwidth between nodes, and tensor parallelism has middling performance unless you have very fast networking and very slow compute. It all depends on what you're doing.
You are correct that bandwidth requirements depends a lot on the exact workload. And that in specific cases, it might be doable to have AM5 for multiple RTX6000Pro. The parent mentioned workloads that are general, and broader than inference-only. In that case I would consider spending a bit extra on the motherboard to ensure that PCIE bandwidth is not an issue.