← Back to context Comment by bigyabai 8 hours ago Not even a cluster of Mac Pros could run a dense 5T parameter model with RDMA, to my knowledge. 1 comment bigyabai Reply zozbot234 7 hours ago SOTA models are reportedly MoE, not dense.
SOTA models are reportedly MoE, not dense.