Comment by mips_avatar
13 hours ago
The more important question than subsidy is what is the tokenomics of running the model. If it's inefficient to run on an nvl72 cluster (or whatever the heck has enough vram to run a 3T parameter model), and k3 isn't very token efficient, then it might not be that compelling of an open weights model.
3T at nxfp4 (which is most of it) is only 1.5TB of vram - so 8x288GB B300 or MI355 will do it if you are careful with context - maybe dp-attn? Certainly not TP. 2 of those together can easily serve it. The new AMD MI400 are at 400GB+ each, so 8x of them will nicely fit with KV to spare.