Comment by johndough
8 hours ago
> But I think it's going to need more than 1536GB of RAM, with a usable and large amount of context, more like 2TB and preferably 2.5 to 3TB.
The model is known to be MXFP4 according to Kimi's release blog post, so the model weights will be less than 1536GB: https://www.kimi.com/blog/kimi-k3
Also, their previous models were native INT4, so it would be weird if they went larger now.
Update: Looks like the model is larger after all (1561.44 GB). Only the MoE weights are MXFP4, while the other weights are BF16 (and a few FP32).
* Sparse Experts: 1481.4 GB
* Dense Experts: 1.9 GB
* Self-Attention: 72.4 GB
* LLM Head: 2.4 GB
* Embeddings: 2.4 GB
* Vision Encoder: 0.35 GB (surprisingly small)
plus some miscellaneous parameters.
Most importantly, we now know that the model has 104B active parameters, which is quite a lot and will make it difficult to self-host efficiently.