Comment by chithanh

7 months ago

KTransformers now supports Kimi K2 for MoE offloading

They claim 14 tps for the 4-bit quant on a single socket system with 600 GB RAM and 14 GB GPU memory.