← Back to context

Comment by girvo

20 hours ago

DeepSeek v4 Flash, various quantised versions of Kimi K2.6, MiniMax 2.7, Qwen 3.5 “full sized, with a dual spark setup you can fit some decent setups on here

My single spark has me running Qwen 3.6 27B and antirez’s specially quantised DeepSeek v4 Flash (which is shockingly impressive)

3 comments

girvo

Reply
Tepix  15 hours ago

Kimi K2.6 does not run well on 256GB.

  • zozbot234  9 hours ago

    Have you tried it? It would be slow for sure, but the main limitation AIUI would actually be storing the context in RAM - models like Kimi and GLM have high demands there which limit your ability to get meaningful aggregate throughput via large batches.

  • girvo  11 hours ago

    True, I might be thinking of some of the communities four-Spark clusters for it; it’s already int4 right?