Comment by MadameMinty
17 hours ago
Then check out those:
https://github.com/prabanta-dev/janas <- this, focuses on Qwen3-Next-80B-A3B.
https://github.com/Niko1221/Strata <- Qwen3.8-Flash-Next. I use this (tuned) and I cannot understate how satisfied I am.
https://github.com/antirez/ds4 <- DeepSeek 4 family + more.
s/understate/overstate/ I mean, I get up to 90 tok/s on consumer hardware (16 GB VRAM + 64 GB RAM) using IQ3_S, which so far is holding up quite well.