Comment by ericd

6 hours ago

I honestly wouldn’t bother with local models right now unless I either had a 5090 and was happy with running Qwen 3.8 27B, or a pair of DGX Sparks running DSv4 flash, or better, 2x6000 RTX Blackwells. Those are the kinds of rigs that the local model enthusiasts are running. With the GPU setups, you’re looking at generally >100tps generation in single stream, and >10k tps of prefill, so it’s snappier than Claude code, which somewhat makes up for it being dumber.

That said, it is really cool to be able to run an LLM on eg a Mac laptop. Just not a better experience on almost any metric for interactive use than eg Claude Code, beside privacy and guardrails.

4 comments

ericd

Reply
gruez  6 hours ago

>I honestly wouldn’t bother with local models right now unless I either had a 5090 and was happy with running Qwen 3.8 27B

How's the actual performance of Qwen 3.8 27B? On deepswe it supposedly performs slightly worse than gpt 5.6 luna high[1], but I can't help but think they've been benchmaxxed.

[1] https://deepswe.datacurve.ai/, https://unsloth.ai/docs/models/qwen3.8#benchmarks

  • ericd  6 hours ago

    Not sure, I haven't run it, I've just been running DS V4 Flash non-stop since it came out, and that's replaced a lot of my Claude Code usage. People seem very impressed, though, it seems like it trades vram/world knowledge for extra thinking time, which I think is a good trade for local. tbf, I've heard luna's not great at coding. Fast and good for things like classifiers, summarization, though.

    A friend and I were actually discussing today how benches show Luna Max at about par on coding with Sol Medium, but how it's nowhere near in reality. We were speculating that maybe it's because a lot of benches are best-of-n, and should probably be worst-of-n, because variance in performance is killer with large coding projects. Consistency is what lets you actually build on this stuff.