I use Qwen 3 Coder Next daily on my mac as my main coding agent. It is incredibly capable and its strange how you are painting this picture as if its a fringe use case, there are whole communities that have popped up around running local models.
Can I doubt your claim? I have had such terrible luck with AI coding on <400B models. Not to mention, I imagine your codebase is tiny. Or you are working for some company that isnt keeping track of your productivity.
I am trying super hard to use cheap models, and outside SOTA models, they have been more trouble than they are worth.
Absolutely. So my codebase is huge, it's a monolith. But my work is in very specific parts of the codebase, I don't pull the entire code base into context (and I don't think that is common practice even with claude) - I start at a specific point with a specific task and work with the agent to achieve something clearly defined, for example writing tests, extracting things into separate files, refactoring or even scaffolding a new feature. You have to periodically start new threads, because you'll start hitting the limits of the context, but I max it out at over 200k because I have the memory overhead on my 128gb mbp to do that, so I can get quite a lot done.
I really recommend trying the Qwen models - 3 coder next is really incredible. GLM 4.7 flash is also incredibly performant on modest hardware. Important things to consider is setting the temperature and top_p and top_k values etc based on what is recommended by the provider of the model - a thing as simple as that could result in a huge difference in performance.
The other big leap for me was switching to Zed editor and getting its agent stuff just seamlessly integrated. If you run LM Studio on your local machine it's super easy and even setting it up on a remote machine and calling out to LM Studio is dead simple.
Yesterday, I got Qwen-Coder-Next to build a python script that reads a Postman collection, pulls the data from it to build a request to one of the endpoints, download a specific group of files whose URLs were buried in the JSON payload in that endpoint, then transform then all to a specific size of PNG, all without breaking a sweat. I didn't even have to tell it to use Pillow, but it did everything to a T.
Use case means everything. I doubt this model would fare well on a large codebase, but this thing is incredible.
Yesterday I test ran Qwen3.5-35B-A3B on my MBP M3 Pro with 36GB via LM Studio and OpenCode. I didn’t have it write code but instead use Rodney (thanks for making it btw!) to take screenshots and write documentation using them. Overall I was pretty impressed at how well it handled the harness and completed the task locally. In the past I would’ve had Haiku do this, but I might switch to doing it locally from now on.
I suppose this shows my laziness because I'm sure you have written extensively about it, but what orchestrator (like opencode) do you use with local models?
I managed to get qwen2.5-coder:14B working under ollama on an Nvidia 2080 Ti with 11GB of VRAM, using ollama cli, outputting what looks like 200 words-per-minute to my eye
It has been useful for education ("What does this Elixir code do? <Paste file> ..... <general explanation> "then What this line mean?")
as well as getting a few basic tests written when I'm unfamiliar with the syntax. ("In Elixir Phoenix, given <subject under test, paste entire module file> and <test helper module, paste entire file> and <existing tests, pasted in, used both for context and as examples> , what is one additional test you would write?")
This is useful in that I get a single test I can review, run, paste in, and I'm not using any quota. Generally I have to fix it, but that's just a matter of reading the actual test and throwing the test failure output to the LLM to propose a fix. Some human judgement is required but once I got going adding a test took 10 minutes despite being relatively unfamiliar with Elixir Phoenix .
It's a nice loop, I'm in the loop, and I'm learning Elixir and contributing a useful feature that has tests.
I think this is directing coders towards self-sufficiency and that's a good thing. If they don't end up using it for agentic coding, they can use it for running tests, builds, non-agentic voice controlled coding, video creation, running kubernetes, or agent orchestration. So no, it's not evil, even if it doesn't go quite as expected.
I use Qwen 3 Coder Next daily on my mac as my main coding agent. It is incredibly capable and its strange how you are painting this picture as if its a fringe use case, there are whole communities that have popped up around running local models.
Can I doubt your claim? I have had such terrible luck with AI coding on <400B models. Not to mention, I imagine your codebase is tiny. Or you are working for some company that isnt keeping track of your productivity.
I am trying super hard to use cheap models, and outside SOTA models, they have been more trouble than they are worth.
Absolutely. So my codebase is huge, it's a monolith. But my work is in very specific parts of the codebase, I don't pull the entire code base into context (and I don't think that is common practice even with claude) - I start at a specific point with a specific task and work with the agent to achieve something clearly defined, for example writing tests, extracting things into separate files, refactoring or even scaffolding a new feature. You have to periodically start new threads, because you'll start hitting the limits of the context, but I max it out at over 200k because I have the memory overhead on my 128gb mbp to do that, so I can get quite a lot done.
I really recommend trying the Qwen models - 3 coder next is really incredible. GLM 4.7 flash is also incredibly performant on modest hardware. Important things to consider is setting the temperature and top_p and top_k values etc based on what is recommended by the provider of the model - a thing as simple as that could result in a huge difference in performance.
The other big leap for me was switching to Zed editor and getting its agent stuff just seamlessly integrated. If you run LM Studio on your local machine it's super easy and even setting it up on a remote machine and calling out to LM Studio is dead simple.
Yesterday, I got Qwen-Coder-Next to build a python script that reads a Postman collection, pulls the data from it to build a request to one of the endpoints, download a specific group of files whose URLs were buried in the JSON payload in that endpoint, then transform then all to a specific size of PNG, all without breaking a sweat. I didn't even have to tell it to use Pillow, but it did everything to a T.
Use case means everything. I doubt this model would fare well on a large codebase, but this thing is incredible.
The thing I'm most excited about is the moment that I run a model on my 64GB M2 that can usefully drive a coding agent harness.
Maybe Qwen3.5-35B-A3B is that model? This comment reports good results: https://news.ycombinator.com/item?id=47249343#47249782
I need to put that through its paces.
Yesterday I test ran Qwen3.5-35B-A3B on my MBP M3 Pro with 36GB via LM Studio and OpenCode. I didn’t have it write code but instead use Rodney (thanks for making it btw!) to take screenshots and write documentation using them. Overall I was pretty impressed at how well it handled the harness and completed the task locally. In the past I would’ve had Haiku do this, but I might switch to doing it locally from now on.
I suppose this shows my laziness because I'm sure you have written extensively about it, but what orchestrator (like opencode) do you use with local models?
I've not really settled on one yet. I've tried OpenCode and Codex CLI, but I know I should give Pi a proper go.
So far none of them have be useful enough at first glance with a local model for me to stick with them and dig in further.
4 replies →
I managed to get qwen2.5-coder:14B working under ollama on an Nvidia 2080 Ti with 11GB of VRAM, using ollama cli, outputting what looks like 200 words-per-minute to my eye
It has been useful for education ("What does this Elixir code do? <Paste file> ..... <general explanation> "then What this line mean?")
as well as getting a few basic tests written when I'm unfamiliar with the syntax. ("In Elixir Phoenix, given <subject under test, paste entire module file> and <test helper module, paste entire file> and <existing tests, pasted in, used both for context and as examples> , what is one additional test you would write?")
This is useful in that I get a single test I can review, run, paste in, and I'm not using any quota. Generally I have to fix it, but that's just a matter of reading the actual test and throwing the test failure output to the LLM to propose a fix. Some human judgement is required but once I got going adding a test took 10 minutes despite being relatively unfamiliar with Elixir Phoenix .
It's a nice loop, I'm in the loop, and I'm learning Elixir and contributing a useful feature that has tests.
I think this is directing coders towards self-sufficiency and that's a good thing. If they don't end up using it for agentic coding, they can use it for running tests, builds, non-agentic voice controlled coding, video creation, running kubernetes, or agent orchestration. So no, it's not evil, even if it doesn't go quite as expected.