Comment by thih9

GLM 5.2 awq4 via Opencode, it was better than corporate's fave Sonnet 4.8 and a but worse than Opus 4.8. overall quite capable of a lot of what the dev team needed. Not as good as Sonnet 5 but also never runs our of tokens.

The flip side is that it takes four H200s to run, and that will only let you cache context for maybe three users.

Fingers cross our Blackwells show up and Kimi 3 really releases weights, because at some point devs are spending a significant portion of their salary on tokens and it's somehow cheaper to buy these ridiculous DGX servers and rack and run them.

I don’t know if others would find this useful, but previous did have custom harnesses etc.. but tools have improved so much that I drastically simplified.

That said, even the foundational models fail at the hard parts of my code so I use it opportunistically.

I have reduced down to just using zed, will three locally hosted models.

Qwen 3.6 27b on 1x3090 llama.cpp with 128k context ~50tps

Qwen 3.6 35B-A3B on 1x titan v + 2x1080ti llama.cpp with full context ~30tps

GPT-OSS 120b on pure cpu (slow)

I just use zeds parallel agents, task switching, stopping and fixing the code when a model gets stuck.

This still lets me stay engaged, and to modify code to be maintainable etc…

It gets me 80% there and I use to keep a subscription but often times just using googles AI mode is just as good.

That said I have 30 years of experience and insist on knowing how my code works, so this gets me 80% of the short term benefits while not depending on a 3rd party to keep my code moving forward.

Your mileage will vary and 2*5060ti 16gb cards would get around 100/tps with Qwen 3.6 35B-A3B on cards that are widely available.

To be honest the more modern cloud models are using draft tokens etc… that while they are superior for common coding tasks are degrading with more domain specific tasks.

That is just the cost of the draft model being ~10-20% of the foundation models size, and even the biggest Blackwell GPU is limited to ~250/tps so MoE or draft models are required for scaling performance at the foundational level IMHO.

The hard part is my use case are the OOD or small examples in corpus level, the above hurts there.

A Lamborghini may be nice, but I personally need a minivan more.

I'm using qwen 3.6 35B unsloth 4 bit with my 5950x (128 gb memory) and a 3060 12 gb gpu with a self made harness.

At 10k context I get about 40 tps generation and 500 tps prefill. At 100k context I get about 25 tps generation and 400 tps prefill.

It works, but I often use gpt or claude to make a detailed enumerated plan of what I want to do first, then have qwen follow it.

I'm not sure if it is economical or not, but I have solar on the roof so the power use is not really an issue and I already have the hardware.

The biggest benefit for me is that it's all done locally, and I know the harness is not uploading anything or sending telemetry to someone else.

I'm using Kimi K3 + OpenCode. I pay their API pricing, costs about $5 / hour (and chews through ~10 million tokens / hour) during continuous use when I have one or two sessions running and doing their thing.

Can't comment on how it compares to plans (I really don't like the limitations and general shenanigans I see around plans, so I've never tried them).

It is notably slower than Fable / Opus / Gemini, but also vastly cheaper than their API pricing.

I am using GLM-5.2 via Ollama Cloud in the $20/month plan. With the same plan I can get many different API keys that I use to run my OpenWebUI server, my opencode and pi dev sessions. I am usually running 2 to 4 sessions concurrently, and I never hit quota limits. At work I get Claude, and I was getting reports that I was spending $75 per hour of work on Opus.

I use opencode or pi harness with the deepseek api for all my at home coding usages.

Deepseek is at least on par with Sonnet (ghcopilot at work)- I don’t use opus, too spendy and I don’t need that level of ability.

The cost is for me was $5/6 months of use. Not a big user I guess! It’s good though, fast enough and incredibly inexpensive.

Been testing Qwen 3.6 28B on a 5090, and it’s also quite good for “free”.

I mostly do small self serving embedded projects based on esp32, so not very complex.

I'm using Qwen 3.6 27B on a macbook with Pi. It's alright, it runs fairly quick (40 tps for quality version, 80 for the fast). It doesn't tend to one shot things but I'm generally comfortable fixing the bugs myself afterwards or prodding it a little bit. I find the harness matters a lot. "Continue" (the vscode extension) worked horribly, OpenCode was ok but its vibecoded internals make me view it as a security nightmare so I'm hesitant to run it, so I've settled on Pi for now.

Claude and ChatGPT are good deals right now, with the subsidies. They produce things faster and better. I guess not cheaper, in that inferrence on my macbook is basically free, although the macbook itself definitely wasn't. My focus on running local is around three principles:

1. I don't want to support surveilance capitalism by giving these companies my data anymore, when I can avoid it. And LLM companies want to vacuum up every detail of your life.

2. I don't find these companies to be remotely trustworthy, and I find them hostile to a healthy society, so I want to avoid giving them money going forward

3. I think they're going to start charging a lot more

> I’m asking because i keep hearing that open weight models are cheap and efficient - is that really the case in practice?

I think that's the case for people who compare it to proprietary models paid via API—which I think is irrelevant given the majority of people daily driving AI coding are doing on a subscription plan.

The better analysis then is not about AI coding, since there's no subscription plan for Kimi K3.

Instead, compare the cost of running some agentic _task_ that isn't coding which can only be done via API. Think of all the startups wrapping around ChatGPT and Claude to provide some additional set of tools, context, data and hoping to turn it into a profitable service.

To those companies, which are many, open models are the difference between the math working out today vs. praygeing they can scale fast enough to find profitability.

qwen3.6 27B q5, llama.cpp, RTX 3090, pi, cost: electricity bill

While I am grateful for open weights models I never found much use of them in the past, barring those I could run myself. This changed with deepseek 4 - it is staggeringly cheap, even if the performance definitely isn't near sota and it's not particularly fast either.

When I expect to need a lot of tokens and the task isn't too difficult I use sota to plan and create a thorough set of instructions and let deepseek chip away at it. With thorough instructions the quality tends to be satisfactory, and you pay something silly like $15 for 600m tokens.

GLM 5.2 seems like a decent price/perf and Kimi 3 has some real nice performance for an open weights model, but gpt 5.6 is unexpectedly affordable (especially if you don't automatically use Sol at max) so I don't think either is worth it atm. The exception is when you're working on something that US models get cold feet about, which seems like a constantly growing list. For me Fable is already too much of a headache in this regard, but chatgpt is still okay-ish. Hopefully it'll last. If not, there's Kimi.

tldr SOTA for most things because gpt 5.6 is token efficient. If I expect to burn a lot of tokens I use deepseek 4.

I use DeepSeek 4 with the VSCode CoPilot plugin. I pay about $10 month on the pay-as-you-go plan.

It's not as good as the frontier models I use at work, but it's plenty capable for the types of tasks I am using it for.

Kilo code with direct API payment to DeepSeek. It costs pennies per day event at max.