← Back to context

Comment by apatheticonion

9 hours ago

Here's an anecdote. I have repurposed my 6 year old 32gb M1 MacBook Pro to run Qwen 3.6 35b a3b and I used to give it nightly maintenance tasks that span my OSS github repos.

I gave it mundane tasks like updating dependencies, writing changelogs, etc. It ran about 2 hours a night.

My MBP consumes 0.1w idle and about 10w running inference. I pay $0.33/kwh. If it runs for 2 hours a day over 365 days - it will cost $2.41/year (excluding idle time).

Excluding the cost of hardware - the running costs alone are more expensive than DeepSeek's API and the inference quality is much lower. The economics are even worse on AMD AI Max mini PCs and worse still with a dedicated GPU setup.

That said - my Apple Silicon example is orders of magnitude cheaper than western inference providers and offers unrivaled privacy.

Perhaps a local AI setup would be compelling for small/medium business users... and it would be practical if the hyperscalers didn't buy all of the potatoes on Earth.

2 comments

apatheticonion

Reply
aftbit  4 hours ago

I'm not sure how many tokens/sec you were running through your MBP. Let's just go with 35 tok/sec for the sake of having a number. 35 tok/sec * 3600 sec/hr * 2 hr/day * 365 day/yr => 91 million (output) tokens. At DeepSeek's price of $0.60/million tokens, that's $55.18.

Of course that's assuming it was just generating flat out all the time - no prefill or anything. That's not likely. Probably much of the time (and tokens) would be in prefill and cache hits. And as you say, DeepSeek v4.1 Flash is a much stronger model than a 35B two generation old Qwen.

So yep, DeepSeek's API is pretty cheap. They're not paying anything like $0.33/kwh for one, and they're able to keep util much higher than you are, and they are subsidizing so they can get more data and code to train on.

If you were paying for Fireworks to host the model in the US with no training and ZDR, it would be $1.80 per million tokens (3x price), so a grand total of $165.56/yr.

I'd argue that at those values, the price difference really doesn't matter one way or the other. What matters is output quality, human time required, privacy, and sovereignty. Also, there's something to be said for (nearly) zero marginal cost reducing the barrier to playing with an idea. If I had to spend $40 to try out an idea over a few million files, I might think about it first. If I could just load it up on my existing server and let it cook overnight, I would not worry about it. Maybe I should consider electricity marginal costs more, but practically I don't really.