Comment by simonw
1 day ago
The problem with assuming that tokens can only get more expensive is that the Chinese open weight LLM firms have dropped models which have a known, fixed price that can never get more expensive (since we can run them on hardware we own).
Well, I guess we're not discussing the same thing. The cost of cloud tokens are going to go up. They won't ever be cheaper. They're generating far more tokens than my AMD 395+ w/128GB at a much cheaper rate.
I agree though, it can't get cheaper than the cost of hardware it's just without sufficient documentation of the actual costs to run the cloud models, we can't really know what the "true" cost of each token is. I assume there's an economist out there somewhere that could figure it out though. Certainly, the cost should approach at a minimum a open weights model running on a local machine.
I've succesffully got Qwen3-coder-next to loop and generate sufficiently competent code and from what I can tell, the difference between this and the cloth is how quickly the gen happens and perhas how interactive it has to be.