Comment by minraws
13 hours ago
By next year we will likely see GLM 5.3 level performance in Qwen 4 MoE models that can run on most higher end consumer systems.
I don't think there is a reason to care about any of it, just get a few dollars on a few openrouter or similar platforms and just use the models for now, wait for cheaper models that are good enough for you to become available for your hardware.
Sure you might never see frontier performance on consumer hardware but do you really need that though? Does anyone? And if you ever do just get a 30-day sub for that work, and cancel it.
Yeah performance at a given compute budget will likely continue to improve.
We are running Qwen 3.8 27B FP8 with 4x 5060 16gb at the office for a small development team. It is doing pretty well for implementing backend and frontend tasks, and is already useful. In 6 and 12 months from how I expect it to get even better. Some planning is still done with a stronger model, but even that is maybe not strictly necessary.
Qwen 3.8 wasn't that nice in my experience Qwen series has lagged a lot but I think Qwen 4 would be better they cleaned up the architecture.
And we are likely to see many companies start building 27-32B MoE models custom built for use cases now that Rubin is going to make training costs much lower.
I expect training costs to get under 100k USD within a year for small models with another 100k maybe for RL.
I think just having all your own data from claude and stuff on your system that you can feed these models and have companies offer great base 27B models will be how it works.