Comment by stackedinserter
5 hours ago
The main gotcha for local models is insane hardware requirements.
Even for $10K you get mediocre performance.
5 hours ago
The main gotcha for local models is insane hardware requirements.
Even for $10K you get mediocre performance.
I think this is worth revisiting once we get some solid 3rd party numbers from the new mac studio ultras, which admittedly are a bit over $10k with a 2tb ssd + 256gb ram. I think I'd be seriously considering it if I had a $200/month subscription of some kind.
This situation has improved quite a bit recently, Qwen Flash Next will run on a $4000 PC and can reliably implement small features on its own (feels comparable to Opus 4.5). It's a bit slow but pretty effective.
Not anymore. Qwen 3.8 27B is so great that 5k investment gives you excellent speed and quality.