Comment by hamdingers
4 months ago
This is a ridiculous mindset. Llama 3.1 8B can do lots of things today and it'll still be able to do those things tomorrow.
If you baked one of these into a smart speaker that could call tools to control lights and play music, it will still be able to do that when Llama 4 or 5 or 6 comes out.
If you pay $1,500 for a Mistral ASIC that is beaten by a $15 Qwen ASIC that comes out six months later, you'd be feeling pretty dang ridiculous.
I'm equally capable of making up numbers to support my perspective but I don't see the point.
The point is that the GP's mindset is not very ridiculous if you value things by a price/utility ratio. Software and hardware advancements will lead to buyer's remorse faster than people get an ROI from local inference.
1 reply →
I'd pay for $100 chip that replaces anthropic sub and works 10x faster, even for 12 months.
Edit: assuming model owners will let this happen, which they wont