Comment by TobiasJacob
3 hours ago
I'm still hoping some future LLM is good and small enough for people to run local AI for everyday life.
3 hours ago
I'm still hoping some future LLM is good and small enough for people to run local AI for everyday life.
It's interesting trying to be an AI "moderate," which as the owner of a 16gb card I definitely am. Still working on the balance, but it really does feel like I'm going to land on something like "use external AI services to build a few tools, but use my local rig to actually handle everything personal once those tools are built."
Already here. 3.8-27b is good enough.
5 years ago it would have been science fiction
Do you find it intelligent enough? What hardware are you using?
For me, qwen 3.8 27b on my m5 max is so slow and dumb that I can't think of a single use for it. Haiku, Luna, or data center scale open models are ten times faster and ten times smarter (and inference so cheap that it's essentially free). It was a nifty toy to brag to friends about but why would I waste my time goofing with something so slow and dumb?
For roughly the same reason you might slow down and read a book on how to do something as opposed to speedrunning 20 youtube videos?