Comment by PlatoIsADisease
7 days ago
>with little power and without triggering its fan.
This is how I know something is fishy.
No one cares about this. This became a new benchmark when Apple couldn't compete anywhere else.
I understand if you already made the mistake of buying something that doesn't perform as well as you were expecting, you are going to look for ways to justify the purchase. "It runs with little power" is on 0 people's christmas list.
It was for my team. Running useful LLMs on battery power is neat for example. Some simply care a bit about sustainability.
It’s also good value if you want a lot of memory.
What would you advice for people with a similar budget? It’s a real question.
But you arent really running LLMs. You just say you are.
There is novelty, but not practical use case.
My $700, 2023, 3060 laptop runs 8B models. At the enterprise level we got 2, A6000s.
Both are useful and were used for economic gain. I don't think you have gotten any gain.
Yes a good phone can run a quantised 8B too.
Two A6000 is fast but quite limited in memory. It depends on the use case.
3 replies →