Comment by Kiro
1 day ago
They are not necessarily cheaper. The commercial models are heavily subsidized to a point where they match your electricity cost for running it locally.
1 day ago
They are not necessarily cheaper. The commercial models are heavily subsidized to a point where they match your electricity cost for running it locally.
In the arguably-unique case of Apple Silicon, I'm not sure about that. The SoC-integrated GPU and unified RAM ends up being extremely good for running LLM's locally and at low energy cost.
Of course, there's the upfront cost of Apple hardware... and the lack of server hardware per se... and Apple's seeming jekyll/hyde treatment of any use-case of their GPU's that doesn't involve their own direct business...