Comment by bigyabai
2 days ago
Nvidia and AMD both have their own unified memory laptop SOCs, now. Apple Silicon's GPU is relatively weak, it's one of the less-efficient ways to use 100w for compute.
Even the fastest Apple Silicon chips like the M5 Max and the M3 Ultra still put up worse GPU compute performance than last-gen laptop RTX 4080 chips. And they don't scale, the largest M3 Ultra cluster you can configure is still ~2,000x smaller than a DGX SuperPOD. There's a reason Apple discontinued their rackmount hardware, there's very little demand for Apple Silicon in the datacenter.
As I mentioned above you can't take the data center into the cafe somewhere. We're talking about running local models here.
But you may be able to connect with your laptop to your home server, even from some cafe, and run the LLM remotely.
I always connect back home when I am away and I want access to a beefier computer. On the home server, "Wake on LAN" is enabled, so I can power it on and off from my home router.