Comment by ActorNightly

There are endless posts on HN about how Macs are good for LLMs. Somoene is running some model on their Mac for local inference.

What the posts dont mention is how unusable that experience is with dogshit slow tokens/second. To make a local model usefull you need to run the highest parameter models at 100+ tok/sec, otherwise you are just better off paying for cloud inference. So either all those people are dumb as hell, or Apple is doing clever advertising. And I personally have more faith in the tech sector.