Comment by behnamoh
2 months ago
Apple is very late to the AI party. By the time M7 is shipped, Nvidia will announce 6090 and people will be buying used (3|4|5)090 GPUs to run local models at much better performance than heat throttled M7.
2 months ago
Apple is very late to the AI party. By the time M7 is shipped, Nvidia will announce 6090 and people will be buying used (3|4|5)090 GPUs to run local models at much better performance than heat throttled M7.
This a significant misunderstanding of which party it is Apple wants to attend.
And 6090 will have 48GB of RAM compared to something like an M7 Max that might have 192GB or an M7 Ultra that might have 768GB.
The M7 Max and M7 Ultra will likely prefill-bottlenecked at 100GB+ scale inference. Layered 6090s would not be.
Neural Accelerators in M5 are already 4x faster than M4 at prefill. With M7, especially if they focus on AI like this article claims, it likely will have excellent prefill compute.
What people? Are you seriously thinking the hundreds of millions of customers Apple have is going to be buying run-to-the-ground GPUs second hand and build local workstations for AI? Might as well ask them to self host email while you’re at it.
The difference between these two is that one of them is an unsolved research problem that we’ve all spent far too much time on, and the other is just running an LLM.
I would prefer a Studio if it does a decent enough job even if throttles a bit under load, way less power usage and noise than those GPUs plus the PC you need to put those in.
If you're fine overpaying for a throttling computer, you could buy 40-series cards and underclock them to the same TDP of a Mac Studio.
You'd probably get faster prefill speeds, as well as better drivers for accelerated transcode and gaming applications.
Yeah but you could heat your whole house!
RAM is a commodity and nvidia will be paying the same prices. The used market will reflect the cost of RAM. nvidia owns the top of the market but many of us don't need that.
same people who bought all mac mini for ai?
Theyve dropped the ball bigtime.