Comment by sokoloff
12 hours ago
Why would they sell a device with 256GB of RAM as the lowest-spec device rather than making 8 32GB or 16 16GB machines as their entry-level?
Apple’s not exactly famous for their low pricing on spec upgrades nor competing based on being the price leader…
If 256GB of RAM enables them to run on-device AI models that (for reasons) are a key feature differentiator?
Personally, I think there's no way memory heavy inference moves on-device (vs cloud) due to the economics, but it's not impossible technology + platforms go that way for currently unforeseeable reasons.
I think there’s a realistic chance consumer inference moves on-device. I think it really depends on marketing.
My non-tech friends and family would probably be served perfectly fine by local models today, if they had a working web search tool. Their queries are often “soft” and don’t have an exact answer. My mom and aunt used it to pick a hairstyle, my mom used it to get an image of what a room would look like with particular drapes in it, etc. Stuff I think mid-sized local models like Gemma or smaller Qwens could do without issue. They just don’t have a device that will run them.
Businesses won’t move. They need a huge context so they can stuff a bunch of Confluence pages in it and 300 tools and it needs to read an entire codebase and yada yada. The hardware depreciation and electricity will probably make it a net zero or even cost more than paying for API access.
The economic argument in favor of cloud inference: higher utilization is always going to have a ROI for inference hardware.
But maybe that hardware becomes so commoditized that it's not difficult to obtain / stuff in a box.
1 reply →
Right. I’m not arguing that Apple wouldn’t offer a 256GB model if they could make money doing it; I’m puzzled as to why they wouldn’t offer several lower-spec models as the entry-level into and then progressive upgrades within that line, since only some people need that 256GB feature differentiator of running frontier-level models on their MacBook Pro.
And I'm saying, if 256GB of memory is a requirement for running customer-expected local models (and local models are preferred for some reason).
Think past on-device inference... imagine what on-device training could do. And that would need a lot of RAM.