Comment by FuriouslyAdrift
8 hours ago
I've found most of the frontier coding models require somewhere between 300GB to 1TB to run with full capabilities.
8 hours ago
I've found most of the frontier coding models require somewhere between 300GB to 1TB to run with full capabilities.
If only we could buy 1TB of unified memory in a Mac for $1k-$2k in total hardware costs. Apple would basically be able to extinguish the entirety of the market cap for Nvidia, OpenAI, Anthropic, and others all at once.
In 10 years, I hope my MacBook Pro can run today's frontier models and has 1TB of unified Memory.
Why can’t Apple launch a $50k product for $1k? Everyone would buy it!
To go further down this pipe dream - Anthropic / OpenAI would buy them all and still price out the consumer. There's no end-run in this scenario.
The Nvidia GB300 DGX Station, which isn't even going to hit 1TB total memory, is expected to launch at almost $100k. Bit of a pipe dream with memory prices where they're at.
There are multiple server systems available right around the $100k range that have 512B of GPU RAM right now (4x AMD Instinct MI300A)
GIGABYTE G383-R80-AAP1 for example
They want you to buy four 256GB Studios and link them with ThunderBolt.
Yes, particularly if that memory is designed and engineered by Apple in house like Apple Silicon in house and manufactured by TSMC on shore somewhere in the United States.
I’m bullish on Apple because of that. Tech waves always oscillate between mainframe/thin-client models at first, then commodity hardware catches up. Apple is well positioned to deliver that with the M series, all it takes is for the current AI bubble to pop a bit and memory costs go down.
The people who train the frontier models want to recover their costs, so they're not going to let you do that.
> Apple would basically be able to extinguish the entirety of the market cap for Nvidia
I don't think you understand why people buy Nvidia hardware if you're beating the "just add more dual channel DDR, bro" drum. Apple wouldn't even be able to extinguish AMD with a product like that, it's all slow memory being fed into a raster-first GPU architecture.
The work on LLM in a Flash will probably help, and Apple's NVMe architecture is well suited to maximize throughput could allow their devices to work better on larger models than other vendors.
[flagged]