Comment by Nevermark
20 days ago
If it makes the models smarter, someone will do it.
From any individual, up to entire countries, not participating doesn't do anything except ensure you don't have a card to play when it happens.
There is a very strong element of the principles of nature and life (as in survival, not nightclubs or hobbies) happening here that can't be shamed away.
The resource feedback for AI progress effort is immense (and it doesn't matter how much is earned today vs. forward looking investment). Very few things ever have that level of relentless force behind them. And even beyond the business need, keeping up is rapidly becoming a security issue for everyone.
If Moore's Law had fully kicked over twice more we'd all have 64GB GPUs, enthusiasts would have 2x64GB, and data center build outs wouldn't be needed.
Eventually GPU memory is going to creep up and local models will powerful enough.
I agree. I also think we have only hit the surface of model efficiencies.
Apple's M3 Ultra with RAM up to 512GB shared directly across CPU/GPU/NPUs is a great example of an architecture already optimized for local models. I expect Apple will start offering larger RAM sizes for other form factors too.
And prices for RAM will drop eventually, because of the extreme demand for RAM with higher densities.
It reminds me of the huge infra investments in Sun and Cisco during the first .com boom, and then 5-10 years later those fancy Sun boxes were out performed by Grandma's Windows XP box.