Comment by overfeed
21 hours ago
Everyone knows the path followed by every single technology mankind has ever developed - including the incredible run that was Moore's law. Sure, some will think "This time is different" - but is it?
21 hours ago
Everyone knows the path followed by every single technology mankind has ever developed - including the incredible run that was Moore's law. Sure, some will think "This time is different" - but is it?
It is, because just like you mentioned, there's no exponential physics to back up growth. LLMs have to stop scaling up compute, because there's only so much resources.
But even if we think back to the previous decade or two (sans AI), most of tech has been a a red queen's race - developing and improving stuff, without any real substantial results.
Microservices and orchestration are not better than old monoliths. Electron apps are not better than native. Most programs people actually use have decades-old codebases.
Video games of today don't look substantially better or play different than ones from 15 years ago.
It doesn't need to run indefinitely. I don't think anyone sane thinks it's going to run indefinitely. It just needs to run long enough. Moore's law was what, 50 years? If AI keeps improving at the rate it currently is for 20 years, the world will be very different when that curve flattens out, in a way that makes most of our current concerns are entirely irrelevant.
Those cycles have shortened considerably. Just look at the lifespan of media, integrated circuits, software, products. There have been so many big model releases just in the last three months I've lost count. We're now closer to weeks between steps.
Maybe if China stops releasing capable models the pace will slow down a little, but they have lots of incentives not to.