Comment by slwvx
3 days ago
My impression is that solar (and maybe wind?) energy have benefited from learning-by-doing [1][2] that has resulted in lower costs and/or improved performance each year. It seems reasonable to me that a similar process will apply to AI (at least in the long run). The rate of learning could be seen as a "pace" of improvement. I'm curious, do you have a reference for the deceleration of pace that you refer to?
[1] https://emp.lbl.gov/news/new-study-refocuses-learning-curve
[2] https://ourworldindata.org/grapher/solar-pv-prices-vs-cumula...
Why would a the curve of solar prices be in any way correlated with the curve of AI improvements ?
The deceleration of pace is visible to anyone capable of using Google.
u/ipaddr is probably referring to
(As you know) You're referring to Wright's Law aka experience learning curve.
So there's a tension.
Some concerns that we're nearing the ceiling for training.
While the cost of applications using foundation models (implementing inference engines) is decreasing.
Someone smarter than me will have to provide the slopes of the (misc) learning curves.
I was not aware of (or had forgotten) the term "Wright's law" [1], but that indeed is what I was thinking of. It looks like some may use the term "learning curve" to refer to the same idea (efficiency gains that follow investment); the Wikipedia page on "Learning curve" [2] includes references to Wright.
[1] https://en.wikipedia.org/wiki/Experience_curve_effect
[2] https://en.wikipedia.org/wiki/Learning_curve
> It seems reasonable to me that a similar process will apply to AI
If its reasonable, then reason it. Because it is a highly apples to oranges comparison you are making