Comment by specialist

u/ipaddr is probably referring to

  1) the dearth of new (novel) training data. Hence the mad scramble to hoover up, buy, steal, any potentially plausible new sources.

  2) diminishing returns of embiggening compute clusters for training LLMs and size of their foundation models.

(As you know) You're referring to Wright's Law aka experience learning curve.

So there's a tension.

Some concerns that we're nearing the ceiling for training.

While the cost of applications using foundation models (implementing inference engines) is decreasing.

Someone smarter than me will have to provide the slopes of the (misc) learning curves.