Comment by hackerlight

I expect a plateau in depth before breadth.

Breadth for example means better multi-modality and real-world actions/control. These are capabilities that we haven't scratched the surface of.

But improving depth of current capabilities (like writing or coding) is harder if you're already 90% of the way to human-level competence and all of your training data is generated by human output. This isn't like chess or go where you can generate unlimited training data and guarantee superhuman performance with enough compute. There are more fixed limitations determined by data when it comes to domains where it's challenging to create quality synthetic data.