Comment by pvtmert

You don't need a (one huge) model to do everything. You need specialized & smaller models that are very good at specific tasks. Collaborating among themselves.

The fact that we see stagnation in terms of billions of parameters shows that efficiency does not scale linearly with the model size. More of an S shaped chart. The middle was Claude 3.5. Since then, it is more about integrating and collaborating with different systems.