Comment by theptip

> automated AI research is going to get us to "smarter models with fewer parameters"

That is indeed what we have seen over the last 5 years or so. Algorithmic improvements have made a massive difference to every dimension of efficiency and performance.

AI researchers can do things like discover new attention algorithms to optimize inference efficiency (likely part of why Anthropic and OpenAI are slashing inference costs in their frontier models and no longer capacity crunched).