Comment by somenameforme

Something worth repeating is that in spite of their ever-expanding competence the models remain, fundamentally, token prediction algorithms. And so given a comparable base of training material, and a comparable algorithmic implementation, you should expect to see comparable answers to most questions.

This is also why I think the 'acceleration' hype is probably unjustified. What seems vastly more likely is that we start to reach an asymptotic 'perfection' where models can accurately and consistently provide meaningful next token prediction with minimal hallucinations. And then a race downward to compact this into the most efficient package possible.

This would mean the models still would not be able to create things that do not follow in a reasonable probabilistic fashion given a knowledge of [everything], and so that is the domain, 'applied creativity', where I'd expect humans to remain dominant.