Comment by nayroclade

The core argument here, as far as I can discern it, seems to be: A trillion dollars has been spent scaling LLMs in an attempt to create AGI. Since scaling alone looks like it won't produce AGI, that money has been wasted.

This is a frankly bizarre argument. Firstly, it presupposes that _only_ way AI becomes useful is if turns into AGI. But that isn't true: Existing LLMs can do a variety of economically valuable tasks, such as coding, even when not being AGI. Perhaps the economic worth of non-AGI will never equal what it costs to build an operate it, but it seems way too early to make that judgement and declare any non-AGI AI as worthless.

Secondly, even if scaling alone won't reach AGI, that doesn't mean that you can reach AGI _without_ scaling. Even when new and better architectures are developed, it still seems likely that, between two models with an equivalent architecture, the one with more data and compute research will be more powerful. And waiting for better architectures before you try to scale means you will never start. 50 years from now, researchers will have much better architectures. Does that mean we should wait 50 years before trying to scale them? How about 100 years? At what point do you say, we're never going to discover anything better, so now we can try scaling?