How can a something that generates such a massive surge of interest, investment and research into AI not be a step toward it?
Saying it’s not a step towards AGI is basically saying AGI isn’t possible at all, because it means that all our efforts are making zero progress on AGI. That’s not a falsifiable position to take.
If you’re serious, the parent post literally said “AGI isnt going to look like this”.
…but realistically, how would a LLM that could easily refine itself from experiences, and had a very large context, let’s say, a billion tokens, be meaningfully different from AGI?
It could learn. It could remember things. It could generate human like output from a complex context.
Sure, it’s just a stochastic parrot… but if it can refine the model from real world inputs (learn new tricks, learn games, etc) and generate large scale (entire books worth) of coherent conversation and interactions… where do you draw the line between that and actual AGI?
Large contexts (35k tokens) are here right now. Refining models is here right now. They’re just expensive and slow (inference and training).
Maybe the current architecture doesn’t scale up beyond that and it’s a dead end, but my gosh.
If you don’t think what we have is a step towards AGI you really have to work hard to make your definition of AGI very very difficult to attain.
An AGI needs to be able to take an abstract concept and apply it to create a solution to a problem it has not encountered before at all - not sure LLMs can do that really. The lack of mathematics might be quite limiting there.
Because a research paper claimed that they believe it's a basic and incomplete AGI. However, said paper then goes on to actually say LLMs aren't the way forward if people bother to read it.
One comment on HN called it a "baby AGI" after linking to the paper.
Yeah but LLMs could be a small component of future AGI. You can’t deny that models that fool so many people into thinking the models really reason are a step towards AGI.
Honest question: How do you know it is a step towards AGI?
If you accept that AGI is possible at all…
How can a something that generates such a massive surge of interest, investment and research into AI not be a step toward it?
Saying it’s not a step towards AGI is basically saying AGI isn’t possible at all, because it means that all our efforts are making zero progress on AGI. That’s not a falsifiable position to take.
If you’re serious, the parent post literally said “AGI isnt going to look like this”.
…but realistically, how would a LLM that could easily refine itself from experiences, and had a very large context, let’s say, a billion tokens, be meaningfully different from AGI?
It could learn. It could remember things. It could generate human like output from a complex context.
Sure, it’s just a stochastic parrot… but if it can refine the model from real world inputs (learn new tricks, learn games, etc) and generate large scale (entire books worth) of coherent conversation and interactions… where do you draw the line between that and actual AGI?
Large contexts (35k tokens) are here right now. Refining models is here right now. They’re just expensive and slow (inference and training).
Maybe the current architecture doesn’t scale up beyond that and it’s a dead end, but my gosh.
If you don’t think what we have is a step towards AGI you really have to work hard to make your definition of AGI very very difficult to attain.
An AGI needs to be able to take an abstract concept and apply it to create a solution to a problem it has not encountered before at all - not sure LLMs can do that really. The lack of mathematics might be quite limiting there.
3 replies →
Because a research paper claimed that they believe it's a basic and incomplete AGI. However, said paper then goes on to actually say LLMs aren't the way forward if people bother to read it.
One comment on HN called it a "baby AGI" after linking to the paper.
Eye roll inducing.
Yeah but LLMs could be a small component of future AGI. You can’t deny that models that fool so many people into thinking the models really reason are a step towards AGI.
Current-generation LLMs give many people an impression of reasoning. I think reasoning about things is a big part of AGI.