Comment by jayd16
6 days ago
Something can be much better than before but still be a dead end. Literally a dead end road can take you closer but never get you there.
6 days ago
Something can be much better than before but still be a dead end. Literally a dead end road can take you closer but never get you there.
But dead end to what? All progress eventually plateaus somewhere? It's clearly insanely useful in practice. And do you think there will be any future AGI whose development is not helped by current LLM technology? Even if the architecture is completely different the ability of LLMs to understand humans data automatically is unparalleled.
To reaching AI that can reason. And sure, as I wrote, large language models might become a relevant component for processing natural language inputs and outputs, but I do not see a path towards large language models becoming able to reason without some fundamentally new ideas. At the moment we try to paper over this deficit by giving large language model access to all kind of external tools like search engines, compilers, theorem provers, and so on.
When LLMs attempt to some novel problems (I'm thinking of pure mathematics here) they can try possible approaches and examine by themselves which approaches are working and not and then come to conclusions. That is enough for me to conclude they are reasoning.
You're in a bubble. Anyone who is responsible for making decisions and not just generating text for a living has more trouble seeing what is "insanely useful" about language models.
I don’t think you’re right about that. LLMs are very good for exploring half-formed ideas, (what materials could I look at for x project?), generating small amounts of code when it’s not your main job, and writing boring crap like grant applications.
That last one isn’t useful to society, but it is for the individual.
I know plenty of people using LLMs using for stuff like this, in all sorts of walks of life.
Anthropic and OpenAI researchers themselves certainly use AI--do you think they generate text for a living.
1 reply →
> the ability of LLMs to understand
But it doesn't understand. Its just similarity and next likely token search. The trick is that turns out to be useful or pleasing when tuned well enough.
Implementation doesn't matter. In so much as human understanding can be reflected in a text conversation, its distribution can be approximated using a distribution in next token prediction. Hence there exist next token predictors which are indistinguishable from a human over text--and I do not distinguish identical behaviors.