Comment by kromem
1 year ago
LLMs can't is such an anti-pattern at this point I'm surprised that anyone still dares to stake it. The piece even has an example of a $10k bet around a can't being proven false in under a day, but somehow doesn't think maybe their own can't examples are on similarly thin ice?
In particular, the line about "what models can't do tells us what they don't know" is infuriating.
No, that's not the case at all. At least in a number of instances, what they can't do is because of what they do know.
As an example, one of thecan'ts I got from HN a year ago for GPT-4 was a variation of a classic logic puzzle. And indeed, the model can't solve it - nor can most major models since.
But it's not because the model can't solve the logic - it's because the token similarity to the standard form biases the output towards the standard solution. A hack as simple as changing the nouns to emojis can allow the model to get the correct answer and work through the logic successfully every attempt because it breaks that similarity bias.
People are way too confident around a topic where what's 'known' is more mercurial than maybe any field since 1930s particle physics.
I'd strongly recommend deleting 'never' or 'can't' from one's vocabularies on the subject unless one enjoys ending up with egg on their faces.
An LLM will probably be able to do most of what human minds can do like reason, predict, hypothesize, research, and even get hooked up to other systems to: Visualize, smell, taste, balance, and even direct the movement of limbs, but an LLM can't and won't ever be able to: Feel pain, bliss, anger, sadness, can't feel positive/negative, can't eat/drink, be hungry, feel fatigued, get excited, enjoy things, dislike things, contemplate, meditate, feel warm or cold (though it can detect it), can't feel dizzy (though it can know when it's off balance) - any action where having an experience is a necessary part of what it's doing and the output of it, an LLM is not sufficient to deliver on and never will be.
To compare to a brain, the LLM is like the prefrontal cortex or language and decision network in the outermost layer, but we would still need the amygdala in that metaphor - emotional drives, urges, episodic first-person memories, and experiential components that accompany the language and complete it with personhood.
For raw sensations and tactiles we might need that innermost brain stem - which is probably more chemistry than computation - for the "lights to be on". For example, some jobs will require not just language intelligence, and not just personhood, but for the light behind the images and feelings in the sensations, so that it feels (and would be) alive.
Amen brother. I had feelings like this which I wanted to share, but you hit the nail on the head.
interestingly enough, it seems that ChatGPT-4 can now detect the variations of the classic logic puzzle and solve it, so that can't is now a couldn't.
We may be talking about different logic puzzles? The only model I've seen that didn't need some rather extreme adjustments to eventually solve it was Mistral large.
what's your puzzle? mines the river crossing puzzle
4 replies →