Comment by yusina
9 months ago
> understand the work
LLMs don't understand. It's mind-boggling to me that large parts of the tech industry think that.
Don't ascribe to them what they don't have. They are fantastic at faking understanding. Don't get me wrong, for many tasks, that's good enough. But there is a fundamental limit to what all this can do. Don't get fooled into believing there isn't.
> LLMs don't understand. It's mind-boggling to me that large parts of the tech industry think that.
I think you might be tied to a definition of "understanding" that doesn't really apply.
If you prompt a LLM with ambiguous instructions, it requests you to clarify (i.e., extend prompt to provide more context) and once you do the LLM outputs something that exactly meets the goals of the initial prompt, does it count as understanding?
If it walks like a duck and quacks like a duck, it's a duck,or something so close to a duck that we'd be better off calling it that.
> If you prompt a LLM with ambiguous instructions, it requests you to clarify (i.e., extend prompt to provide more context)
It does not understand that it needs clarification. This behavior is replicated pattern
So you have two prompts, one is ambiguous and the second is the same prompt but with the ambiguity resolved.
In the first prompt the replicated pattern is to ask for clarification, in the second prompt the replicated pattern is to perform the work. The machine might understand nothing but does it matter when it responds appropriately to the different cases?
I don't really care whether it understands anything at all, I care that the machine behaves as though it did have understanding.
1 reply →
What is the difference? What would actual understanding look like?
9 replies →
> If it walks like a duck and quacks like a duck, it's a duck,or something so close to a duck that we'd be better off calling it that.
Saying “LLMs match understanding well enough”, is to make the same core error if we were to say “rote learning is good enough” in a conversation about understanding a subject.
The issue is that they can pass the test(s), but they dont understand the work. This is the issue with a purely utilitarian measure of output.
I think most of us agree with Searle that a Chinese room does not understand Chinese.
https://en.m.wikipedia.org/wiki/Chinese_room
Nor does a neuron.
Argumentum ad populum, I have the impression that most computer scientists, at least, do not find Searle's argument at all convincing. Too many people for whom GEB was a formative book.
[flagged]
> If you define a grammar for a new programming language and feed it to an LLM and give it NO EXAMPLES can it write code in your language?
Yes. If you give models that have a cutoff of 2024 the documentation for a programming language written in 2025 it is able to write code in that language.
1 reply →
In my experience it generally has a very good understanding and does generate the relevant test cases. Then again I don't give it a grammar, I just let it generalize from examples. In my defense I've tried out some very unconventional languages.
Grammars are an attempt at describing a language. A broken attempt if you ask me. Humans also don't like them.
1 reply →
I've had LLMs do that more than once.
> NO.
YES! Sometimes. You’ll often hear the term “zero-shot generation”, meaning creating something new given zero examples, this is something many modern models are capable of.
> If you define a grammar for a new programming language and feed it to an LLM and give it NO EXAMPLES can it write code in your language?
Neither does your average human. What's your point?
5 replies →
> If you define a grammar for a new programming language and feed it to an LLM and give it NO EXAMPLES can it write code in your language?
Of course it can. It will experiment and learn just like humans do.
Hacker news people still think LLMs are just some statistical model guessing things.
4 replies →
I don't believe the user meant "understand" in the classical biological and philosophical sense, or were otherwise attempting to anthropomorphize the systems. They were speaking from the practical experience of "this thing takes a somewhat ambiguous input with unique constraints and implements the ask more-or-less as intended".
They understand. Anything able to reason about any arbitrary request and form a plan tailored to that request understands well enough to qualify for the verb. The mechanism behind it may feel hollow or fake. But if its responses reliably show understanding, the LLM understands - by any ordinary measure.
Rote learning is a term that exists which specifically punctures this output oriented measurement of understanding.
Nearly every argument like this has the same fatal flaw, and it's generally not the critique of the AI, but the critique reflected back on to humans.
Humans also don't understand and are frequently faking understanding, which for many tasks is good enough. There are fundamental limits to what humans can do.
The AI of a few months ago before OpenAI's sycophancy was quite impressive, less so now which means it is being artificially stunted so more can be charged later. It means privately it is much better than what is public. I can't say it "understands," but I can say it outclasses many many humans. There are already numbers of tasks based around understanding where I would already choose an LLM over a human.
It's worth looking at bloom's taxonomy (https://en.wikipedia.org/wiki/Bloom%27s_taxonomy): In the 2001 revised edition of Bloom's taxonomy, the levels were renamed and reordered: Remember, Understand, Apply, Analyze, Evaluate, and Create. In my opinion it is at least human competitive for everything but create.
I used to be very bearish on AI, but if you haven't had a "wow" moment when using one, then I don't think you've tried to explore what it can do or tested it's limits with your own special expertise/domain knowledge, or if you have then I'm not sure we're using the same LLMs. Then compare that experience to normal people, not your peer groups. Compare an LLM to people into astrology, crystal healing, or homeopathy and ask which has more "understanding."
I do agree with you - but the big difference is that humans-who-are-faking-it tend to learn as they go so might, with a bit of effort, be expected to understand eventually.
Does that actually matter? Probably not for many everyday tasks...
Um, moving the goal post?
The claim was LLMs understand things.
The counter was, nope, they don't. They can fake it well though.
Your argument now is, well humans also often fake it. Kinda implying that it means it's ok to claim that LLMs have understanding?
They may outclass people in a bunch of things. That's great! My pocket calculator 20 years also did, and it's also great. Neither understands what they are doing though.
It's fun to talk about, but personally he whole "understanding" debate is a red herring, imo what we actually care about when we talk about intelligence is the capacity and depth of: second order thinking, regardless of the underlying mechanism. I think personally key question isn't "do LLMs understand?" but, "can LLMs engage in second order thinking?" The answer seems to be yes - they can reason about reasoning, plan their approaches, critique their own outputs, and adapt their strategies, o1 has shown us that with RL and reasoning tokens you can include it in a single system, but our brains have multiple systems we can control and that can be combined in multiple ways at any given moment: emotions, feelings, thoughts combined into user space, 3 core systems input, memory, output. The nuances is in the fact that various reasons (nature + nurture), various humans appear to have varying levels of meta control over the multiple reasoning systems.
Why are you pretending to be participating in a debate? You mention things like "moving the goalpost", "counter[arguments]", and "arguments", as if you did anything more than just assert your opinion in the first place.
This is what you wrote:
> LLMs don't understand.
That's it. An assertion of opinion with nothing else included. I understand it sucks when people feel otherwise, but that's just kinda how this goes. And before you bring up how there were more sentences in your comment, I'd say they are squarely irrelevant, but sure, let's review those too:
> It's mind-boggling to me that large parts of the tech industry think that.
This is just a personal reporting of your own feelings. Zero argumentational value.
> Don't ascribe to them what they don't have.
A call for action, combined with the same assertion of opinion as before, just rehashed. Again, zero argumentational value.
> They are fantastic at faking understanding.
Opinion, loaded with the previous assertion of opinion. No value add.
> Don't get me wrong, for many tasks, that's good enough.
More opinion. Still no arguments or verifiable facts presented or referenced. Also a call for action.
> But there is a fundamental limit to what all this can do.
Opinion, and a vague one at that. Still nothing.
> Don't get fooled into believing there isn't.
Call for action + assertion of opinion again. Nope, still nothing.
It's pretty much the type of comment I wish would just get magically filtered out before it ever reached me. Zero substance, maximum emotion, and plenty of opportunities for people to misread your opinions as anything more than that.
Even within your own system of opinions, you provide zero additional clarification why you think what you think. There's literally nothing to counter, as strictly speaking you never actually ended up claiming anything. You just asserted your opinion, in its lonesome.
This is no way to discuss anything, let alone something you or others likely feel strongly about. I've had more engaging, higher quality, and generally more fruitful debates with the models you say don't understand, than anyone here so far could have possibly had with you. Please reconsider.
6 replies →
Excellently put.
meh. I feel this is just a linguistic shortcut, similar to how _trained_ biologists can talk about a species or organism evolving some trait. Of course the organism isn't _really_ evolving with any goal in mind, but that's clear to the speaker and audience. Whether or not LLMs understand (very unlikely), it's clear what we mean by an LLM "understanding": has the context + prior training to make reasonable predictions. But no one wants to write that each time.
That's an interesting take and in fact one I could get behind.
But I'm afraid that most folks using the term mean it more literally than you describe.
Exactly. The whole point of all the LLM companies is to get grandma to use it. If you say understand about a technology with the desired appeal of Facebook, then you’re talking to everyone and words matter extra hard.
They understand tho, it's different than how it's done in our brain but they solve task that would be impossible to do without understanding. I would even say that they can now reason through problems thanks to powerful reasoning models like Gemini 2.5 Pro and o3.
How do you know?
Extraordinary claim requires extraordinary proof. I don't know, but I'm also not the one claiming something.
(Besides, we know what LLMs do, and none of those things indicate understanding. Just statistics.)
You can create a new game with new rules never seen before
You can explain this to an LLM
The LLM can then play the game following the rules
How can you say it hasn't understood the game?
5 replies →
Definition of understanding is based on connecting relations. If there is one thing a llm can do its connecting relations. So I am not sure why you say llms are not understanding.
What is the limit my system will reach?
Thats an interesting word to pick on. Understanding still means something here in a relative sense.
Asking a short question but in a serious way: so what?
You are asking why it is meaningful to use terms for what they mean instead of making up things?
Well, I prefer it that way, but the spirit of "AI" seems to go in another direction, and the leadership of US government also does, so maybe times are just changing.