Comment by twobitshifter
10 hours ago
exactly! so the arguments against the AI not prompting itself is not a refutation just as it would not be for a person.
10 hours ago
exactly! so the arguments against the AI not prompting itself is not a refutation just as it would not be for a person.
Uh?
Human _can_ check themselves. They don't _always_ check themselves.
Motor _can_ move vehicle. They don't _always_ move vehicle.
LLM _cannot_ check themselves. They _never_ can. It is not that some don't, they just cannot, they are not a system complex enough to do so.
So, yes, it is a refutation. If you have something that _never_ can move a vehicle, this thing does not qualify as a motor, even if some motor, sometimes, don't move a vehicle.
And if your next argument is "yeah but I would argue you don't need to check yourself to be conscious or to understand things", then you just redefine the definition that is owned by your interlocutor. Your interlocutor is saying that this is a criteria they are expecting. Good for you if you are not expecting this criteria. But the problem is that the answer is not "this criteria is not expected", the answer is "I change the criteria from 'being capable to in some circumstances' into 'does always do it in any circumstances'".
The OP said: nozzlegear 19 hours ago [–]
Yes, LLMs don't think on their own, for one; they think when you invoke them.
My rebuttal is that people only think when invoked just the same and can enter states where there is no consciousness just the same. The OP has already accepted that LLMs think, but it seems that you are arguing they do not? This car business is confusing and the LLMs not checking themselves is also wrong, there’s even a benchmark for this https://correctbench.github.io/
No.
OP says: LLMs never think when not invoked.
What you said: I have example where, sometimes, human think when invoked.
That's the difference: human brains are intrinsically different because they are built to be able to think without being invoked, even if there are situations where they think when invoked.
There are tons of obvious examples of human thinking without being invoked. Just take a bath and you will see :)
To be clear: the person I was replying to asked if the way a human thinks was any different from an LLM with a context window. That's the context of my answer. An LLM is a machine, it can't do anything unless we invoke it or give it the instructions and capabilities to do so. It has no free will, it can't just decide to compose a symphony one day unless those are part of its instructions. It can't do anything unless we tell it to do so and give it the capabilities to do so, it doesn't even exist unless it's loaded into memory. That's obviously different from human consciousness, and that's the whole of the point that I'm making.
You can argue that humans are just biological machines reacting to external stimuli, but that's a philosophical argument that I'm not interested in having and frankly, I think it'd be selling yourself short a little bit.
> LLM _cannot_ check themselves. They _never_ can. It is not that some don't, they just cannot, they are not a system complex enough to do so.
All modern agentic harnesses can do this. Nobody uses raw LLM for anything remotely complex. There's always some external system in place. That system is part of the "thought process".
Adjacency doesn't matter here, only what the result of the system of pieces is.
Check themselves does not mean: do a loop.
It means having self-control on their action and being aware of them. If you ask a system, it will respond, it cannot choose to not respond (even if the response if "I don't want to response", it still "run", still do the work). If you don't ask a system, it will not respond.
Adjacency is the point of the thread here. Saying "you say X is important to decide if the thing is intelligent/understanding/conscious, so let me just change X in the middle of the discussion and say that X does not matter".
That is exactly my first comment in this thread: I don't care if AI think or whatever, my reaction was about these "counter-arguments" that totally miss the point and make the person who push them ridiculous. If you want to have a counter-argument, you first need to understand the interlocutor, not just spew whatever rebuttal you constructed that answer something unrelated to what the interlocutor brought to the conversation.
6 replies →