Comment by cauch
8 hours ago
Check themselves does not mean: do a loop.
It means having self-control on their action and being aware of them. If you ask a system, it will respond, it cannot choose to not respond (even if the response if "I don't want to response", it still "run", still do the work). If you don't ask a system, it will not respond.
Adjacency is the point of the thread here. Saying "you say X is important to decide if the thing is intelligent/understanding/conscious, so let me just change X in the middle of the discussion and say that X does not matter".
That is exactly my first comment in this thread: I don't care if AI think or whatever, my reaction was about these "counter-arguments" that totally miss the point and make the person who push them ridiculous. If you want to have a counter-argument, you first need to understand the interlocutor, not just spew whatever rebuttal you constructed that answer something unrelated to what the interlocutor brought to the conversation.
> Check themselves does not mean: do a loop.
I'm not sure I understand. How do you do it?
In my thought process, I quite literally stop myself, and say "ok, think about what you just said" to check myself. I literally initiate that loop. If I don't, then I'm not using my own mental agency, and just using my firm coded priors.
I will say that I do seem to have a stop, what you said is wrong logic check voice that pops up without me initiating it. But, it's unreliable, and not too much different than all the content monitoring system used for the streaming clients, that will terminate with "content violation" immediately after the "incorrect" words are sent. I don't think integration is important, just the behavior of the overall system.
There is no "loop" in the brain, it is all part of a same line of thought. This is visible because, while you can sometimes have a "two voices / dialectic" way of thinking, you can have the exact same thinking in one-go that does not look like a loop at all.
In fact, in the large majority of the time, you don't process "as a loop" at all, you just continuously progress in your reflection without needing a "second voice" to retrigger you. The fact that sometimes we do this is just something we can do, not the result of something needed for our brain to work. For AI systems, this is something needed because the "answering" part is not able to do the loop on its own. And building a bigger system that combine an "answering" part and a "loop" part does not fit this, does not create a self-reflective system, it just makes a non-self-reflective system and a workaround bundled together.
It's a bit like if the "answering" part was unable to provide only one answer and was always producing plenty of different possible answers, including contradictory ones. Then, you can add an external part that will just pick one answer (and add it to the context so the next large set of answer will not be inconsistent), without any intelligence to it. The whole system will look like a human. But we know that the system is not "living" and "aware", because a "living" or "aware" system has its own opinion, while this system is just generating convincing sentence without seeing any hierarchy or value or meaning in each one.
I would claim that, if you think without introspection (that loop), then there is virtually no self check. I'm not sure what "self check" you see that the brain has. Could you describe this "self check in a line of thought"? How do you perceive the check there? This is a genuine question. It definitely doesn't align with how I think about things. I ponder and talk to myself to iterate verify and test my understanding of my own thoughts.
2 replies →