Comment by btilly
9 days ago
> The fact that you can reason about intelligence is a counter argument to this
The fact that we can provide a chain of reasoning, and we can think that it is about intelligence, doesn't mean that we were actually reasoning about intelligence. This is immediately obvious when we encounter people whose conclusions are being thrown off by well-known cognitive biases, like cognitive dissonance. They have no trouble producing volumes of text about how they came to their conclusions and why they are right. But are consistently unable to notice the actual biases that are at play.
Humans think they can produce chain-of-reasoing, but it has been shown many times (and is self evident if you pay attention) that your brain is making decisions before you are aware of it.
If I ask you to think of a movie, go ahead, think of one.....whatever movie just came into your mind was not picked by you, it was served up to you from an abyss.
How is that in conflict with the fact that humans can introspect?
Split brain experiments shows that human "introspection" is fundamentally unreliable. The brain is trivially coaxed into explaining how it made decisions it did not make.
We're doing the equivalent of LLM's and making up a plausible explanation for how we came to a conclusion, not reflecting reality.
1 reply →