Comment by SlinkyOnStairs

> Why do you need a specific organization of molecules for a phenomenon similar to consciousness to arise?

They would have to be similar because the argument is that they are similar; "AI is like the human brain" only holds true if it actually is like the human brain, not merely a superficial resemblance.

What I'm describing in that prior comment is how a lot of people drastically simplify the resemblance in order to make it feel true; That the lack of a Jesus Christ coming down from the heavens to tell everyone they have immaterial souls that the computers don't makes the comparison more true.

> We don’t really know or understand the relationship between the raw physics and again what we consider consciousness, so it’s simply a statement of “we can’t refute that these systems exhibit something similar” because we don’t know enough to refute that

Therein lies the conflict: "You can't prove it's not conscious" is an unfalsifiable statement. You can't engage with the argument because it's proponents will always claim victory, often with their own interests at play. All concerns about "superintelligence" or the long term ethics of "when do our robots become sufficiently intelligent that they'd be slaves?" have been subsumed into the AI marketing machine. However sincere one might try to address the issue, they look like a Sam Altman stooge by association.

It's like claiming the quantum fluctuations inside a pet rock as a "consciousness", even if observed directly any measurement of random noise can still be dismissed as "nuh-uh it just takes billions of years to have a thought".

More practically with current AI systems, we can look inside them pretty well and there genuinely is nothing there. Standalone LLMs are purely feed-forward systems. Their failure modes show that they perform no meaningful thought or world modelling during inference. They're just language models.

The reasoning and agentic systems are even easier to introspect. We know how they work, we can look at the full prompts & context they operate on. There is nothing there.

This is what sets AI apart from animals, which are given the benefit of the doubt on their intelligence.