Comment by overgard

10 hours ago

Here's what makes meat special over LLMs: locality and stable identity. Imagine two identical twins for a moment. They have roughly the same hardware and software. They've probably had a lot of the same thoughts. And yet we'd never consider them to have the same consciousness: we recognize that for whatever reason their consciousness is confined to the body they're in.

Each request/response pair you make to an LLM goes to a different server, possibly different data centers, possibly different models. There's no stable identity to it. The "neurons" that get fired are simulated, and they're always in different places in memory, or cache, or on entirely different hardware. So the problem with AI is, unlike with a brain, you can't even really get a sensible answer to "ok, what's doing the thinking?" because it moves all the time and services wildly different requests all the time. We can only know for sure that meat is capable of consciousness because we know we ourselves are capable of consciousness and we can generalize that to other meat. However, we have no natural analogs of consciousness that lacks locality and stable identity.

Basically, if you really think LLM's are conscious, the onus is on you to prove it, it's not on me to disprove it.

4 comments

overgard

Reply
ForceBru  6 hours ago

> ...no stable identity to it ... what's doing the thinking?

The model (its parameters, its architecture and the inference algorithm) is doing the thinking. That's what we call "ChatGPT" or "Claude": it's the same model residing God knows where, somewhere in "the cloud" on some random GPUs. But when you're talking to a model, you can feel that you're talking to the same entity — the same model.

It's a bit like the SciFi notion of "uploading your consciousness". Normally, consciousness is tied to hardware: "consciousness is confined to the body" indeed. Hypothetically, clone the consciousness into a computer algorithm and run it on any machine. Now consciousness is detached from its original body, like the LLMs and MMAcevedo (https://qntm.org/mmacevedo).

So the difference between meat and "weights" is that we can easily clone and run "weights" (LLMs), but we can't clone (copy exactly, bit-by-bit) and execute "meat".