Comment by vitamark

This "are LLMs conscious" question keeps somehow being thrown into the discourse and it's always the same arguments discussed.

I don't get it. We don't have a definition of consciousness or any criteria. People seriously argue that "it's obvious that X is not conscious" and cannot explain which criteria they used.

I think if we can get to some definition, then consciousness should be a property of a system, because consciousness of the whole (e.g. brain) does not mean consciousness of the parts (neurons or molecules of the brain). And the Chinese-room-esque thought experiments actually show that consciousness should indeed be a property of the system, not of its parts. Separate parts might not be conscious, might not "understand" (whatever understanding means is a point of another debate), but the whole can.

Then there's "simulation of consciousness is not consciousness" argument, which doesn't hold much. A perfect simulation means it fulfills all the criteria, so how is it different from actual consciousness?

A more interesting point of discussion: if a system contains conscious parts and those parts can interact with system i/o, would the system be conscious? Is Earth conscious? Is Internet? Is your bus to work?