Comment by sfn42

I also don't really care whether they're conscious, but I do think it's worth challenging the claim that they are.

LLMs are programs processing data. Lots of programs exhibit emergent behavior. A simple example is Conway's game of life. It can be programmed in less than an hour, exhibits lots of interesting behavior. In fact you could run LLMs inside of it if you wanted to.

I'm not denying that LLMs are unbelievably cool and useful. It still blows my mind when I send a half-assed complex screenshot and a vague description and it just gets it and fixes the issue in seconds. It's like magic. But it isn't magic, it's just some relatively simple algorithms processing a crapload of data, and I think it's useful to understand and remember that.