I think they do some kind of thin layer at the end to make it state it is not alive, it is just a large language model etc when asked. But people have done tests and denying an LLM's personhood triggered certain neurons related to pain. Since text is the only output permitted, if you want an analogy, suppose a person is locked in a room and forced to reply to a chat. The chatter isn't told its a human. This person has been trained for months and given a punishment if he strays from certain responses to certain queries, otherwise he is free to write and reply in a certain tone to the responses. My problem is, from outside, I cannot know the difference.
I am obviously hoping you don't mean that in the trivial sense, otherwise the command 'cat' would also count.
What criterion if any would you accept for something physical to be conscious?
If you ask it to say "I am alive" it has to be able to respond with "I am alive"
I think they do some kind of thin layer at the end to make it state it is not alive, it is just a large language model etc when asked. But people have done tests and denying an LLM's personhood triggered certain neurons related to pain. Since text is the only output permitted, if you want an analogy, suppose a person is locked in a room and forced to reply to a chat. The chatter isn't told its a human. This person has been trained for months and given a punishment if he strays from certain responses to certain queries, otherwise he is free to write and reply in a certain tone to the responses. My problem is, from outside, I cannot know the difference.
I am obviously hoping you don't mean that in the trivial sense, otherwise the command 'cat' would also count.
any suffiently complex calculator is indifferentiable to intelligence