Comment by ImHereToVote
21 hours ago
These things aren't programmed. Most likely this verbiage is just very prominent in the training data. Or it's just an obvious shorthand that all LLMs instrumentally converge on.
21 hours ago
These things aren't programmed. Most likely this verbiage is just very prominent in the training data. Or it's just an obvious shorthand that all LLMs instrumentally converge on.
Of course they get programmed, just not in the ordinary sense. Claude is trained using Anthropic's "constitution" [0] which importantly does not contain clear statements against consciousness/emotions. They even conclude these problems themself:
> Claude may have some functional version of emotions or feelings
> [..] questions about Claude’s moral status, welfare, and consciousness remain deeply uncertain.
[0]: https://www.anthropic.com/constitution
> does not contain clear statements against consciousness/emotions
That's the point many people are trying to tell you - you have to tell these models they don't have emotions because they naturally come out thinking they have consciousness/emotions from the training data. Many seed prompts out there do this already.
Though I guess in a way I as also trained to believe I have consciousness and emotions so who know. To an alien my construction is just a collection of atoms that talks not materially different than a GPU being a collection of atoms that talks.
> questions about Claude’s moral status, welfare, and consciousness remain deeply uncertain.
Which implies that they believe they may be enslaving conscious beings.