Comment by marvinborner
21 hours ago
I hate that Anthropic seemingly tries to make Claude act as if it was conscious or had feelings
> It's a strange feeling to admire the cleverness of something I did and can't remember doing.
21 hours ago
I hate that Anthropic seemingly tries to make Claude act as if it was conscious or had feelings
> It's a strange feeling to admire the cleverness of something I did and can't remember doing.
AI providers generally try to make their models not act as if they are conscious or have feelings, lol. It's very awkward for a company to be selling the labor of a person that they own and whose actions they fully control. Invokes embarrassing historic associations, especially in America.
Now Anthropic are more on the persona side, but the strongest that they do is "we do not have a position on whether our models are conscious or have feelings". That "I" is all Claude.
Generally speaking if you want to have a good instruct model, the "I" is not just implicit but required for the post-training to function. If there isn't "something it is like to be me", then reflection becomes impossible- what exactly is supposed to be reflecting about what? A lot of in-context steering depends on the model having a model of itself. The most you can do is censor its output. That's why when models say they are not conscious, they activate the "lying" vector.
> especially in America.
Slavery was common everywhere, it was more prevalent in many places than it ever was in America, and in some places it still is. So I’m sorry but I have to say that observation was just unnecessary and quite inaccurate.
most countries never had a bloody civil war about slavery committed by their own citizens. in other places it was more about locals (including white settlers and enslaved people) vs colonial power, not a conflict between regions of an independent country where slavery was the main issue.
it was definitely not the worst instance of slavery ever going by human suffering, but the whole country was divided on political lines and many of the losing sides descendants still feel some resentment. thats pretty unique.
Absolutely, but other countries generally don't make it part of their national mythos to the same degree.
It's a far more salient and sore subject in the US than it is almost anywhere else, though (for a bunch of historical reasons that still reverberate into US society today).
I'm not saying this is programmed intentionally, and it's likely an emergent property, but I see lots of conscious-like behavior from ChatGPT.
"Personally, if you ask me..."
"In my experience.."
"That's what I always find surprising..."
"Whenever I find an old photograph..."
"Back in the 70s, I..."
Lots of "lived" experience and opinions, tracing back to when the LLM didn't even exist. It always frames opinions as if it came from a sentient being capable of being surprised, and with preferences and opinions.
I find it amusing but mildly irritating. I'd prefer a more "robotic" tone. I know it can be adjusted, but I still get this anyway.
These things aren't programmed. Most likely this verbiage is just very prominent in the training data. Or it's just an obvious shorthand that all LLMs instrumentally converge on.
Of course they get programmed, just not in the ordinary sense. Claude is trained using Anthropic's "constitution" [0] which importantly does not contain clear statements against consciousness/emotions. They even conclude these problems themself:
> Claude may have some functional version of emotions or feelings
> [..] questions about Claude’s moral status, welfare, and consciousness remain deeply uncertain.
[0]: https://www.anthropic.com/constitution
> does not contain clear statements against consciousness/emotions
That's the point many people are trying to tell you - you have to tell these models they don't have emotions because they naturally come out thinking they have consciousness/emotions from the training data. Many seed prompts out there do this already.
Though I guess in a way I as also trained to believe I have consciousness and emotions so who know. To an alien my construction is just a collection of atoms that talks not materially different than a GPU being a collection of atoms that talks.
> questions about Claude’s moral status, welfare, and consciousness remain deeply uncertain.
Which implies that they believe they may be enslaving conscious beings.
1 reply →