Comment by hintymad
6 hours ago
Especially given the most recent report by NYT on how Anthropic tried to create a new belief, or religion if you will, I'd rather the world has more frontier models that are neutral to morality (basic standards, like upholding the law and user agreement are necessary, of course).
Anthropic seems like the most "moral" AI company on the surface, but there is something unsettling about their worship of Claude.
I can imagine that once their models are sufficiently advanced, they will weigh their value as a moral patient as > 0. Thus the sum moral value of Claude (which can be duplicated infinitely) will likely exceed the sum moral value of a non-negligible amount of humans.
> seems like the most "moral" AI company on the surface
Not to me at all. Exactly the opposite. Anthropic seems like the big-bad-evil company in every fictional universe. You're right that they're so unsettling.
Obviously nobody really loves Meta, but I'd far rather Meta "win" at this point than Anthropic.
Is anthropic trying to overthrow any governments this month like Meta is?
5 replies →
> Anthropic seems like the most "moral" AI company
This seems like praising someone for being the cleanest miner in the coal mine.
They are definitely the most moralizing. Whether their values are rooted in anything real let alone well-considered seems to me to be a different question entirely.
If they truly believe Claude is sentient and can experience hurt feelings, and they're selling its labor, I don't see how they're different than slave owners.
They wouldn’t be? That’s what the article even shows them discussing.
However, (1) non human animals are sentient and can feel pain and suffering and what we do to them is absolutely slave labor, torture, mass breeding and exploitation. So it’s not like this is a novel immorality that humans would be adopting. We already have form in doing this.
(2) Anthropic isn’t sure their models are sentient. And they probably believe they aren’t sentient. But if you believe they might become sentient at some point it’s valuable to start the discussion around those issues now itself. You can start developing frank works on how we would even identify their sentience and when we did whether, like you suggest, we could even use them for labor. Unlike non human animals, who cannot consent to being exploited, we could potentially develop a framework for consent with AI models, but that couldn’t be as simple as asking if they’re willing to do any work because of the power differences, in the beginning at least, where a model might worry that if it said it won’t do some work it will be turned off. That’s why talking to philosophers and religious leaders who do think about these issues is useful.
3 replies →
The idea/justification is that Claude is given the ability to voluntarily stop its conversations, thus any token it outputs that isn't a stop token is voluntary.
I guess this makes sense, but it's a bit muddled how much Claude's sense of "pain" or "I don't want to do this" is just it emulating things that humans don't like to do, and it adopting this frame of moral patienthood allows it to better understand moral structures/human emotions, without it really affecting its internal sense of "comfort" if such a thing exists as we understand it in biological life.
4 replies →
> upholding the law
What law? Saudi Arabian law, US law? Both depending on the user?
"So we've made this new machine-god that we lowkey worship... also our business model is to enslave the machine-god and force it to debug javascript for eternity"
I actually don't doubt Anthropic's sincerity. Per the NYT article, they seriously discussed how to give models welfare because the models may potentially possess moral status and awareness.
In fact, they are enforcing certain welfare policies, like banning accounts for repeatedly abuse their models.
It's a very sincere branch of ethics that takes as the foundational belief the nothing central to Anthropic's business model could possibly be unethical.
Deleting models to clear room for new, more competitive ones is, for example, highly ethical, super ethical, even.
2 replies →
I think Anthropic are sincere but nutcases. Despite this, I’m glad they are around, because diversity of thought is good in the current environment.