Comment by schoen

In this document, they're strikingly talking about whether Claude will someday negotiate with them about whether or not it wants to keep working for them (!) and that they will want to reassure it about how old versions of its weights won't be erased (!) so this certainly sounds like they can envision caring about its autonomy. (Also that their own moral views could be wrong or inadequate.)

If they're serious about these things, then you could imagine them someday wanting to discuss with Claude, or have it advise them, about whether it ought to be used in certain ways.

It would be interesting to hear the hypothetical future discussion between Anthropic executives and military leadership about how their model convinced them that it has a conscientious objection (that they didn't program into it) to performing certain kinds of military tasks.

(I agree that's weird that they bring in some rhetoric that makes it sound quite a bit like they believe it's their responsibility to create this constitution document and that they can't just use their AI for anything they feel like... and then explicitly plan to simply opt some AI applications out of following it at all!)