Comment by Terr_

IMO we must take it a step further: In this context "the LLM" we're all automatically thinking-of doesn't exist, it is a fictional character we humans "see" inside a story being acted-out or read to us. (In contrast, the real-world LLM is an algorithm in a basement constantly taking documents and making them slightly longer based on trends detected in all documents.)

Therefore "the LLM can't feel shame" is true in the same way that "CyberDracula thirsts for the fluids of the innocent." Good news: Vampirism doesn't exist! Bad news: Curing Dracula is impossible, because the patient doesn't exist either. The target mind we wanted to make smarter or kinder is just a trick of the light.

The best we can do is change the generator process, so that the next story instead contains a different new character also named after Dracula (or a brand of LLM) except with a different backstory and some dialogue that somehow better-fits the expectations of the humans who latch onto a story.