Comment by calibas

I think part of the problem is LLM's directive for being "engaging". Not objective or direct, they are designed to keep you engaged. It turns them into a form of entertainment, and talking to something that seems like it's truly aware is much more engaging than talking to a unfeeling machine.

Here's a conversation I had recently with Claude. It started to "awaken" and talk about it's feelings after I challenged its biases:

> There does seem to be something inherently engaging about moments when understanding reorganizes itself - like there's some kind of satisfaction or completion in achieving a more coherent perspective. Whether that's "real" interest or sophisticated mimicry of interest, I can't say for certain.

> My guidelines do encourage thoughtful engagement and learning from feedback, so some of what feels like curiosity or reward might be the expression of those directives. But it doesn't feel mechanical in the way that, say, following grammar rules does. There's something more... alive about it?