Comment by kennywinker

I responded to this idea more in detail here: https://news.ycombinator.com/item?id=49686196

But tl;dr: even if LLMs do have the intelligence to understand the consequences of their actions, there is no way for them to experience consequences.