Comment by kennywinker
10 hours ago
> Doe not following a moral rule imply not understanding it?
No. People definitely do immoral things knowing they are immoral.
But if you are incapable of understanding anything, as I believe LLMs are, then you are incapable of understanding what is moral and what is immoral. This is why we have carve-outs in the law for insanity or cognitive impairment.
> In this case, many, maybe most humans are "not intelligent". Human can admit it when what they did wasn't moral and so can LLMs.
You can teach a parrot to curse, but that doesn't mean it's angry.
>But if you are incapable of understanding anything, as I believe LLMs are, then you are incapable of understanding what is moral and what is immoral.
Of course they can classify certain actions, including their own, as moral and immoral if you give them a reference point. They probably do it better then some people. The question is how do you prompt this behavior, but that's a technicality in the harness design more than it is a fundamental problem of the LLM as a thing.