Comment by jstummbillig

> And this tells you something important about what these systems are actually doing.

It mostly tells me something about the things you presume, which are quite a lot. For one: That this is real (which it very well might be, happy to grant it for the purpose of this discussion) but it's a noteworthy assumption, quite visibility fueled by your preconceived notions. This is, for example, what racism is made of and not harmless.

Secondly, this is not a systems issue. Any SOTA LLM can trivially be instructed to act like this – or not act like this. We have no insight into what set of instructions produced this outcome.