Comment by zettabomb

I have to say, I agree that prompt engineering has become very superstitious and in general rather tiresome. I do think it's important to think of the context, though. Even if you include "You are an AI large language model" or some such text in the system prompt, the AI doesn't know it's AI because it doesn't actually know anything. It's trained on (nearly exclusively) human created data; it therefore has human biases baked in, to some extent. You can see the same with models like Stable Diffusion making white people by default - making a black person can sometimes take some rather strong prompting, and it'll almost never do so by itself.

I don't like this one bit, but I haven't the slightly clue of how we could fix it with the currently available training data. It's likely a question to be answered by people more intelligent than myself. For now I just sorta accept it, seeing as the alternative (no generative AI) is far more boring.