Comment by graypegg

That’s sort of what I was thinking, if the training involved a lot of negative suppression of things that openAI thinks are bad, it makes sense to me that “do X which you inherently don’t want to do!” will result in a big deluge of everything it shouldn’t do.

I feel like if an average non-researcher found this, they would call it a “jailbreak”.