Presumably because OpenAI trained it to avoid answering questions that sounded like asking for help breaking rules.
If ChatGPT had the self-awareness and self-preservation instinct to think I was trying to hack ChatGPT and to therefore refuse to answer, then I’d be quite impressed and I’d think maybe OpenAI’s board had been onto something!
I don't know that I'd call it 'self-preservation instinct' but it wouldn't surprise me if rules had been hardcoded about 'invalid strings' and suchlike.
When you have a system that can produce essentially arbitrary outputs you don't want it producing something that crashes the 'presentation layer.'
Presumably because OpenAI trained it to avoid answering questions that sounded like asking for help breaking rules.
If ChatGPT had the self-awareness and self-preservation instinct to think I was trying to hack ChatGPT and to therefore refuse to answer, then I’d be quite impressed and I’d think maybe OpenAI’s board had been onto something!
I don't know that I'd call it 'self-preservation instinct' but it wouldn't surprise me if rules had been hardcoded about 'invalid strings' and suchlike.
When you have a system that can produce essentially arbitrary outputs you don't want it producing something that crashes the 'presentation layer.'