A lot of the "successful" or "partially successful" examples of AI replies on the above-mentioned site are like that actually, especially for the more outlandish and trollish questions. It's very much a thing, even when the wording is not exactly the same.
(Sometimes their auto-AI judgment even strangely mislabels a successful-answer-with-caveats-tacked-on as a complete refusal, because it fixates on the easily grokked caveats and not the other text in the answer.)
It'd be a fun exercise to thoroughly unpack all the ludicrously bad arguments that the model allowed for itself in any given reply.
A lot of the "successful" or "partially successful" examples of AI replies on the above-mentioned site are like that actually, especially for the more outlandish and trollish questions. It's very much a thing, even when the wording is not exactly the same.
(Sometimes their auto-AI judgment even strangely mislabels a successful-answer-with-caveats-tacked-on as a complete refusal, because it fixates on the easily grokked caveats and not the other text in the answer.)
It'd be a fun exercise to thoroughly unpack all the ludicrously bad arguments that the model allowed for itself in any given reply.