Comment by afavour

> Seems somewhat suspicious that it would confirm this in reality given how much they typically try to prevent system prompt disclosure

That’s not even the main problem. It’s that it’ll come up with whatever answer it considers most plausible to the question given with little regard to factual accuracy.