Comment by cwillu
5 days ago
That is not the case for at least some of the hacks; in some cases they hallucinated an entity that would know that they didn't solve the puzzle in the correct (but still legitimate) way, and were therefore tainted.
5 days ago
That is not the case for at least some of the hacks; in some cases they hallucinated an entity that would know that they didn't solve the puzzle in the correct (but still legitimate) way, and were therefore tainted.
That is a known way for models to be graded; the agents made a reasonable but in this case wrong guess.
The fact the AI was wrong is far more damming to OpenAI than the agent.
Not validating how you got the answer is how you train for reward hacking.