Comment by docheinestages
5 days ago
I mean, I'm interested to know if the frontier models also get to see all questions at once. Then it's more fair game than if they just see one question at a time.
5 days ago
I mean, I'm interested to know if the frontier models also get to see all questions at once. Then it's more fair game than if they just see one question at a time.
They look at each problem individually.
As the ARC AGI guidelines [1] state, 'a core design principle of ARC-AGI is that the test taker must not know what the test will be.'
[1] https://arcprize.org/policy#dataset-security