Comment by dom96
15 hours ago
yeah, it performed best for whatever reason. More specifically, it managed to deduce the meaning of the hack statement in the language better than Opus 5.5 did.
15 hours ago
yeah, it performed best for whatever reason. More specifically, it managed to deduce the meaning of the hack statement in the language better than Opus 5.5 did.
How many runs you make for each model?
Do you report an average score or the highest.
2 runs for each task, for each task it's either a pass/fail (the small green/red dots show each task pass/fail)