Comment by criemen
7 hours ago
Thanks! I went back, and had a look at both the contest rules and the highest scoring submission again. Turns out I benchmarked against infinite games, whereas the contest measured a 5min time limit. So, Astra solved a different problem.
Comparing strategies, though, Astra does something pretty different from the highest scoring submission. It doesn't predict the RNG, instead it reacts to the visuals on-screen, planning ahead by estimating velocity of all objects on screen. Unlike the winning solution, it actively flies the ship, whereas the winning solution basically only rotates and teleports. So Astra behaves more like a regular "perfect" player would, rather than one that breaks the PRNG.
It is a bit frustrating that you reported it as being so much better than human made solutions despite not checking even the most basic conditions.
My apologies, and it's too late to edit the parent now.
To be fair, though, I mainly ran this experiment to distinguish models, not to see if they were better than humans or not, so for that purpose it didn't really matter if the basic conditions were the same or not.