Comment by gadtfly
13 hours ago
Did it play by looking at screenshots and sending clicks, or was there other mediation/symbolization?
It sounds like it might have been actually played in real time, which would be very important to distinguish.
I have recently seen other harnesses letting agents play real-time games in what seems like discrete time slices, turning eg Portal into something turn-based https://www.youtube.com/watch?v=ruuGXFAmiOE
There's a bot data stream already; it's probably hooked up to that rather than screencap. Yes, I believe these were playing in real time.
It plays from a special API. Some things that are impossible with normal UI are possible with API, like selecting a unit under other units. Invisible units are also reported via the API.