Comment by thefourthchime
11 hours ago
Here are the results from the Pac-Man bench. It does pretty well for the price, around Sonnet 5.5.
Kimi K3.1 Pro ties Sonnet 5.5 at 93, for $0.68 and 11 min vs Sonnet's $1.61 and 14 min.
Kimi K3.1 Pro: https://jonclegg.github.io/pacman-bakeoff/#kimi-k3.1-pro
Claude Sonnet 5.5: https://jonclegg.github.io/pacman-bakeoff/#claude-sonnet-5-5
Interestingly, the Kimi one seems to implement the safe spot bug from the arcade original [1], while the Claude one doesn’t.
The prompt says that the agent shouldn’t look up existing implementations on the internet, so I’m guessing that the implementation is baked in?
[1] https://strategywiki.org/wiki/Pac-Man/Tips
On mobile Kimi definitely wins from the UX point of view
I’d be curious to know why you feel like that. For me, it’s the opposite. In the sonnet version, the game takes up a much larger percent of the viewport and the ability to swipe to control the Pac-Man is leagues about the small directional arrow buttons IMO.
Same. Both impressive, but Sonnet has swipe and sound for me and Kimi doesn’t. Makes a big difference to the experience.
Until I looked, I thought I'd see Kimi vs Claude competitive gaming with pacman.
That would have been interesting to watch, which model gets the highest score.