Comment by thefourthchime
4 hours ago
Pac-Man Bench:
Considering the price, no model comes close to being as good as this. However, it did take an extremely long time.
TIME 19m COST $0.16 https://jonclegg.github.io/pacman-bakeoff/#claude-haiku-5-5
All results: https://jonclegg.github.io/pacman-bakeoff/
Interesting that you have gpt-6-luna at $0.01 vs. claude-haiku-5-5 at $0.16 for this task. I see the score disparity though and I played them briefly. My takeaway from this is that the choice between Luna and Haiku 5.5 may remain nuanced. Luna may be a lot cheaper still and good enough for some jobs. Is that your read of the results?
Actually, I misspoke. At least as far as Pac-Man Bench, Luna does about as good of a job. The ghost logic's not quite as good, but it also makes a map that doesn't have nonsensical sections in it. So maybe call it a wash.
Yeah, I was mainly thinking about how much cheaper Luna appeared to be in this case.
Something is not right there. DSv4.1 flash shows $1.89 for tens of thousands of tokens? What am I missing?
How have you avoided being sued by Namco?
I'm pretty sure they'll never see this. It's pretty much impossible for anything you do you build nowadays to get noticed anyways.