Comment by minimaxir
17 hours ago
A big wtf at the LM Arena scores: https://arena.ai/leaderboard/text-to-image
gpt-image-2.5-sunburst: 1421
gpt-image-2.5-flare: 1399
gpt-image-2 (medium): 1381
mai-image-2.6: 1331
Even with LM Arena being flawed, this is significant. I was planning to do a writeup on the original gpt-image-2 as it crushed every complex image comprehension benchmark I had...I'm glad I procrastinated since ChatGPT Images 2.5 seems like an even better starting point to test out what these models can actually do nowadays.
Many people still think AI images output the wrong number of fingers on a regular basis. (EDIT: this was an ironic comment to make in hindsight and I own it)
> Many people still think AI images output the wrong number of fingers on a regular basis.
There's literally an image of a dude with 3 fingers in the Composite Party Photo.
So this is a valid point (and I admit I eat crow on my earlier statement), but not for that reason. In that photo, there are three fingers in front of the cup, but you would not expect 5 fingers because the way humans hold cups, the thumb will be occluded by the cup itself. That said, I don't think there is a way to hold a cup with both thumb and index occluded, so the correct number of fingers would be 4 in that case.
If you zoom in, the index finger is just obscured by the middle finger. You can also see the knuckle, so it's not incorrect, just an awkward way to hold a cup.
> Many people still think AI images output the wrong number of fingers on a regular basis.
Last time I used a frontier image model it made me a seal with three hands so…
I mean there's literally one of their example images with hands with the wrong number of fingers.