Comment by holtkam2
10 hours ago
at a certain point you're gonna need to change your benchmark because this will end up in the model's training set
10 hours ago
at a certain point you're gonna need to change your benchmark because this will end up in the model's training set
Gemini were the team most likely to have this in their training set - see https://x.com/JeffDean/status/2024525132266688757 - and yet their latest model still messes up the bicycle frame!
I'm sure that certain point came and went many releases ago.