Comment by orlp
1 day ago
I'd really like some clarity on what that means. E.g. DeepMind has 'cheated' with this in the past, claiming that AlphaZero only took 4 hours to reach super-human chess levels while conveniently leaving out the fact that it was 4 hours x 5000+ TPUs. Sure it's impressive that it only took 4 hours wall-clock but it's very misleading as to cost.
Can we get a number in Blackwell GPU-hours, kWh, or some other compute-scaled metric?
Depends what your metrics are. If you suddenly found a way to have 9 women make a baby in one month that is huge.
I'm not denying that, but I'd still like to know what that cost.
They did say that. "3 hours of ChatGPT Pro thinking compute"
Yes, what does that mean?
It means the level of effort that a ChatGPT Pro plan summons when thinking. For 3 hours.
If folks are going to analyze this claim with a critical eye, I'd be zeroing in on "average" rather than acting like this measurement is somehow unclear.
>OpenAI has 'cheated' with this in the past, claiming that AlphaZero [...]
That doesn't sound right
Oops, edited.