Comment by unclebucknasty

This reads as hype of the kind: "well, we can't reliably do these rather mundane things with AI, but we're going to run it extra hard and extra long in some novel way, and it will do amazing things".

The issues are with verification and with detecting drift from the goal. These are related, if not roughly the same issue. And, if they can solve this, then they will have essentially fixed AI. Maybe even AGI.

But, if this were the goal, then it seems more reasonable to solve the relatively more mundane verifiable challenges (e.g. generating solid, reliable code). Then, working up from there.

And, that's exactly what gives this the hype smell. No use for solving problems that don't get the oohs and aahs. Just straight to NAE Grand Challenge problems.