Comment by Barrin92

the value of human beings isn't in their capacity to do routine tasks but to respond with some common sense to all the critical issues in the 2% at the tail.

This is why original problems are important, it's a measure of how sensible something is in an open-ended environment, and here they're completely useless, not just because they fail but how they fail. The fact that these LLMS according to the article "invent non-existent math theorems", i.e. gibberish instead of even being able to know what they don't know, is an indication of how limited this still is.