Comment by baranul

And that's the trap. Relying on a "perfectly" crafted test or a LLM to verify what is unknown. Yes, LLMs can be very useful, but perhaps it's best for us to be realistic.

> making the computer check its own work

Kind of like the Spanish teacher telling his students they can grade their own tests, then being surprised that Billy was always giving himself 100%, when he's nowhere near that bright or fluent.

It wouldn't be so bad, if people were more upfront with being unsure or made it clear they were extrapolating from smaller and limited data. But usually, like many of these unusually cocky LLMs, what is too often reported to the public is "perfection" and many inconvenient truths "swept underneath the carpet".