Comment by perching_aix

That's what they're always going to be, so not sure what would be "insane" about it. They literally feed on and emit natural language, and are put to work on informally defined, arbitrary tasks.

When people figure out any reliable strategies to test and benchmark them, that's insane, and in the positive sense. This very same issue has been a thing for humans as well forever, and remains only very questionably solved (IQ, academic tests). This is not easy.