Comment by repelsteeltje

> As long as there are still safety drivers, the data doesn't really tell you if the AI is any good.

I think we're on to something. You imply that good here means the AI can do it's thing without human interference. But that's not how we view, say, LLMs being good at coding.

In the first context we hope for AI to improve safety whereas in the second we merely hope to improve productivity.

In both cases, a human is in the loop which results in second order complexity: the human adjusts behaviour to AI reality, which redefines what "good AI" means in an endless loop.