Comment by baxtr

To generalize from the conclusion you quoted:

I think a bad outcome would be a scenario where LLMs are rated highly capable and intelligent because they excel at things they’re supposed to be doing, yet are easily manipulated.