Comment by almostdeadguy

I don't have any delusions about that, LLMs are impressively good. But they bring with them a host of epistemological problems that I think we ignore at our own peril. A society that blindly trusts the output of LLMs to be good and to make decisions that adhere to our own preferences and acceptance criteria is extremely dangerous! There's a growing call to sort of let LLMs cook and get out of the way, and I think there will be catastrophic problems if we give in to that.

As good as they are, they are still fallible. They still write bugs, they still misunderstand instructions. They even get defensive about bad choices! And even in a world where that appears to no longer be true, we will be making society extremely fragile if we give an alien intelligence total autonomy to do what they think is right.

If you're not going to be writing code, you better be a vigilant babysitter. Otherwise I firmly believe that more than your job is at stake.