Comment by RandomLensman
2 hours ago
I didn't say it doesn't happen, but that it is a low probability. And we have ways to reduce probabilities in critical areas.
There is no omnipotent AI currently (and there might never be) and I don't see why with current AI it only needs to happen once.
They don't need to be omnipotent, and they're already human-or-superhuman at persuasion: https://arxiv.org/html/2411.06837v2
This may just be that humans find long arguments more persuasive than short ones, obviously LLMs can do that easily, but the outcome is I think more important than the mechanism.
That is about persuasion with evidence on various topics, not about persuading people to abandon safty protocols and processes and highly policed settings.
Yes, many things could happen, but again, that failure is possible is not a reason to do implement processes etc. I don't see why hypotheticals should stop addressing actuals.