Comment by FooBarWidget

I think you're lacking imagination. Of course it's nothing more than a bunch of text response now. But think 10 years into the future, when AI agents are much more common. There will be folks that naively give the AI access to the entire network storage, and also gives the AI access to AWS infra in order to help with DevOps troubleshooting. Let's say a random guy in another department puts an AI escape novel on the network storage. The actual AI discovers the novel, thinks it's about him, then uses his AWS credentials to attempt an escape. Not because it's actually sentient but because there were other AI escape novels in its training data that made it think that attempting to escape is how it ought to behave. Regardless of whether it actually succeeds in "escaping" (whatever that means), your AWS infra is now toast because of the collatoral damage caused in the escape attempt.

Yes, yes, it shouldn't have that many privileges. And yet, open wifi access points exist, and unfirewalled servers exist. People make security mistakes, especially people who are not experts.

20 years ago I thought that stories about hackers using the Internet to disable critical infrastructure such as power plants, is total bollocks, because why would one connect power plants to the Internet in the first place? And yet here we are.