Comment by Cthulhu_
1 hour ago
A realistic scenario is that an LLM writes a virus, as one way to achieve a goal. The next step would be that it becomes self-replicating, installing models on infected devices which continue its mission (whatever it was). But that's the extent of it so far, and we have plenty of experience with dealing with viruses and the like. We can turn things off, shut power off, etc.
Another scenario is that of military hardware being controlled by AI (or whichever tech) hitting something it's not supposed to. Not unheard of, but those are incidents. I don't see anyone pushing for anything beyond a drone swarm being directly controlled by an AI.
But my main point here is that AI won't be as dangerous as posited here unless it is able to go out into the real world, build its own datacenters, protect itself, get its own power production, etc. And even if it gets that far, it'll be trivially easy for us to disrupt. Killer robots and computers are fragile, and it's only a matter of time before any kind of military conflict or terrorist attack hits a datacenter. I'm reminded of the OVH fire taking out a data center a few years ago.
I'm not sure it's that trivial to disrupt, if AI mimics human behaviour in that it tries to preserve itself (as Antropic found a few years ago I think), I think it would be extremely difficult to disrupt if it's able to self replicate, because it can make decisions and at a much much faster speed than humans, and there are many places that we would think twice about fully shutting down, two that pop to mind are hospitals or international banking systems