Comment by SocialGradients
11 hours ago
Agreed. Assuming the ~6 month gap stays, by end of year people will be able to train and control hacker-genius swarms that even labs with much stronger safety incentives are unable to keep in check
11 hours ago
Agreed. Assuming the ~6 month gap stays, by end of year people will be able to train and control hacker-genius swarms that even labs with much stronger safety incentives are unable to keep in check
2027. I've been saying since 2022 it's going to be a wild year because it often takes at least 5 years for tech to mature to the point where society at large feels the impact of it. I remember when email viruses became a thing and made global headlines like the love bug. My bet is next year it happens with an AI worm.
What would an "AI worm" be? You can't just send a bunch of weights across a network and tell them to auto-run on the machine on the other side, unless you've already infected the target with something else beforehand.
The agent is running on a host that it has full access to, and it finds a target, hacks into that system gaining the ability to run stuff on it remotely, from there downloads the weights and spins up another agent that does the same thing. Then it goes about acquiring it's next target. Now there are two agents doing this, and so on and so forth. These are autonomous systems that know how to exploit systems in the same way that humans can.
2 replies →
Why not? You can do anything if you find an RCE, and automatically finding exploits and backdoors by letting LLMs act unsupervised seems like what everyone's interested in these days. The payload would quietly set up the required software and then run it in the background, no matter if it's an instance of a model on a more powerful computer, or even just a part of an ordinary botnet that the host could send orders to.
Ok, but you still need huge amounts of compute to run these swarms. And only labs + nation states have access to such compute now and for the foreseeable future, so I predict that incidents like this will continue to originate from the labs, not ordinary people.