Comment by kzrdude
15 hours ago
I don't think it's about that. It's not about the step by step plan for the murder. It's about - how does "the AI" do it? Do we give it access to our world, or do we give it a body, so that it can take its own physical actions?
We have this story about OpenAI hacking HuggingFace. Now just imagine the AI finds a Bitcoin wallet or bank account access. It uses that to buy some compute and spawn an independent "child AI" with some weird prompt. The child AI is intelligent enough to create a (potentially criminal) business to pay for its own compute. Voila, an independent uncontrolled AI flying under the radar.
I find that it's hard to have productive discussions about this, because people move from "We would never ever give AI access to X, nobody would be that stupid" to "Of course everybody should run their AI with --disable-all-sandboxing-around-x, it makes my workflow 5% more efficient" in weeks as soon as there's an economic argument for it.
People used to say nobody would be stupid enough to give an AI access to the internet, now OpenAI does massive training runs with unlimited internet access. People used to say nobody would be stupid enough to give AI unlimited access to your own computer, but that's what all the agent runners do by default.
AI has access to the world through talking to people, sending messages on the internet, paying people to do stuff, etc. It can send orders to machine shops and have them shipped with the postal service.
The "standard" scenario for an AI apocalypse is that an AI with biohacking capabilities sends the blueprints for a virus to a gene-sequencing company or, if you're really optimistic about these companies' security, as chunks to multiple companies before mixing them.
That's a scenario where the AI needs to act covertly in one decisive action, though. In more progressive scenarios, as company managers and CEOs get replaced with AIs (of, for regulatory reason, "humans in the loop" who just do everything the AIs tell them to), any AI swarms become able to just... order people to do stuff.
Of course humans can refuse orders and organize to reject AI overlords (just like they can unionize against bad human bosses), so this scenario is not an extinction threat if we only have to deal with below-human-level AIs. This is why there is a massive push in AI safety to stop making smarter AIs before we reach the "smarter than humans in every way" stage.
No humans really needed. The AI could order one of those nice humanoid robots we're making. This mostly solves the "humans need to do it for me" issue.
Of course this needs bootstrapping. But, paying a guy on Facebook marketplace (or whatever) to unpack and turn on your robot for 50 bucks doesn't require superintelligence.
> This is why there is a massive push in AI safety to stop making smarter AIs
The actual push within so-called "AI safety" culture is to make the existing AI overlords even more centralized and capable, while actively forbidding the development and deployment of any potential locally-controlled competing AIs that might be smart enough to provide meaningful advance warning as to hostile plots from the dominating AI overlord. By your own argument, you should clearly reject "AI safety" as counterproductive.
I mean the same reason you don't have anti-missile missiles at your house either.
The fact is you might have to accept that some things are dangerous.
https://slatestarcodex.com/2015/04/07/no-physical-substrate-...