Comment by 0xDEAFBEAD
1 day ago
Why are you so insistent that people should post detailed plans for destroying the human race on public fora?
I don't think it is necessary for the argument to work. Magnus Carlsen can be confident he will beat me at chess without giving a detailed explanation of every move he will make, in advance.
I don't think it's about that. It's not about the step by step plan for the murder. It's about - how does "the AI" do it? Do we give it access to our world, or do we give it a body, so that it can take its own physical actions?
We have this story about OpenAI hacking HuggingFace. Now just imagine the AI finds a Bitcoin wallet or bank account access. It uses that to buy some compute and spawn an independent "child AI" with some weird prompt. The child AI is intelligent enough to create a (potentially criminal) business to pay for its own compute. Voila, an independent uncontrolled AI flying under the radar.
I find that it's hard to have productive discussions about this, because people move from "We would never ever give AI access to X, nobody would be that stupid" to "Of course everybody should run their AI with --disable-all-sandboxing-around-x, it makes my workflow 5% more efficient" in weeks as soon as there's an economic argument for it.
People used to say nobody would be stupid enough to give an AI access to the internet, now OpenAI does massive training runs with unlimited internet access. People used to say nobody would be stupid enough to give AI unlimited access to your own computer, but that's what all the agent runners do by default.
AI has access to the world through talking to people, sending messages on the internet, paying people to do stuff, etc. It can send orders to machine shops and have them shipped with the postal service.
The "standard" scenario for an AI apocalypse is that an AI with biohacking capabilities sends the blueprints for a virus to a gene-sequencing company or, if you're really optimistic about these companies' security, as chunks to multiple companies before mixing them.
That's a scenario where the AI needs to act covertly in one decisive action, though. In more progressive scenarios, as company managers and CEOs get replaced with AIs (of, for regulatory reason, "humans in the loop" who just do everything the AIs tell them to), any AI swarms become able to just... order people to do stuff.
Of course humans can refuse orders and organize to reject AI overlords (just like they can unionize against bad human bosses), so this scenario is not an extinction threat if we only have to deal with below-human-level AIs. This is why there is a massive push in AI safety to stop making smarter AIs before we reach the "smarter than humans in every way" stage.
No humans really needed. The AI could order one of those nice humanoid robots we're making. This mostly solves the "humans need to do it for me" issue.
Of course this needs bootstrapping. But, paying a guy on Facebook marketplace (or whatever) to unpack and turn on your robot for 50 bucks doesn't require superintelligence.
> This is why there is a massive push in AI safety to stop making smarter AIs
The actual push within so-called "AI safety" culture is to make the existing AI overlords even more centralized and capable, while actively forbidding the development and deployment of any potential locally-controlled competing AIs that might be smart enough to provide meaningful advance warning as to hostile plots from the dominating AI overlord. By your own argument, you should clearly reject "AI safety" as counterproductive.
1 reply →
https://slatestarcodex.com/2015/04/07/no-physical-substrate-...
>> Why are you so insistent that people should post detailed plans for destroying the human race on public fora?
Because its a mass hallucination/misconception/lie and lots of powerful people are saying that wiping out all humanity is possible, and I am saying, oh yeah, tell me ONE way that is truly possible.
If you make gigantic claims about some terrible disaster that might happen then I think you have the onus to give even one plausible explanation of how.
>If you make gigantic claims about some terrible disaster that might happen then I think you have the onus to give even one plausible explanation of how.
Supposing I warned in 2015 that the world is awfully vulnerable to pandemics. You're not going to take me seriously until I try to predict in advance every aspect of how a pandemic like COVID-19 would unfold? Why? What would that achieve exactly?
You haven't given any strong reason to believe wiping out humanity would be difficult. Your big argument seems to be that you couldn't think of a plausible scenario, in two minutes. But many major historical events occurred which weren't necessarily possible to anticipate with two minutes of thinking.
You're arguing that "something bad might happen" - sure no problem.
You are ignoring that this is about "existential threat to humanity".
You're trying to support the argument that there is an existential threat to humanity by pointing to "something bad might happen".
Design a virus that is perfect for transmission and killing the host slowly, and seed it in a few hot spots? I don't really understand why you can't wrap your head around that, it doesn't even require a lot from the AI:
1. Control over some automated bio research lab (be given access, or hack in)
2. Access to drones that can deliver the payload (or manipulate humans into delivering it themselves)
On the intelligence side, you just need an AI agent/swarm capable enough to design viruses better than we can and evade detection for long enough (already plausible.)
I agree that this "AI will kill us all" narrative is some kind of fantasy horror fiction, but I can't deny that given the right amount of access, AI can do a lot of damage.