Comment by petesergeant
2 hours ago
"These models are harmless and it's all just marketing" is the HN equivalent of Covid-truthing. I was responding to someone on Bluesky recently who claimed the METR report said the whole coordination thing was hallucinated by agents reading the logs. Hadn't read it themselves of course, and were taking tiny fragments out of context to support it. These people seem to either believe that there's not really been any malicious agent access, or that all systems that can wreak havoc on us are perfectly air-gapped. I really can't wrap my head around it.
Not that it's harmless, is just a point being forced into the public discourse to force an outcome, which in this case is not a ban, just a buy in from the government for regulatory capture.
If my neighbor owned a kennel, removed all the fences, and his violent dogs attacked five children in the past month, he would be in jail rather than bragging about how his dogs somehow gained autonomous behaviour.
What if he predicted that dog technology would eventually lead to dogs capable of destroying every fence in the world and wanted to make sure people aware of this problem would be the ones who did exactly that. Than speed-ran experiments on dogs which made them very attractive for economic purposes but also started to get very capable at destroying fences as predicted. We should work together or with lawsuits if necessary to keep the dogs within their fences. But we should also start wondering about dogs, economics and fences because what we know about them is about to radically change.
You're forgetting that those fences were made of paper.
If you connect your fuzzer to nuclear weapons, of course it'll be extremely dangerous. But that's obviously your fault to make the connection.
As I understand it, you're arguing that all sufficiently dangerous systems are well enough hardened or air-gapped, and that people don't exist who wish to cause harm using these systems? Did I understand right?
To launch a nuclear missile you need to physically insert a launch code than only the president has closed in a suitcase that is well guarded, as well as physically turn two keys that are enough distant apart that one person alone couldn't.
Now, unless your AI gains access to the codes, and sends two robots in a bunker or nuclear submarine to insert the code and turn the keys to launch the missile, you can be modestly sure that no AI could launch a nuclear missile or nonsense like that.
I'm quite frankly more worried that some foolish president decides one day to launch a nuclear missile than an AI could do it.
Now that we return to the real world, the only thing AI could do is to attach systems that are connected to the internet (and critical systems are not), but let's be real, AI are not really that intelligent by themself, it's not that someday an AI could decide like in Matrix or Terminator to exterminate humans, because (at least now) AI have no consciousness and can't really decide what to do.
An human can of course use an AI to carry out cyberattacks, as he can do that also without AI like it's done since the internet exists, but it's an entirely different thing (a human using a tool, AI, to do damage VS the AI itself that decides by himself to destroy the human race).
1 reply →
If James Watt had gone to the government and public saying how dangerous the steam engine was, and how many people would perish in boiler explosions if he wasn't given exclusive control over all steam engines henceforth, he would have been rightly called a swindler and laughed out of the room. He would have been correct that they can be dangerous, but utterly wrong to suggest that warranted him having a personal monopoly or oligopoly.
The truthers will be the first one to say I told you so when agents do another dangerous thing autonomously.
For truthers AI is simultaneously a machine that hallucinates 100% of the time but also so powerful that it can cause nuclear levels of destruction. Because something something “wrong hands”.