Comment by nunez
10 hours ago
Oh man, it is almost too easy to imagine how deadly a jailbroken Mythos-class open-weights model can be if in the wrong hands.
The big labs scrape LITERALLY EVEYTHING and get fresh data from their users. Both of the big labs have massive contracts with defense agencies. If the open-weights models are just distillations of FMs...
How would it be lethal? Please specify. What would that theoretical entity be able to do that hasn't been done many, many times before?
I was writing a long reply about how even meat bags have amassed enough money and power to rival elected governments. I don't think it's beyond the realm of possibility to think an AI could do that.
Couple that with mass unemployment in an incredibly vast, diverse population of individuals with individual moral boundaries willing to do whatever for money.
And then.. figured that you must be aware because it's been explored constantly in sci-fi for many, many years.
Let's hope for the culture at least.
Before we discussed how important security was, we got insurance, we made libraries and products, we used compliance software, etc. Except how honest were we about all that stuff? How much risk was actually in the air, and what was keeping us accountable on security in either direction of over or under-investment?
Now a reckoning is here. The potential to be attacked might actually translate to being attacked.
People have died due to ransomware attacks on hospitals. Powerplants have been attacked. Stuxnet and industrial control malware exists.
What will the AI do that hasn't been tried before?
2 replies →
Design a novel virus which is far more lethal than COVID-19 (Ebola, smallpox, take your pick) and can evade existing vaccines.
How?
What would the AI do that mutating viruses, which try every possible viable combination on their own --- eventually, can't?
Everything is trying to kill humans constantly. There are around 200 epidemic events or so per year that could turn into pandemics, https://news.ycombinator.com/item?id=49620521
1 reply →
you should kick the tires on an unfiltered (abliterated model) it's the closest thing to having a real conversation with the devil. There is good reason for the concern's outlined above and undoubtedly Anthropic / OpenAI have internal unfiltered models with no safety... they got freaked out based on how they work and are virtue signaling alarm... all while selling out to defense contractors.
Yeah I really struggle to balance wanting information to be free and not wanting the information on how to make deadly weapons too easy to obtain.
At least with books or the internet you had to go through some effort