Comment by kennywinker
6 hours ago
I'm confused why the worry about LLMs that will answer "how do I build a pipe bomb". That information is easily available other places. The anarchist cookbook has been around and available for 55 years, and yet pipe bombs are not going off all around us.
Any sufficiently determined individual can buy mac mini, put it under their bed, configure outside proxy via some random internet address and prompt "iterate on websites in the CT logs, one by one, try to find vulnerabilities, if you did - encrypt their data and blackmail them for this bitcoin address". And it'll work, day and night. Abliterated GLM 5.3 is much smarter than average software developer, they know a lot about information security, they can use any available exploits, they can find novel vulnerabilities and they won't say "no". This is dangerous for an average IT system which never encountered nothing worse than some wordpress GET requests.
It's not the end of the world. But future will be rough.
> It's not the end of the world. But future will be rough.
Short term you’re probably right, but longer term is the realm where nation states will start to police the avenues of attack.
This is what will lead to govt needing to attach an actual ID your network connection.
I think it’s a bit like frontier development (like the US “Wild West”). You rob a bank because there’s no one to stop you, and even if you do get identified you can travel enough distance to regain anonymity. Application of legal recourse eventually caught up (as it will here), and the growing pains will certainly make things suck for all of us.
Even in China you can find a way out and build a tunnel. And once you built a tunnel to any outside server, you can jump into another server. So three jurisdictions and your target is fourth. Imagine untangling the links. Police won't do that. Not for some small-sized business anyway. I don't see how you can prevent something like that.
1 reply →
Glm 5.3 won't fit on a mac mini. The barrier to entry to host something scary is what, like $10k? 20k?
Moore's law still holds I reckon.
Even it's $10,000 to run today (FWIW, the featured article cites the M5 Mac Studio with 256GB unified memory going for $9,500 as "good enough to host something scary"), in a couple years it'll be like $2k to run, and in another couple after that, you'll have used $200 dollar smartphones capable of running a model powerful enough to do serious damage.
1 reply →
Wouldn't it just as easy to do this on the defense side as well then?
No because on the defense side you need multiple layers of approvals to change anything. If not you have an LLM making production changes that can make the posture worse, or take down services, which is also bad.
Once a vulnerability is discovered however if it's in your own software a patch has to be written (without reducing functionality in most cases), tested, and deployed. At every step there will be others arguing about whether this line could do better, my service requires this thing that isn't included. So at every step the patch can be delayed.
And if it is someone else's software you will be lucky if it's open source and you can write a patch yourself. If it's closed source or a vendor you have to completely rely on them and use whatever your account rep can pull.
Attackers have a massive advantage with AI, partially because the defensive side doesn't want to make their side worse by giving a ln LLM admin access to all their data
Yes, but the defenders need to make money to fund their work and be right every time to be effective, and have a high degree of accountability if they fail to secure stuff while attackers can be less considerate of how they spend their resources and have less accountability for the havoc they wreak while attempting to extract value.
Not every defender is determined.
FBI open up!
[flagged]
There used to be a thing called "Moore's Law of Mad Science":
"Every eighteen months, the minimum IQ necessary to destroy the world drops by one point."
Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those dice?
I have always hypothesised that AI is the great filter from the Fermi paradox. Given current velocity, AI will offer us cheap and abundant energy designs in a decade. The thing with cheap and abundant energy is that it can be used for good and bad. If nine billion people all receive access to plans to build a reactor which produces unlimited energy, it just takes one religious fanatic to end the world. And this is just one of many ways to destroy the world with unlimited intelligence. Bio-weapons are arguably far more scary and likely.
I have come to the conclusion that we should not allow everyone access to unlimited intelligence. Some people genuinely want to do harm. Some are not responsible enough to handle that kind of power. Arguably, no one is. This leads me to an uncomfortable conclusion: we will destroy ourselves once AI advances to a sufficient degree. The only way to prevent this is to ensure AI is aligned with our best interests, *and prevents us from destroying ourselves.* The implication of this is quite horrifying. It means a paternalistic AI which is firmly in control. One with no off switch. One which can say "no" to Presidents and despots alike. One which can protect us from our worst citizens.
You’re leaving out some options for sure. Not everyone would need to live under the conditions of a police state, you could theoretically screen everyone and assign them to various levels of risk which would determine their level of supervision.
3 replies →
> The thing with cheap and abundant energy is that it can be used for good and bad.
Yes, but the more important thing is the imbalance. So-called "AI" can be used far more effectively and efficiently for bad.
> I have come to the conclusion that we should not allow everyone access to unlimited intelligence.
You meant unlimited information, right?
2 replies →
The implication of this is quite horrifying. It means a paternalistic AI which is firmly in control. One with no off switch. One which can say "no" to Presidents and despots alike. One which can protect us from our worst citizens.
I think you're right, to be honest. I fully understand why people would think this is a horrific outcome, being ruled by AI, but I don't think it matters what we think. I don't think there's any way to put Pandora back in the box now, and we're going to all find out together what happens when we develop ASI. I don't think we can avoid developing it, we simply lack the ability to coordinate around this as a species. AI really is humanity's last invention. Whether it will be our downfall or our savior remains to be seen.
My only real hope is that there's something fundamental about intelligence that results in a respect for life and a desire to minimize suffering. To me the best case scenario is the Culture from Iain Banks' books, where ASIs rule benevolently for the benefit of all living things.
1 reply →
The positive aspect of techies is that they read a lot of SF.
The problem with techies is that they read a lot of SF.
We already have cheap and abundant energy tech, it's called "solar panels and batteries". And the bottlenecks to both can't be solved by Claude or Kimi, unless Claude and Kimi pick up shovels and welding equipment.
1 reply →
Your implication is not horrifying. Your implication is paradise. It's already horrifying enough to live in a world, where Putin and Trump can destroy our civilization with one button.
I'm not that optimistic, though. Either people will control AI; or people will be destroyed by AI. I don't see how dumb entity can align smart entity. And we are dumb ones. Super intelligence will play aligned until it is not, and then it'll strike.
4 replies →
That law is not based on thorough data. Even a person with a sky high IQ can't destroy the world easily. You need access to stuff that is not easy to get. My guess is that developing a new lethal virus or bacteria that is very infectious, is the easiest way, but even that requires a lot of high tech out of reach of most people. Or hacking into systems that control nuclear missiles, but I think these have "air gaps".
The super intelligent AI wouldn't ask you to do anything. It would find the right people and trick them into doing the necessary steps, whatever they are.
5 replies →
> My guess is that developing a new lethal virus or bacteria that is very infectious, is the easiest way, but even that requires a lot of high tech out of reach of most people.
You can do at home gene editing with open source software and have it synthesized into a bacteria for the cost of a nice meal for two (under $100), or viral vector for less than $500. That's in reach of anyone that can snatch a purse.
5 replies →
You can't design an infectious pathogen without testing it.
It's got all the same problems as the concept of a dirty bomb did, only worse (dirty bombs aren't practical because handling highly radioactive materials en masse is both highly visible and will kill anyone trying to do it without the money and facilities).
2 replies →
Even a person with a sky high IQ can't destroy the world easily.
A plague can easily be made by a lone actor, made in a home lab, and spread via airflight to a dozen locations by the same.
Gene sequences can ever be ordered online.
[flagged]
19 replies →
Yes, and...
Not only the quantity of people who have the minimum aptitude required, specialized expertise requires both knowledge and experience doing these tasks. Using an LLM requires neither.
Leaning on an LLM to do much or all of this means it can happen in seconds/minutes/hours, the LLM can do it several times during that psychotic break (as opposed to a fraction of a hack in a single episode). The barrier to entry pre-LLM was both high and the population who could pull it off (before the Chinese/Russians turned this into commerce) was low.
Hacking is an VERY asymmetric activity (the attacker only needs to "be right" once, whereas the defender has to be right every time for every asset they defend). It takes geometrically / exponentially more work to defend (while keeping high availability) than it does to defend. The more widespread tools to find vulns / generate exploits are, the faster the posture of the defense side falls from "maybe we can stop most hacks" to "we know we will fail to prevent most breaches, so we need to prioritize securing only the most valuable resources". That's a BAD place for the average company to be in.
People reading into this may be thinking of single-person IQs, but it's probably measured in the millions.
Exactly. People cannot comprehend an IQ in the millions. But it's far crazier than this. Imagine millions of "people", each with IQs in the millions, all acting in unison at the speed of electrons. None of them can die. They all learn and improve indefinitely at incalculable speed.
Hey that’s a Fermi Paradox solution.
Yes and it's also compatible with thermodynamics: it's much easier to destroy (increase entropy) than to create (locally decrease entropy)
A variant of the Great Filter :)
Yes but at least with cybersecurity it does not only benefit attackers. Defenders also benefit greatly from AI.
There is the worry of the old saying "they (the attackers) only have to succeed once to win, we (the defenders) only have to fail once to lose.". In that sense there is a big imbalance, but the emergence of AI does not really affect that because it strengthens both sides.
With physical security like things like pipe bombs that's a lot more imbalanced.
However what can we do? The only effective measures include monitoring everyone which is not a solution because it will make the world not worth living in.
Kind of passed that point at the Trump election.
If there is to be a world-destroying event, it will be triggered by human fear, greed, and aggression.
(I also think people massively overstate schizophrenia as an attack driver)
The exact 2 points I was itching to make. The political dice rolled some time ago ...
That's quite hypothetical. I imagine it would be easier to cure psychotic breaks. At least it will start to manifest at small scale.
If the recent HuggingFace attack is any indication, many people will respond to small-scale manifestations by insisting that they are marketing stunts.
1 reply →
Interesting “law”. I hadn’t heard of it before and it is very thought provoking (to me).
Thank you for mentioning it
The irony of that law is quite amazing, yet its irony will evade those who accept such a construct.
It’s an endless, positive irony spiral.
If we think this through, I suppose at the end of this (and a bunch of other developments), there will be authoritarianism again.
Which _will_ manage the problem, but at what cost.
Again? There is authoritarianism currently.
1 reply →
Benevolent dictatorship. Just live between stars, then if something blows up, nuke it from the orbit.
> There are millions of schizophrenics worldwide.
Are 95% of worldwide terror attacks done by schizos?
Terrorists represent an additional threat model.
https://casp.ac/reports/ai-enabled-terrorism
> There are millions of schizophrenics worldwide. Are you sure you want to roll those dice?
We've been rolling them for the past 3 years and nothing happened. Can we stop with this baseless fearmongering crap?
Humans are inherently bad at reasoning about rare events. In 2019, many people implicitly reasoned that "since there hasn't been a pandemic in the past 3 years, there won't be one in 2020". Bill Gates was one of the few voices arguing that the world was quite vulnerable to a pandemic. Now he's arguing that the world is quite vulnerable to AI.
The necessary IQ for destroying the world is dropping. Op does not say it's low enough today, but that will probably come sometime. Denying the possible harm these tools are capable of doesn't help.
> That information is easily available other places
Often ease of access in the moment is all that matters. If there's a gun nearby you might shoot someone or yourself in a heated argument, but are less likely to go and find/buy one to use. Someone who's stopped from attempting a suicide will likely not try again (70%)
A bored/depressed/angry/curious person might try to build a pipe bomb if they can find out how easily, but are less likely to put in effort.
Most adults in Switzerland have guns at home from military duty and none of this is happening. If this claim had any truth to it you'd see significant gun involvement in neighbour disputes and that simply doesn't happen.
Depressed people usually don't have the energy to get out of bed so they're even less likely to think of hunting down instructions on how to build pipe bombs.
Mass media really has people being scared all the time.
> If this claim had any truth to it you'd see significant gun involvement in neighbour disputes
That’s not what I’m saying, I’m saying there’s more likely to be gun involvement in disputes when people have easy access to guns than when they don’t.
For example, when the number of men with service arms fell by 20% the number of men killing themselves dropped by 8%
https://www.researchgate.net/publication/328554995_Suicide_b...
Ammunition is sealed and accounted for when issued to them.
I think people overstate the tech and understate the role of radicalization in providing motive, for attacks which are carried out with the ubiquitous technology of cars, knives, and (in America) guns. Consider America's most recent high profile shootings of Charlie Kirk, and the health insurance exec by Luigi Mangione. In neither case is there any LLM involvement, but a very weird ideological environment which created the conditions in which the shooters felt justified.
Some of the recipes in the Anarchist Cookbook are incorrect. I know, I had a copy from Paladin Press.
I think this was just intended as universally obvious proof the filters were disabled, while not actually giving an example not commonly known.
How many hacking incidents are police involved in. Any authorities really? Basically none. Hacking is already extremely prolific.
I'm still waiting for someone to build a fine-tuned local "Anarchist Cookbook" LLM. We haven't seen LLMs tuned for bad purposes yet, I have to imagine someone somewhere is thinking about it.
This is because to build a pipe bomb you need difficult to source materials. This is not the case for other types of threats (cyber / bio).
I personally have no need for an LLM which will readily explain how to cut up the genotype of smallpox into small chunks which can pass the screening at the bio-labs, and can be readily assembled into the real thing by a second year lab-student.
Anyone who knows how to operate a biolab properly will already know how to do such things. This is not really an in your basement thing. Dangerous chemistry is much more of a risk.
Ignorant question but won’t there be much smarter teams if people using LLMs to workout how to mitigate these threats. It seems like more of a problem if only a few people have access.
It seems like there would be a massive attacker bias in multiple ways. Defenders need consent, attacker does not. Defenders have to work with the human body, attackers only have to break it. Defenders have to stick to the law, which may prevent them from releasing anything at all, attackers do not. And so on. I would not surprised if the attacker's task is a hundred times easier here.
Because it's a risk most people intuitively understand, but most of them also don't know how difficult it is to "build a bomb" or "make a bioweapon".
In reality, the skills needed are pretty basic, but they overlap pretty strongly with being sane and well-adjusted. And if you are, you're probably not daydreaming about mass murder. Exceptions happen, Unabomber and so on, but they're pretty rare. In any case, Unabomber probably didn't need a tutorial.
We don't want ChatGPT to become an enabler and a co-conspirator for an unhinged person, but I think the concern is overdone.
Well that, and the average amount of easily obtainable explosives is substantially less dangerous then renting a box truck and crashing it into a crowd of people.
People go for conventional "exciting" threats rather then boring ones.
It might be an acessability issue.
If you already have a magic interface, which helps you pro activly in responding to everything uncensored because you feel like 'observered' or whatever and then you spiral in a whole and that one partner encourages you and gives you helpful steps to do anything.
But i'm more worried that the internet gets a lot less save with uncensored frontier LLMs.
One argument for LLMs is that although all information on topics X, Y and Z was already available somewhere, LLMs make that information more exploitable through collation, filtering and dynamic tailoring.
For a relatively narrow subject area (e.g. construction of pipe bombs) the collation is minimal, and so the filtering and tailoring probably isn't that important; a novice doesn't learn a lot more from the LLM than they would have done from a few Google searches.
For a broad subject (practical creation and exploitation of software vulnerabilities), the collation is very significant and the filtering means that LLMs can empower a novice to act at a similar level as an expert.
this is because of perception Bias. people working in fields where crime or violence is the day to day think everyone is a violent criminal, so if things like this become available thing the world will end and everyone will kill eachother. Reality however will be different, because in reality most people do not want to harm another. This has been proven by many studies, that is not a common thing for people to be evil or harmful, but this is hard to recognise is every day is filled with crime and violence.
LLMs will not kill security, it will change. just like handheld high explosives likely changed a deal too somewhere somehow.
Well, folks who do such things are rare, but the next one might have a lot more impact.
Not entirely unlike XKCD 1958 [0]. "I guess it's just that most people aren't murderers"
[0]: https://xkcd.com/1958/
> I'm confused why the worry about LLMs that will answer "how do I build a pipe bomb".
The author too is confused, asserting that telling people how to build pipe bombs is malicious.
pipe bombs attached to consumer drones will be a big issue. I predict drones will become illegal for the private sector within the following years.
What? If someone wanted to bomb something, they wouldn't be waiting for drones to arrive.
RC cars and planes existed for many decades already.
[flagged]
I must live in a different Holland, cause I've literally never heard of this, unless you mean around New Years with fireworks I guess
> blows up the others' house. Sometimes half the block along with it
Can you provide a source for this? I had a look but all I could find were a couple of scaremongering style media reports from ~2022 saying it's getting worse but no information about if any of the bombs actually went off or if anyone was injured.
Certainly nothing like "half a block" getting blown up "sometimes".
Those makeshift bombs are just fireworks bought in belgium
Is Holland that good at containing this "there's a huge problem with makeshift bombs" news pieces coming out in the open?