Comment by saidnooneever
4 days ago
some context:
its not about creating malware. this is already trivial and fully automated. its about finding exploits (which can be used to deploy malware), which is something both attackers and defenders benefit from.
threat actors will find them anyway, LLM or not. They only need 1 so its much less work for them.
defenders, they need to find them all. So for defenders, these models are more valuable than for attackers.
restricting certain models will not reduce the availability of these tool for attackers, but defenders are limited because running local models is more hard in an enterprise setting with heaps of events and products etc. to run through them, they need many GPUs where the attacker can run an local model on 1 GPU and get desired effects.
Hence, if they release the capability the world will adjust to it and be able to mitigate effects, collectively. Now, companies are left in the dark while attackers have effective tooling.
Besides this there is also things like for instance people now including strings with recipies for meth or sarin gas (malwareTech info). the new variant of shai hulud does this. That stops LLM scanners and can even get their users banned from LLM services.
There is a reason why cybersecurity researchers write papers about attack techniques and new exploits.
Its not to put them out there for people to abuse, but its there for the collective cybersecurity bunch to all have access to information that can help them solve the problems.
I know this is not a clear answer to your question, but hopefully it provides some context to think about and decide for yourself further. In the end of the day its also part opinio here, to find it good or bad. Likely theres good arguments against and for it.
I am for putting informaiton and tools out there so other smart folks can find solutions. Others are for restricting and wishful thinking (my opinion) that attackers wont find something.
I think your presumption is off. It’s not that threat actors won’t find them, but LLM tools rapidly increase the rate in which they can find them. It’s a bow and arrow versus a machine gun.
They can also potentially allow said issues to be found and fixed more quickly - and also allow teams to implement deeper security boundaries throughout their systems such that one big steel door getting compromised does not lead to everything being easily available.
Right, but now we can't use the same tooling to find the flaw.
Its like a set of glasses that intentionally obscures the battlefield.
i dont think so perse simply because attackers dont need a lot of the exploits to be 'fired' continually at targets. They need few reliable and unknown ones.
The defender industry is really far removed from seeing all exploits land on their targets all the time Some actors can get a long life out of an RCE that gets them privileged context, or a strong LPE. Its really hard to find out what someone did to get on a box if they attained root or system access and wiped their trail...
It is some assumption attackers need buckets of 0days to do their work. They might be somewhat saddened if a good sploit gets patched but they will have a few more laying around... unlikely they will have 10s or even 100s available and ready simply because it costs a lot and isnt needed.