Is it a stretch to imagine a compromised LLM writing compromising code? I've been operating under the assumption that LLM code might be bad and not working, but I hadn't thought about it compromising a system.
Yes and no. We used to have rampant script kiddies back in the '90s and early 2000s. After 2005-ish, well maybe 2010, most systems became adequate enough to not trust users' input, bug bounties, security as a separate role, etc, etc.
It would take at least some knowledge to hack, not just a random script from a forum.
The attack doesn’t use AI, but an AI could use the attack. (and the researchers definitely used AI to write the paper!)
Is it a stretch to imagine a compromised LLM writing compromising code? I've been operating under the assumption that LLM code might be bad and not working, but I hadn't thought about it compromising a system.
huh?
AI could enable script kiddies to pull this off.
Script kiddies have always been a problem you have to defend against. This is nothing new.
Yes and no. We used to have rampant script kiddies back in the '90s and early 2000s. After 2005-ish, well maybe 2010, most systems became adequate enough to not trust users' input, bug bounties, security as a separate role, etc, etc.
It would take at least some knowledge to hack, not just a random script from a forum.
Now, with LLMs, it's the '90s all over again.
1 reply →
Script kiddies will soon have capabilities that nation states once upon a time could only dream of.
1 reply →