Comment by 33a
3 months ago
We use a mix of static analysis and AI. Flagged packages are escalated to a human review team. If we catch a malicious package, we notify our users, block installation and report them to the upstream package registries. Suspected malicious packages that have not yet been reviewed by a human are blocked for our users, but we don't try to get them removed until after they have been triaged by a human.
In this incident, we detected the packages quickly, reported them, and they were taken down shortly after. Given how high profile the attack was we also published an analysis soon after, as did others in the ecosystem.
We try to be transparent with how Socket work. We've published the details of our systems in several papers, and I've also given a few talks on how our malware scanner works at various conferences:
So, from what I understand from your paper, you're using ChatGPT with careful prompts?
You rely on LLMs riddled with hallucinations for malware detection?
I'm not exactly pro-AI, but even I can see that their system clearly works well in this case. If you tune the model to favour false positives, with a human review step (that's quick), I can image your response time being cut from days to hours (and your customers getting their updates that much faster).
You are assuming that they build their own models.
He literally said "Flagged packages are escalated to a human review team." in the second sentence. Wtf is the problem here?
What about packages that are not "flagged"? There could be hallucinations when deciding to (or not) "flag packages".
4 replies →
> We use a mix of static analysis and AI. Flagged packages are escalated to a human review team.
“Chat, I have reading comprehension problems. How do I fix it?”
Reading comprehension problems can often be caught with some static analysis combined with AI.
"LLM bad"
Very insightful.