Comment by ofjcihen
3 days ago
No, it can’t. Partially stems from the garbage the models were trained on.
Example anecdata but since we started having our devs heavily use agents we’ve had a resurgence of mostly dead vulnerabilities such as RCEs (CVE in 2019 for example) as well as a plethora of injection issues.
When asked how these made it in devs are responding with “I asked the LLM and it said it was secure. I even typed MAKE IT SECURE!”
If you don’t sufficiently understand something enough then you don’t know enough to call bs. In cases like this it doesn’t matter how many times the agent iterates.
To add to this: I’ve never been gaslighted more convincingly than by an LLM, ever. The arguments they make look so convincing. They can even naturally address specific questions and counter-arguments, while being completely wrong. This is particularly bad with security and crypto, which generally isn’t verified through testing (which only proves the presence of function, not the absence).