Comment by tptacek
2 months ago
Why does it work well now, after 20 years of this kind of tooling being next to useless? Do you work in this space? How much about how bad SAST tools are do I need to explain?
2 months ago
Why does it work well now, after 20 years of this kind of tooling being next to useless? Do you work in this space? How much about how bad SAST tools are do I need to explain?
Maybe it's unleveraged potential, I don't know. I am also not entirely convinced that they're next to useless. Sanitizers, for example, are excellent for mitigating all sorts of security issues. Those are traditional static analysis tools (that, by the way, fit the arrangement I described of using these reports as nucleation sites for LLM triage).
I did walked you through how I would do it. Would you change your response if I said I work in this space? It seems like an irrelevant point in this discussion.
You don't need to explain anything. This is on a flagged thread, obscure and unseen. I'm actually surprised by how invested you are in this apparently irrelevant matter.
I'm a software security person! This is not irrelevant to me.
In summary: the existing program analysis tooling in this space has been ineffective for decades, despite hundreds of millions of dollars invested in the tooling. If it is effective now, that strongly indicates that the LLM component of it isn't irrelevant; nothing else in the field has changed.
Note that everybody in this story concedes the LLM involvement. The only person who isn't is you, and you're not actually involved. (I'm not either, but I'm agreeing with --- checks again --- everybody involved in the story).
I concede the LLM involvement. But I want to be more specific in the description of the role it plays in the solution.
If it is a central role, then there is nothing to loose from describing it better. That's why this feels so strange. You disagree with me, but you don't present an arrangement in which the LLM plays a role different to what I described. In fact, no one here did. It's like you're not disagreeing with me, but trying to make me stop describing how to achieve a similar quality system out of free pieces.
8 replies →