Comment by InTheArena
4 days ago
This is completely infeasible in the age of mythos. The reality is that the velocity is just not going to feasible from a security PoV without leveraging these tools.
4 days ago
This is completely infeasible in the age of mythos. The reality is that the velocity is just not going to feasible from a security PoV without leveraging these tools.
Analysing codebases with LLMs to find security vulnerabilities is completely unrelated to committing code generated with LLMs
It's a fair comparison. There's a fair amount of plausible-sounding bullshit being peddled as a transparent advertisement for an ai-driven "code security" firm.
and how do you propose fixing the hundreds, if not thousands, of valid, impactful security bugs that frontier models will find?
That seems like an unfounded assumption. Why should one assume that Git Annex has hundreds or thousands of critical, exploitable security vulnerabilities?
2 replies →
If you can't fix them without LLMs, then you can't fix them. You probably shouldn't be trusted with maintaining the codebase in the first place.
13 replies →
In ten years we'll look at human written code like the unreliable garbage it is, and never rely on anything that wasn't at least seriously looked over by an LLM. It won't be even close.
Alternate take: in ten years we'll be pulling our hair out cursing at the world over how we could possibly accept "10k lines added, 8k lines removed" as the normal everyday churn of software development. We'll curse the morons who gave up understanding our own code.
> never rely on anything that wasn't at least seriously looked over by an LLM
I can imagine LLMs becoming a mainstay, but what you are describing isn't wholly different from sufficiently advanced static code analysis - where you'd want more determinism than most LLMs normally provide.
The problem is that such a thing might take a decade and billions of dollars of investments to create per-language (e.g. actually useful code analysis for Java, for Spring Boot, for processing and validating form data, and DB schemas and document processing and rendering reports etc., literal domain checks for anything and everything that is common across various enterprises) so nobody wants to do that, so it's easier to throw LLMs at it and call it good enough.
I remember back in the pre-2023 days where SonarQube was a big deal for Java static analysis, and I let it rip across an entire 120k line project at one point upon which it found something like seven issues, out of which only one or two were actual bugs. It was almost entirely useless. I think even Qwen would've done leagues better today.
Most bugs are far too nuanced to be caught by static analysis imo, you do need to actually understand what's going on in the program, the intent, the environment, etc. instead of blindly verifying if everything technically checks out, compilers already do a perfect job at that.
1 reply →
Yes, the same way 10 years from now-10years, we'd all be looking back at how insane it was for people to drive cars.
Man do I enjoy my totally real full self driving.
I find this attitude really weird. I just checked and I have 8,793 miles driven on my car and of that I'd say ~8750 of them were done by fsd (self driving). These days most of my interventions are to pick a different parking spot, and I can't remember the last time I had a serious disengagement, it's always just me wanting to drive a different route, park in a different location, or sometimes to handle things like car washes and the like.
For me, for all intents and purposes, self driving is here today.
You're being sarcastic, but I do enjoy it. I just took a Waymo recently and it was thrilling, it felt great to feel the wind and the sun, listen to music, and get where I was going without having to drive there. I still like driving, obviously, but being able to decide one or the other is wonderful.
In ten years we'll be drowning in subtle bugs introduced by the unreliable garbage that is machine-generated code, and the industry will hopefully have learned to never rely on anything that wasn't at least seriously looked over by an actual thinking human being that understands it. We'll look back on our youthful idealism and cultish faith in this new technology with embarrassment.
I'm a bit more pessimistic.
> In ten years we'll be drowning in subtle bugs introduced by the unreliable garbage that is machine-generated code
Yes. But replace
> and the industry will hopefully have learned to never rely on anything that wasn't at least seriously looked over by an actual thinking human being that understands it.
with: "and the industry will throw even more LLMs at the problem, producing an even deeper soup of garbage that in some cases perform a tiny bit better, and when things do break it's always the fault of someone else. So for example a bank denies you a mortgage or an insurance company fails to process your claim, and you are almost certain that it's due to some slopcode somewhere, but you have to suck it up because the world has become accustomed that this is just how things are done."
It's a way of breaking computers that I'd never thought I'd see. We're wilfully taking the one cool thing about computers – them exactly interpreting instructions carefully crafted by humans to do exactly the right thing – with bucketloads of vibes that hopefully mostly do the right thing most of the time ("the tests pass"). What the hell are we doing.
In ten years, crypto will be the world’s primary currency. Oh look, stupid hot takes aren’t that hard to have after all.