Comment by dabinat
14 hours ago
It’s interesting how AI may both raise and lower the quality of software. It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal. Time is much more important for a human developer with a salary.
Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.
This is where I believe strong typing (like, Haskell-strong or stronger) and functional programming in general will be a win. The confidence I have that my fixes are localised when fixing Haskell code is infinitely stronger than fixing even Java, not speak about C, code.
Haskell's type system would not easily prevent this bug. It's not good at numeric/logic issues like that. When people say "Haskell makes it impossible to write bugs" they mean "Haskell has enums" (ADTs).
11 replies →
You haven't mentioned the dynamic typed languages that I believe should die -- Python and Javascript. The only good use case for dynamic typing is notebooks (niche of R lang) where you're throwing out the code you just wrote after getting the result you wanted from it.
Imo, formal methods like more expressive/stricter type systems are key to making LLM generated code successful. Of course models will get better, but trusting the output will become much easier with a type system that proves more properties.
What's stronger than Haskell?
6 replies →
Even Lean 4 strong typing
If finding the bugs with LLMs is easy. Then making sure it doesn't break something else is just LLMs finding no bugs. Easy.
That hasn’t been that bad. My real issue has been the time sink involved in following along with the maintainer and jumper through their hoops. Even after I demonstrate a flaw and a potential fix. My schedule is just so busy I need to pencil in time to deal with them.
The missing part of this is that verifying the bug with LLMs is also easy, and so is adversarially reviewing the proposed fix with LLMs.
The only thing left for you to do should be directional decisions. The LLMs should pause and rope you in if the fix involves directional/invariant changes.
No one can keep up with the volume of code AI produces.
We wont stop using AI.
We will use AI to check AI.
Of course this is crazy, but it will also unlock pretty insane scaling and productivity and ultimately we will manage it on either end via requirements and tests.
> it will also unlock pretty insane scaling and productivity
Insane scaling of bloat, bugs, and technical debt I'd say.
> We will manage it on either end via requirements and tests
It is so crazy that this is being touted as a sane strategy. When I was a much worse programmer, I tried to write a big complicated string manipulation function to take two types of scripts in a language and add diacritics. I had the requirements very clear. I had the tests very clearly with all the edge cases. But I didn't have a good and clear picture of how to attack the problem which was quite novel for me. As I got closer to passing all the tests it got exponentially more unruly and confusing. And nearing the end I was frantically changing little bits here and there wincing and praying and hoping the tests would pass. "Please work! Come on!" Then when I got close enough, I could never ever think about touching that mess again.
I was a below average programmer then throwing myself at some novel problem I didn't understand. Throwing LLMs that produce below average code at novel problems and relying on tests and requirements is not where we want to go to make real progress.
(Years later after much learning and coding myself I was able to redo the function in a totally different way. This time I actually understood how to attack the strange problem and made something clean, clear, and robust that just worked. The tests then become a secondary guardrail, not the main force of correction.)
We are seeing such a massive regression from what we've learned over the years of CS.
5 replies →
It's mostly (not entirely, but mostly) finding security issues in old human-written code. It'll eventually start running out of those.
From that standpoint, it's not a crazy setup security-wise. Maybe still crazy for development.
2 replies →
You're suggesting that LLMs get better at fixing bugs/vulnerabilities, but at the same time stop getting better at finding them? What if this difference is inherent and essential?
4 replies →
Volume..... <sigh>
It used to be considered a quality of good code that there would be less code, not more.
Some people always tryin to get the highscore on golf.
1 reply →
In fairness at root this has been going on for awhile. No one can keep up with the volume of machine code that modern more abstracted codebases produce.
We didn't stop using syntactic programming languages we used code to check code.
Not sure it's really crazy at all. It's been an abstraction for programmers probably since we stopped soldering transistors to each other.
2 replies →
[flagged]
[flagged]
> LLMs do find bugs, do save time
They find bugs but whether they save time is nowhere near as clear as you try to insinuate here.
3 replies →
In my experience, there are two ways to use AI: speed or quality. Speed is where you give the AI a task to do and you review it; quality is where you write the code yourself and you get AI to review it. Both are valid for different situations.
My plan for bigger things is mostly:
Generate multiple solutions- they do not to work 100% correctly. And than I check which I would prefer. Which is more to our applications taste.
And than I would take the vibe output as a kind of a ‚plan‘ which I use to implement but not follow 100% and at the end I take my solution and review it. I gain speed with that because I often can quickly see the pros and cons of a solution way better than when I would manually do it and hang on a major roadblock and also I even see such roadblocks in the vibe output - it’s mostly the part with an unnecessary amount of new code that looks nonsensical.
Using a LLM whose output is slowed to the rate of a human programmer as a pair programming partner is a very interesting experience.
Having worked in a few vibe coded codebases over the last few years I can safely say that AI is not raising the quality of anything.
> It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal.
No big deal? It’s not like it’s free… tokens cost money.
Often rounds to free compared to human costs.
When talking about LLM tokens the cost is almost always being implicitly compared to very expensive human developer time.
I don't care if you call it an over-engineered looping machine or what, there are concrete benefits to using LLMs for this. They work faster than developing your own looping algorithm and more often produce useful results than not.
It's not even like fuzzers are valuable because of the process they use specifically either; the value is that they produce a concrete input that you can use as a reproducible test case at that point. The value could be produced by gazing into a crystal ball for all I care, as long as I can use what it gives me to reproduce a bug.
But what's your expectation of the net?
I dislike AI, but if AI finds real bugs then this is in my opinion objectively a positive thing. Of course the question is what constitutes a real bug.
Unfiltered models will help build exploits for the bugs they find, so there is some means of measuring their efficacy.
If you're just talking about security bugs.
There are also non security bugs that don't have exploits but just make the user experience worse.
A.I. is useful for this. But it would be even more useful if all new code were written in Rust or some other memory-safe language.
A.I. could also be used to port C/C++ codebases to Rust, which isn't economically feasible at the moment.
AI will have plenty of security bugs left to find in Rust codebases.
I mean I get the sentiment but Rust won't save you against division by zero, it'll just panic at runtime like every other language.
1 reply →