Comment by ramijames
1 day ago
I'm curious. Do you attribute this to weak and/or incomplete tests? How granular should tests be to have complete coverage so that an AI won't create a converted codebase that "passes tests" but is still functionally inaccurate?
There is no such thing as a complete test suite, there will always be some possible bug that it doesn't catch.
In particular, if you put an LLM in an automated loop of "this test fails, please fix it", there is a pretty good chance that it will simply special case all of the tests, possibly in some contrived way that makes it not at all obvious when you read the code.
This is what I have observed as well. Been through many debates about the "perfect plan" and "perfect tests" fallacies.
Maybe a way of looking at it, to understand the nature of the issue. Have a LLM translate a novel from English to Spanish. Of course it can do that translation at speeds that no human could (score a point for AI). But how good is the Spanish translation? Is the quality better than what humans could do? Wouldn't those who are not fluent in Spanish be more easily impressed?
We then can do all kinds of configuration setups and tests, but how do we know the Spanish was translated perfectly, without a massive detail review (and being already truly bilingual in both English and Spanish)?
As is the usual case in the pursuit of perfection (which nothing in nature ever seems to be), there is going to be mistakes, costs (worth it?), and gray areas. It would be foolhardy for us not to suspect or pass it off as otherwise.
> how do we know the Spanish was translated perfectly, without a massive detail review
You can't. I think that's a large part of why LLMs have caught on much better with programmers: they have ways of making the computer check its own work.
Checking a document is still a laborious manual task. And completely unfulfilling.
1 reply →
This is where fuzzing would be useful. We have an at-least-parity-bug-level oracle with the reference PostgreSQL implementation. Just build a generator of queries (both invalid and valid) and ensure the output matches. The yardstick is how many log10(queries) it can go on average before a discrepancy is found.
It’s 2026, so another basic technique that every team should add to their testing strategy (in addition to proven techniques like fuzzing) is agentic user simulation. Set up an AI agent with access to the product, and prompt it to use the thing in hundreds of realistic use cases, and to report any possible bugs. It will catch a lot of ‘blind spot’ bugs that were previously things that were only caught by humans.
That's the million dollar question. Do you/we/us have tests that cover everything which covers QA as well? If such a mythical beast exists, maybe from remnants of ye olde TDD past and hasn't been modified as such.. then maybe this would be possible to do as such.
I've always played the game that tests are only as useful as you have resources to attribute to them. Mostly all modern development is a compromise between new features and stuff that supports those new features (tests included, but also reviews, code maintenance, docs, etc.).
If LLMs can be utilized to quickly make deep testing possible, I think that's probably a net-positive.
Unit tests don't test for branch coverage.
That's the culprit, because LLMs tend to forget and remove a lot of branch logic in these kinds of tasks. If unit tests don't cover these specific if/elseif/else cases, then they'll just disappear.
They'll also disappear if the LLM is allowed to modify the unit tests, because they sure like to cheat their way around into greenlit test suites. The agentic environment must disallow write access to the unit test files for the agent that writes the code.
If you implement that in your tools, you'll see quickly how the models will try to rewrite the unit tests at all cost, no matter what kind of prompting you've done. Tool policies are the only boundary to successfully guarantee this.
Source: Am building my own agentic environment because of that behavior