Comment by varispeed
20 days ago
AI also quickly goes off the rails, even the Opus 2.6 I am testing today. The proposed code is very much rubbish, but it passes the tests. It wouldn't pass skilled human review. Worst thing is that if you let it, it will just grow tech debt on top of tech debt.
The code itself does not matter. If the tests pass, and the tests are good, then who cares? AI will be maintaining the code.
Next iterations of models will have to deal with that code, and it would be harder and harder to fix bugs and introduce features without triggering or introducing more defects.
Biological evolution overcomes this by running thousands and millions of variations in parallel, and letting the more defective ones to crash and die. In software ecosystems, we can't afford such a luxury.
An example: it had a complete interface to a hash map. The task was to delete elements. Instead of using the hash map API, it iterated through the entire underlying array to remove a single entry. The expected solution was O(1), but it implemented O(n). These decisions compound. The software may technically work, but the user experience suffers.
If you have particular performance requirements like that, then include them. Test for them. You still don’t have to actually look at the code. Either the software meets expectations or it doesn’t, and keep having AI work at it until you’re satisfied.
3 replies →
That's assuming no human would ever go near the code, and that over time it's not getting out of hand (inference time, token limits are all a thing), and that anti-patterns don't get to where the code is a logical mess which produces bugs through a webbing of specific behaviors instead of proper architecture.
However I guess that at least some of that can be mitigated by distilling out a system description and then running agents again to refactor the entire thing.
> However I guess that at least some of that can be mitigated by distilling out a system description and then running agents again to refactor the entire thing.
The problem with this is that the code is the spec. There are 1000 times more decisions made in the implementation details than are ever going to be recorded in a test suite or a spec.
The only way for that to work differently is if the spec is as complex as the code and at that level what’s the point.
With what you’re describing, every time you regenerate the whole thing you’re going to get different behavior, which is just madness.
2 replies →
And that is the right assumption. Why would any humans need (or even want) to look at code any more? That’s like saying you want to go manually inspect the oil refinery every time you fill your car up with gas. Absurd.
1 reply →
Tests don't cover everything. Performance? Edge cases? Optimization of resource usage are not tipically covered by tests.
Humans not caring about performance is so common we have Wirth's law
But now the clankers are coming for our jobs suddenly we're optimization specialists
3 replies →