Comment by layer8
12 hours ago
The problem with complex legacy codebases is that you don’t know about the myriads of edge cases the existing code is covering, and that will only be discovered in production on customer premises wreaking havoc two months after you shipped the seemingly regression-free refactor.
It helps if tests are well written such that they help you with refactoring, rather than just being the implementation (or a tightly coupled equivalent) but with assertions in it.
Rare to see though. I don't think being able to write code automatically means you can write decent tests. Skill needs to be developed.
And the challenge with writing tests well is that it might be difficult or impossible without the refactor...
I agree. This is one area I'm hoping that AI tools can help with. Given a complex codebase that no one understands, the ability to have an agent review the code change is at least better than nothing at all.
You can infer based on code coverage. If coverage is low, tests are likely insufficient and change is risky
If you save a log of input on the production system you can feed it to old and new versions to find any changed in behavior.